From 8f3b30387b44f3c171305ccfa80ba372267d01d3 Mon Sep 17 00:00:00 2001 From: Paul Keen <125715+pftg@users.noreply.github.com> Date: Mon, 24 Aug 2026 07:52:15 +0200 Subject: [PATCH 1/5] docs: 2.0.0 release readiness Audit of everything a 2.0.0 final would ship, and the fixes that did not need lib/ changes. CHANGELOG - A v2.0.0 entry written for someone upgrading from 1.15.1, not a diff of the betas. What to change (the version), what they will see (exact warning text), the five things that can actually break, and what 2.1 removes. Every claim verified against a real install; the beta sections stay as history. Version consistency - README, docs/UPGRADING.md: no more "beta"/"alpha"/"experiment" framing and no beta pins. Gemfile examples say `~> 2.0`. - Gem name: `capybara-screenshot-diff` is the one we tell people to install; `snap_diff-capybara` is a reserved identical mirror. Stated once in the README with the dual-install consequence, applied everywhere else. - Stale "3.0" references in the Rakefile and docs/architecture.md are now 2.1 (#247 fixed the user docs and missed these). Corrections to claims that were not true - docs/drivers.md promised that everything 2.1 removes "warns once per process naming 2.1". `driver: :auto` is silent whenever ruby-vips is present, and the `driver:` setting itself never warns at all even though 2.1 deletes it (`NoMethodError`). Both are now written down as silent, in drivers.md and UPGRADING.md, since a note is the only notice they can get. - README called ruby-vips "Optional". With neither ruby-vips nor chunky_png installed, comparisons raise `Wrong adapter nil. Available adapters: []`. Says so now. - Setup examples no longer teach `driver: :vips`, a line users have to delete for 2.1. Gem hygiene - gemspec: summary/description that describe what the gem does, the rubygems metadata links (source, changelog, bug tracker, docs), and docs/docker-testing.md dropped from the package (it documents bin/dtest, which is not packaged). Dead bindir/executables removed -- the allow-list never matched exe/. - README's links to CONTRIBUTING.md and docker-testing.md are absolute, so they resolve from inside the gem too. - test/unit/gemspec_packaging_test.rb pins the packaged file list: both Bundler.require entry files present (this broke twice), consumer docs in, contributor docs and build files out, capybara the only runtime dependency. Verified it fails when an entry file is unpackaged. - *.gem is gitignored. Release process - The GitHub Release body linked to blob/main on a repo whose default branch is master -- 404 on every release so far. Links to the tag now. - docs/RELEASE_PREP.md was a stale v1.15.1 checklist. It is now a runbook for how releases actually happen: what the workflow does step by step, the trusted-publisher prerequisite for BOTH gem names, prereleases, post-release verification, and what to do when a run fails halfway. - CONTRIBUTING.md pointed at the wrong version.rb and recommended `rake release`, which publishes only one of the two gem names. Verified with real installs on ruby 4.0.6: 1.15.1 -> this master via path:, a canonical-names setup, `Bundler.require` under each gem name from the built .gem, and the dual-install guard with both gems installed. No lib/ changes. Version not bumped. --- .github/workflows/release.yml | 4 +- .gitignore | 1 + CHANGELOG.md | 145 ++++++++++++++++++++++++++++ CONTRIBUTING.md | 24 +++-- README.md | 21 ++-- Rakefile | 8 +- capybara-screenshot-diff.gemspec | 17 +++- docs/RELEASE_PREP.md | 97 +++++++++++++------ docs/UPGRADING.md | 42 +++++--- docs/architecture.md | 4 +- docs/drivers.md | 6 +- docs/snapdiff.md | 3 +- test/unit/gemspec_packaging_test.rb | 47 +++++++++ 13 files changed, 338 insertions(+), 81 deletions(-) create mode 100644 test/unit/gemspec_packaging_test.rb diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index da99c0dc..0c07cc4f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -90,8 +90,8 @@ jobs: body: | ## What's Changed - See [CHANGELOG.md](https://github.com/${{ github.repository }}/blob/main/CHANGELOG.md#v${{ github.event.inputs.version }}) for full details. + See [CHANGELOG.md](https://github.com/${{ github.repository }}/blob/v${{ github.event.inputs.version }}/CHANGELOG.md) for full details. - **Upgrade Guide:** [docs/UPGRADING.md](https://github.com/${{ github.repository }}/blob/main/docs/UPGRADING.md) + **Upgrade Guide:** [docs/UPGRADING.md](https://github.com/${{ github.repository }}/blob/v${{ github.event.inputs.version }}/docs/UPGRADING.md) env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.gitignore b/.gitignore index d2a6596e..ca07113a 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,7 @@ /gemfiles/*.lock /gems.locked /pkg/ +*.gem /spec/reports/ /tmp/ /vendor/sigs/ diff --git a/CHANGELOG.md b/CHANGELOG.md index ad54e1dc..8c39e31f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,151 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [v2.0.0] - unreleased + +**The transitional release.** Everything you run today keeps working. 2.0 makes +`SnapDiff` the canonical namespace, keeps the entire v1 API alongside it as +same-object aliases, and warns — once per process — about what **2.1** removes. +There is no 3.0; 2.1 is the cleanup. + +The sections below the divider are the prerelease notes (alpha1 → beta3) and are +kept as history. This entry is the one to read if you are coming from **1.15.1**. + +### Upgrading from 1.15.1: change the version, run your suite + +```ruby +gem "capybara-screenshot-diff", "~> 2.0" +``` + +That is the whole migration. `screenshot`, `assert_matches_screenshot` and +`capture_screenshot` are unchanged. Your `Capybara::Screenshot::Diff.configure` +block, every `Capybara::Screenshot.*` / `Capybara::Screenshot::Diff.*` setting, and +every legacy constant still resolve — to the *same objects* the new names resolve +to. Baselines are unchanged: capture, encoding, file naming and the `png` default are +the same code as 1.15.1, moved — 2.0 will not rewrite a baseline you already committed. + +Rolling back is a Gemfile edit: pin `"~> 1.15"` and `bundle update`. + +### What you will see in your test output + +One migration notice, the first time the process touches a v1 API: + +```text +[snap_diff deprecation] This process uses the v1 `Capybara::Screenshot*` / +`CapybaraScreenshotDiff*` API. It still works in 2.0 and is REMOVED in 2.1 -- +see docs/UPGRADING.md for the SnapDiff replacements. Silence with +`SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1. +(shown once per process) +``` + +Plus one line per *lazily shimmed* legacy constant you reference, naming your call +site: + +```text +[snap_diff deprecation] `Capybara::Screenshot::Diff::ImageCompare` is deprecated +(constant); use `SnapDiff::Comparison` instead. (called from test/test_helper.rb:12) +``` + +Requiring the gem, the DSL, settings accessors and the eagerly-defined constants +(the error classes, `::VERSION`, `Os`, `Region`, `Reporters::Default`, +`LOADED_DRIVERS`, `AVAILABLE_DRIVERS`) are **silent by design** — +[docs/UPGRADING.md](docs/UPGRADING.md#deprecation-warnings) lists exactly which +names warn and which do not. Silence everything with +`SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. + +### The five things that can actually break + +Everything else is source-compatible. These are not: + +1. **Error class names printed in output are now `SnapDiff::…`.** The class objects + are identical, so `rescue CapybaraScreenshotDiff::ExpectationNotMet` still + catches them — but a CI job that greps the *old* class name out of test output + needs updating. +2. **`defined?` / `const_defined?` on lazily shimmed legacy names returns + `false`/`nil`.** They resolve through `const_missing`, which those checks never + trigger. Move feature detection to the `SnapDiff::` name. Names in the + silent-by-design list are real constants and are unaffected. +3. **Reopening `module Capybara::Screenshot::Diff::Drivers`** (the historical + custom-driver monkey-patch) defines a fresh, empty module that shadows the shim. + Define custom drivers under `SnapDiff::Drivers` instead — and `BaseDriver` is a + mixin now: `class MyDriver < BaseDriver` becomes `include SnapDiff::Driver`. +4. **Stubbing `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` no longer works.** It + is an eager alias; the gem reads `SnapDiff::Drivers::AVAILABLE_DRIVERS`. Stubbing + the legacy name rebinds the alias only, so the test passes for the wrong reason. +5. **`SnapDiff::Config::MAPPING` is gone**, split into `SnapDiff::Config::SETTINGS` + (setting names) and the `@api private` `SnapDiff::LegacyShims::CONFIG_MAPPING`. + +### Act now: what 2.1 removes + +2.1 deletes the v1 namespace trees, the ChunkyPNG driver, `shift_distance_limit`, +and the whole driver abstraction — **libvips becomes the only backend**. Removing +public API in a minor is a deliberate departure from strict semver; publishing the +contract one release ahead is the mitigation. 2.0 warns once per process for each of: + +| You will hear about it when you… | Do this in 2.0 | +|---|---| +| select `driver: :chunky_png` | add `gem "ruby-vips"` and drop the option | +| run `driver: :auto` **without ruby-vips** — nothing in your setup says `chunky_png`, so this warning is the only sign 2.1 will break the process | install libvips + `ruby-vips` | +| set `shift_distance_limit` | use `median_filter_window_size`, `tolerance` or `color_distance_limit` | +| read `SnapDiff::Drivers.loaded` / `.available` | require `ruby-vips` instead of branching on a detected list | +| `include SnapDiff::Driver` in your own driver | nothing — custom drivers have no migration path | + +Two removals 2.0 cannot warn about, so they are written down instead: **`driver:` +as a setting goes away entirely** (`SnapDiff.config.driver = :vips` and +`Capybara::Screenshot::Diff.driver = :vips` raise `NoMethodError` on 2.1 — delete +the line, one backend needs no selection), and the legacy `LOADED_DRIVERS` / +`AVAILABLE_DRIVERS` constants are plain aliases with nothing to hook. + +### Added +- **`SnapDiff` is the canonical namespace** — the implementation lives in + `lib/snap_diff/`. Configuration (`SnapDiff::Config`, one storage behind every + settings surface), errors (`SnapDiff::Error` and friends), `SnapDiff::Region`, + `SnapDiff::Reporters::Default`, `SnapDiff.session`, `SnapDiff::Reporting.register`, + and integration requires under `snap_diff/integrations/…`. Start here for new + code: [docs/snapdiff.md](docs/snapdiff.md) +- **One consolidated config object** — all 27 settings on `SnapDiff.config`, via + `SnapDiff.configure { |config| … }`. Old and new surfaces share one storage, so a + write through either is visible through the other +- **`SnapDiff::Error` is the catch-all** the docs promise: every error the gem + raises inherits it, including `WindowSizeMismatchError` and `DualInstallError` +- **Deprecation warnings name your call site**, so migration is warning-driven + rather than grep-driven +- **Dual-install guard** — installing both `capybara-screenshot-diff` and + `snap_diff-capybara` raises `SnapDiff::DualInstallError` at require time instead of + silently loading files from whichever gem activated first +- **The gem is also published as + [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara)** — identical + content and version, matching this repository. `capybara-screenshot-diff` remains + the name to install; the mirror reserves the forward-looking one. **Install one, + never both.** +- New documentation shipped inside the gem: + [docs/snapdiff.md](docs/snapdiff.md) (canonical API) and + [docs/UPGRADING.md](docs/UPGRADING.md) (every renamed constant, which names warn, + rollback) + +### Changed +- **No `activesupport` at runtime.** 1.x required `active_support/core_ext/…` + without declaring the dependency, so a non-Rails install could fail to load. 2.0 + requires nothing beyond `capybara` +- The images-holder struct is now `SnapDiff::Comparison::Images`, ending the + two-classes-one-name collision with the comparator +- The packaged gem is an explicit allow-list — `lib/`, `docs/`, `README.md`, + `LICENSE.txt`, `CHANGELOG.md`. 1.x shipped `Rakefile`, `gems.rb` and the gemspec, + and omitted the README + +### Fixed +- Annotation color constants resolve under a bare `require "snap_diff"`; a differing + comparison previously raised `NameError` there +- `require "snap_diff/integrations/…"` loads the full `SnapDiff` surface, and + `gem "snap_diff-capybara"` works with `Bundler.require` +- Reporter failure warnings use one brand and name the failing reporter class + +### Unchanged +- Ruby 3.2+, Capybara `>= 2, < 4`, the `screenshot` / `assert_matches_screenshot` + DSL, every capture and comparison option, baseline file names and formats + +--- + ## [v2.0.0.beta3] - 2026-08-23 Fixes the canonical `SnapDiff` entry points, which were incomplete in beta2. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 581f276e..dc658049 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -152,17 +152,15 @@ Include: To release a new version: -1. Update the version number in [lib/capybara/screenshot/diff/version.rb](lib/capybara/screenshot/diff/version.rb) +1. Update the version number in [lib/snap_diff/version.rb](lib/snap_diff/version.rb) — the + only place it lives; the gemspec and the legacy + `lib/capybara/screenshot/diff/version.rb` both read it 2. Update [CHANGELOG.md](CHANGELOG.md) with the new version and date -3. Create a GitHub Release: - - Go to [Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml) - - Click **Run workflow**, enter the version number - - The workflow will: test → tag → publish to RubyGems → create GitHub Release - -Or manually: - -```bash -bundle exec rake release -``` - -This creates a git tag, pushes commits and tags, and pushes the `.gem` file to [rubygems.org](https://rubygems.org). +3. Go to [Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml), + click **Run workflow** and enter the version number. The workflow verifies the version + against `lib/`, tests, tags, publishes **both** gem names, and creates the GitHub Release. + +**Do not use `rake release`.** It is inherited from `bundler/gem_tasks` and publishes only +`capybara-screenshot-diff`, skipping the `snap_diff-capybara` mirror — the two gems must +never diverge in version. The full runbook, including the trusted-publishing prerequisites +and what to do when a run fails halfway, is in [docs/RELEASE_PREP.md](docs/RELEASE_PREP.md). diff --git a/README.md b/README.md index 909e337a..b9757ae0 100644 --- a/README.md +++ b/README.md @@ -11,9 +11,11 @@ Stop shipping UI bugs. Take screenshots in your Capybara tests, commit baselines **Why this gem?** Baselines live in git — review UI changes in pull requests like you review code. Runs offline, works in CI, zero vendor lock-in. Unlike Percy/Chromatic (paid SaaS), nothing to sign up for. Unlike BackstopJS, no Node required. -> **2.0 experiment (beta):** the gem is moving to a `SnapDiff` canonical namespace. Opt in with `gem "capybara-screenshot-diff", "2.0.0.beta3"` (or the latest 2.0.0 prerelease; prereleases are never installed by default — normal installs stay on 1.x). Legacy names keep working; the first legacy API a process touches prints one migration notice (lazily shimmed constants also warn once each — see [which names warn](docs/UPGRADING.md#deprecation-warnings)), silenceable via `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. Writing new code? Start from [SnapDiff — the canonical API](docs/snapdiff.md), which uses canonical names only. Migrating an existing suite? See the [upgrade guide](docs/UPGRADING.md). Share feedback on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166). +> **2.0 is the transitional release.** The gem's canonical namespace is now `SnapDiff`. Upgrading from 1.x is a version bump — every legacy `Capybara::Screenshot::Diff` / `CapybaraScreenshotDiff` name still resolves to the same object and keeps working. The first legacy API a process touches prints one migration notice (lazily shimmed constants also warn once each — see [which names warn](docs/UPGRADING.md#deprecation-warnings)), silenceable via `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. > -> Starting with the 2.0 prereleases the gem is also published as [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara) — identical content and versions under the forward-looking name, matching this repository. Install either; don't install both. +> **2.1 removes what 2.0 warns about**: the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction — libvips becomes the only backend. There is no 3.0. Writing new code? Start from [SnapDiff — the canonical API](docs/snapdiff.md), which uses canonical names only. Migrating an existing suite? See the [upgrade guide](docs/UPGRADING.md). +> +> **Two gem names, one gem.** `capybara-screenshot-diff` is the name to install. The same content is also published as [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara) — identical version, forward-looking name matching this repository — so that name is reserved and resolvable. **Install one, never both**: with both in a Gemfile the gem raises `SnapDiff::DualInstallError` at require time. ## Quick Start (5 minutes) @@ -22,9 +24,13 @@ Stop shipping UI bugs. Take screenshots in your Capybara tests, commit baselines ```ruby # Gemfile gem 'capybara-screenshot-diff' -gem 'ruby-vips' # Optional: 10x faster comparisons +gem 'ruby-vips' # The image backend. Needs libvips — see Installation below ``` +The gem ships no image backend of its own. Add `ruby-vips` (recommended, and the only +backend from 2.1 on) or `chunky_png` (pure Ruby, no system library, removed in 2.1) — with +neither, comparisons raise `Wrong adapter nil. Available adapters: []`. + ```ruby # test/test_helper.rb require 'capybara_screenshot_diff/minitest' @@ -186,7 +192,7 @@ Set `window_size` for consistent dimensions and use `perceptual_threshold: 2.0`
Will this slow down my tests? -Comparisons add ~50ms per image with VIPS. Without `ruby-vips`, ChunkyPNG is used (slower but no system dependency). `stability_time_limit` adds wait time — keep it low (0.1-0.5s) or use `disable_animations` instead. +Comparisons add ~50ms per image with VIPS. If you add `chunky_png` to your Gemfile instead, it is used as a pure-Ruby fallback (slower, no system dependency, and removed in 2.1). `stability_time_limit` adds wait time — keep it low (0.1-0.5s) or use `disable_animations` instead.
@@ -197,11 +203,12 @@ Comparisons add ~50ms per image with VIPS. Without `ruby-vips`, ChunkyPNG is use ## Installation -**Requirements:** Ruby 3.2+. Rails 7.1+ for Rails integration; non-Rails projects supported via `CapybaraScreenshotDiff.serve()`. For the `:vips` driver: [libvips 8.9+](https://libvips.github.io/libvips/install.html). On macOS: `brew install vips`. On Ubuntu: `apt-get install libvips-dev`. +**Requirements:** Ruby 3.2+, Capybara 2–3. Rails 7.1+ for Rails integration; non-Rails projects supported via `CapybaraScreenshotDiff.serve()`. For the `:vips` driver (recommended, and the only backend from 2.1 on): [libvips 8.9+](https://libvips.github.io/libvips/install.html). On macOS: `brew install vips`. On Ubuntu: `apt-get install libvips-dev`. ## Docs - [SnapDiff — the canonical API](docs/snapdiff.md) — setup, config, object map, custom drivers & reporters, canonical names only +- [Upgrading](docs/UPGRADING.md) — 1.x → 2.0, every renamed constant, which names warn, what 2.1 removes, rollback - [Framework Setup](docs/framework-setup.md) — Minitest, RSpec, Cucumber - [CI & Non-Rails Integration](docs/ci-integration.md) — GitHub Actions, reusable action, static sites, baseline updates - [Configuration Reference](docs/configuration.md) — all options explained @@ -211,11 +218,11 @@ Comparisons add ~50ms per image with VIPS. Without `ruby-vips`, ChunkyPNG is use ## Development -After checking out the repo, run `bin/setup` then `rake test`. See [Docker Testing](docs/docker-testing.md) for reproducible CI-matching test runs. +After checking out the repo, run `bin/setup` then `rake test`. See [Docker Testing](https://github.com/snap-diff/snap_diff-capybara/blob/master/docs/docker-testing.md) for reproducible CI-matching test runs. ## Contributing -See [CONTRIBUTING.md](CONTRIBUTING.md) +See [CONTRIBUTING.md](https://github.com/snap-diff/snap_diff-capybara/blob/master/CONTRIBUTING.md) ## License diff --git a/Rakefile b/Rakefile index c0c705ed..3562df55 100644 --- a/Rakefile +++ b/Rakefile @@ -5,13 +5,13 @@ require "rake/testtask" task default: :test -# THE 3.0 SPLIT. +# THE 2.1 SPLIT. # # test/legacy/ holds every test whose SUBJECT is the v1 compatibility surface # -- the old Capybara::Screenshot / CapybaraScreenshotDiff namespaces, their # deprecation warnings, and the gates that keep lib/capybara* alias-only. # Those tests guard the v1 contract for the whole 2.x line, so they stay and -# stay green; in 3.0 they are deleted by the same commit that deletes what +# stay green; in 2.1 they are deleted by the same commit that deletes what # they test: # # git rm -r lib/capybara* lib/capybara_screenshot_diff.rb \ @@ -23,7 +23,7 @@ task default: :test # # `rake test` -- everything, today's gate. # `rake test:canonical` -- exactly what must still pass once test/legacy and -# the v1 trees are gone. THE 3.0 GATE. +# the v1 trees are gone. THE 2.1 GATE. # `rake test:unit` -- unit-sized tests; test/legacy is unit-sized too # (legacy/ marks lifetime, not kind), so it is in. LEGACY_SURFACE_TESTS = "test/legacy/**/*_test.rb" @@ -34,7 +34,7 @@ Rake::TestTask.new(:test) do |t| t.test_files = FileList["test/**/*_test.rb"] end -desc "Run every test that must survive the 3.0 deletion of the v1 surface" +desc "Run every test that must survive the 2.1 deletion of the v1 surface" Rake::TestTask.new("test:canonical") do |t| t.libs << "test" t.libs << "lib" diff --git a/capybara-screenshot-diff.gemspec b/capybara-screenshot-diff.gemspec index fea3dbc2..efa095d5 100644 --- a/capybara-screenshot-diff.gemspec +++ b/capybara-screenshot-diff.gemspec @@ -9,20 +9,27 @@ Gem::Specification.new do |spec| spec.version = SnapDiff::VERSION spec.authors = ["Uwe Kubosch"] spec.email = ["uwe@kubosch.no"] - spec.summary = "Track your GUI changes with diff assertions" - spec.description = "Save screen shots and track changes with graphical diff" + spec.summary = "Visual regression testing for Capybara — screenshot diffs in your test suite" + spec.description = "Take screenshots in your Capybara tests, commit the baselines to git, " \ + "and let your suite fail on unintended visual changes. Runs offline, no SaaS." spec.homepage = "https://github.com/snap-diff/snap_diff-capybara" spec.required_ruby_version = ">= 3.2" spec.license = "MIT" spec.metadata["allowed_push_host"] = "https://rubygems.org/" + spec.metadata["source_code_uri"] = spec.homepage + spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/master/CHANGELOG.md" + spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues" + spec.metadata["documentation_uri"] = "#{spec.homepage}/blob/master/docs/snapdiff.md" # Allow-list: everything a consumer needs at runtime plus the shipped docs. # Build/dev files (gems.rb, Rakefile, the gemspec itself, tests, CI) stay out. spec.files = `git ls-files -z`.split("\x0") .grep(%r{\A(lib/|docs/|README\.md\z|LICENSE\.txt\z|CHANGELOG\.md\z)}) - .grep_v(%r{\Adocs/RELEASE_PREP\.md\z}) # maintainer-only, not user documentation + # Contributor docs: they describe releasing this gem and running its own suite + # via bin/dtest, which is not packaged. Nothing a consumer can act on. + .grep_v(%r{\Adocs/(RELEASE_PREP|docker-testing)\.md\z}) - spec.bindir = "exe" - spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) } + # No executables: the allow-list above never matches exe/, so bindir and + # executables would always be empty. spec.require_paths = ["lib"] spec.add_development_dependency "actionpack", ">= 7.1", "< 9" diff --git a/docs/RELEASE_PREP.md b/docs/RELEASE_PREP.md index a50b1ddc..2b4087bc 100644 --- a/docs/RELEASE_PREP.md +++ b/docs/RELEASE_PREP.md @@ -1,44 +1,81 @@ -# Release Preparation — v1.15.1 +# Release runbook -## Summary +Maintainer-only. This file is deliberately excluded from the packaged gem. -HIGH-severity VipsDriver resize fix (retina halving could enlarge screenshots -and corrupt baselines) plus the driver contract tests that caught it. +Releases are **one `workflow_dispatch`**. There is no `rake release`, no manual +`gem push`, and no local credential. Everything below either happens in CI or has to +be committed to `master` before you dispatch. -## Release Checklist +## What the Release workflow does -### Pre-Release +[`.github/workflows/release.yml`](../.github/workflows/release.yml), dispatched from +[Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml) +with a single `version` input (e.g. `2.0.0`, or `2.1.0.beta1`): -- [x] Update version to `1.15.1` -- [x] Run tests: `bundle exec rake test:unit` (376 runs, 0 failures) -- [x] Update CHANGELOG.md +1. **Verify version** — reads `Capybara::Screenshot::Diff::VERSION` out of `lib/` and + fails if it does not equal the input. This is the guard that makes the dispatch + safe: the version lives in `lib/snap_diff/version.rb` and must already be on + `master`. +2. **Test** — `bundle exec rake test:unit` on Ruby 4.0. +3. **Tag** — creates and pushes `v`. Idempotent: it skips if the tag already + exists at HEAD and fails loudly if it exists at a *different* commit. A failed run + can be re-dispatched without cleanup. +4. **Publish `capybara-screenshot-diff`** — `rubygems/release-gem@v1`, via RubyGems + trusted publishing (OIDC). No API key is stored anywhere. +5. **Publish the `snap_diff-capybara` mirror** — the same gemspec is loaded, renamed + in memory, built and pushed. The mirror gemspec is *generated in CI, never + committed*, so local `gem build` and the `gemspec` directive in `gems.rb` stay + unambiguous. It reuses the credential `release-gem` already set up, which covers + any gem whose rubygems.org settings trust this repo + workflow. +6. **GitHub Release** — body links to the CHANGELOG and upgrade guide **at the tag**. + Prerelease status is auto-detected from the tag, so `v2.1.0.beta1` lands as + *Pre-release* and never displaces the *Latest* badge. -### Release (One Click) +### Prerequisites that live outside this repo -1. Push to GitHub -2. Go to [Actions → Release](https://github.com/snap-diff/snap_diff-capybara/actions/workflows/release.yml) -3. Click **Run workflow**, enter `1.15.1` -4. Workflow will: test → tag → publish to RubyGems → create GitHub Release +- **Both** gem names must trust this repo + `release.yml` as a trusted publisher on + rubygems.org — `capybara-screenshot-diff` **and** `snap_diff-capybara`. If only one + does, step 4 or 5 fails after the tag is already pushed; re-dispatch after fixing. +- Branch protection on `master` must allow the workflow's tag push. -### Post-Release +## Before you dispatch -- [ ] Verify on [RubyGems](https://rubygems.org/gems/capybara-screenshot-diff) -- [ ] Verify GitHub Release created +- [ ] `lib/snap_diff/version.rb` bumped to the exact version you will type into the + workflow. Nothing else holds a version — the gemspec, the legacy + `capybara/screenshot/diff/version.rb` and the mirror gemspec all read it. +- [ ] `CHANGELOG.md` has a section for this version with a real date (not + `unreleased`), written for someone upgrading from the last **stable** release + rather than from the previous prerelease. +- [ ] Docs carry no stale version pins — `README.md`, `docs/UPGRADING.md`, + `docs/snapdiff.md`. Grep for the previous version string. +- [ ] `mise x ruby@4.0.6 -- bundle exec rake test` (full suite, both gates) and + `mise x ruby@4.0.6 -- bundle exec standardrb` are green. +- [ ] CI is green on `master` at the commit you are releasing — the workflow only + runs `test:unit`, which is a subset. +- [ ] `gem build capybara-screenshot-diff.gemspec` and inspect the file list if + anything touched the gemspec allow-list. `*.gem` is gitignored, but delete the + artifact anyway — a stale one in the working tree is confusing. -## What Changed +### Prereleases -### Added -- `SnapDiff::Config` — flat, additive consolidation of all 27 settings - (`SnapDiff.config` / `SnapDiff.configure`); old accessors stay canonical +Nothing special to configure. Use a prerelease version string (`2.1.0.beta1`) in +`version.rb` and in the dispatch input. RubyGems never resolves a prerelease by +default, and GitHub marks the release *Pre-release* on its own. -### Fixed -- `pending_if_new` no longer converts real teardown/after-hook failures into - pending tests (Minitest defers to `after_teardown`; RSpec uses `append_after`; - known residual for consumer `append_after` hooks documented) -- `BacktraceFilter` custom `lib_directory` matches on a path boundary +## After -### Internal -- Guard tests for failure masking and skip-area/VCS-baseline regressions; - two files merged into sole consumers (constant paths preserved) +- [ ] Both gems visible and at the same version: + [capybara-screenshot-diff](https://rubygems.org/gems/capybara-screenshot-diff), + [snap_diff-capybara](https://rubygems.org/gems/snap_diff-capybara). A version + published under only one name is the failure mode to watch for — the two gems + ship identical files and the dual-install guard assumes they never diverge. +- [ ] GitHub Release created, with the right Latest/Pre-release status. +- [ ] `gem install capybara-screenshot-diff -v ` in a scratch dir resolves. -See [CHANGELOG.md](../CHANGELOG.md) for full details. +## If a run fails halfway + +Re-dispatch the same version. The tag step is idempotent and `gem push` rejects a +duplicate version, so the only real hazard is a mirror push that failed for a +credential reason — fix the trusted-publisher settings and re-dispatch. **Never** +retag: the workflow refuses to move an existing tag and you should not do it by hand +either. diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index e7b623e0..f3137a38 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -1,12 +1,12 @@ # Upgrading -## Upgrading to v2.0 (alpha) +## Upgrading to v2.0 ### Overview Version 2.0 introduces a new canonical namespace (`SnapDiff`) for cleaner, more discoverable code. The public DSL remains unchanged — your existing `screenshot` and `assert_matches_screenshot` calls work without modification. This guide covers the optional migration path for settings and the new namespace. -**Status:** `2.0.0.beta3` is an opt-in prerelease. RubyGems never installs prereleases by default resolution — normal `bundle update` keeps you on the 1.x line. The final 2.0.0 ships only after adopter feedback; please report anything surprising on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166). +**Status:** 2.0 is the **transitional** release — the v1 API and the canonical `SnapDiff` API both work. **2.1 removes** everything 2.0 warns about (the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction). There is no 3.0. Migrating on 2.0 is optional; doing it before 2.1 is not. **Estimated upgrade time:** 5–15 minutes (most users need only the Gemfile pin) @@ -14,17 +14,20 @@ Version 2.0 introduces a new canonical namespace (`SnapDiff`) for cleaner, more [SnapDiff — the canonical API](snapdiff.md): the same setup, configuration, and extension points with canonical names only, no legacy shapes to unlearn. -**Breaking changes:** None for the DSL; one migration notice per process plus a deprecation warning per legacy constant you reference (both suppressible), plus two known alpha caveats (see below) +**Breaking changes:** None for the DSL; one migration notice per process plus a deprecation warning per legacy constant you reference (both suppressible), plus the [known caveats](#known-caveats) below --- ### The Short Version (Most Users) ```ruby -# In your Gemfile — the exact prerelease version is required to opt in -gem "capybara-screenshot-diff", "2.0.0.beta3" # or the latest 2.0.0 prerelease +# In your Gemfile +gem "capybara-screenshot-diff", "~> 2.0" ``` +The same content is also published as `snap_diff-capybara`. Install **one** — with both +in a Gemfile the gem raises `SnapDiff::DualInstallError` at require time. + ```bash bundle install bundle exec rake test @@ -114,7 +117,6 @@ Capybara::Screenshot::Diff.configure do |screenshot, diff| screenshot.window_size = [1280, 1024] screenshot.blur_active_element = false diff.tolerance = 0.0005 - diff.driver = :vips end ``` @@ -125,7 +127,6 @@ SnapDiff.start do |screenshot, diff| screenshot.window_size = [1280, 1024] screenshot.blur_active_element = false diff.tolerance = 0.0005 - diff.driver = :vips end ``` @@ -136,7 +137,6 @@ SnapDiff.configure do |config| config.window_size = [1280, 1024] config.blur_active_element = false config.tolerance = 0.0005 - config.driver = :vips end ``` @@ -215,6 +215,16 @@ warns once per process per subject, through the same channel and the same silenc | read `SnapDiff::Drivers.available` | driver detection | require `ruby-vips` instead of branching on a detected list | | `include SnapDiff::Driver` in your own driver class | the driver mixin | nothing — see below | +> **The one removal on this list that 2.0 cannot warn you about: the `driver:` setting +> itself.** `SnapDiff.config.driver = :vips` (and the legacy +> `Capybara::Screenshot::Diff.driver = :vips`, and the per-screenshot `driver:` override) +> is **silent** in 2.0 and raises `NoMethodError: undefined method 'driver='` in 2.1. +> Warning on it would fire on the recommended configuration, so this note is the warning: +> **delete the line.** With libvips the only backend there is nothing to select, and the +> default just works. The same goes for `driver: :auto` on a machine that *has* +> `ruby-vips` — the `:auto` warning above only fires when `:auto` actually falls back to +> ChunkyPNG, because that is the case where 2.1 stops the process comparing at all. + ``` [snap_diff deprecation] `driver: :auto` selected chunky_png because libvips is not available in this process. The chunky_png driver is REMOVED in 2.1, when libvips (the `ruby-vips` gem) becomes required -- install it now, or this setup stops comparing on 2.1. See docs/drivers.md. Silence with `SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1. (shown once per process) (called from /app/test/test_helper.rb:12) ``` @@ -223,7 +233,7 @@ warns once per process per subject, through the same channel and the same silenc mixin, the `SnapDiff::Drivers.loaded` registry, `SnapDiff::Drivers.available` / `SnapDiff::Utils.detect_available_drivers`, and selecting a driver by name. Nothing replaces them, and this guide is not going to pretend otherwise — if you maintain a third-party driver, -say so on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166) before 2.1 ships. +say so on [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) before 2.1 ships. Three spots on the same chopping block stay silent: the legacy `Capybara::Screenshot::Diff::LOADED_DRIVERS` / `::AVAILABLE_DRIVERS` aliases are plain constants @@ -276,9 +286,10 @@ export SNAP_DIFF_SILENCE_DEPRECATIONS=1 --- -### Known Alpha Caveats +### Known Caveats -Two deliberate consequences of the lazy shim design — both flagged for feedback on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166): +Two deliberate consequences of the lazy shim design. Both go away at 2.1, when the shimmed +names go away with them: 1. **`defined?` / `const_defined?` on lazily-shimmed legacy names returns `false`/`nil`.** The shims resolve via `const_missing`, which those checks never trigger. Feature detection like `defined?(Capybara::Screenshot::Diff::ImageCompare)` must move to the `SnapDiff::` name. Everything in [Silent by design](#silent-by-design) is unaffected — those names are real constants, so `defined?`, `const_defined?` and `rescue` all behave as they always did. @@ -331,12 +342,15 @@ All settings and baselines are compatible with v1.x. Simply pin your Gemfile bac ### Summary Checklist -- [ ] Pin `gem "capybara-screenshot-diff", "2.0.0.beta3"` (or the latest 2.0.0 prerelease) in your Gemfile +- [ ] Pin `gem "capybara-screenshot-diff", "~> 2.0"` in your Gemfile - [ ] Run `bundle install` - [ ] Run your test suite to verify no regressions -- [ ] (Optional) Migrate config to the `SnapDiff` namespace +- [ ] Read the warnings it prints — each one names something 2.1 removes +- [ ] Add `gem "ruby-vips"` if you are not already on it (2.1 makes libvips the only backend) +- [ ] Drop `driver:` from your config — it is silent in 2.0 and gone in 2.1 +- [ ] (Optional, but do it before 2.1) Migrate config and constants to the `SnapDiff` namespace - [ ] (Optional) Silence deprecation warnings if not ready to migrate -- [ ] Report anything surprising on [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166) +- [ ] Report anything surprising on [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) --- diff --git a/docs/architecture.md b/docs/architecture.md index dc864310..84521b1c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -127,7 +127,7 @@ Drivers abstract image processing operations. Shared default behavior lives in t **Auto-detection:** `SnapDiff::Drivers.detect_available` tries to load `:vips` first (via `ruby-vips` gem), then `:chunky_png`. The `:auto` driver mode picks the first available. `Utils.detect_available_drivers` is the older name and one-lines into it. -**Registry (ADR-008 step 5b):** `SnapDiff::Drivers.loaded` is the canonical driver-class cache — a `name => class` hash filled lazily by `Utils.find_driver_class_for`, and the registration point for custom drivers (the legacy `Capybara::Screenshot::Diff::LOADED_DRIVERS` is an eager same-object alias, so registrations through either land in the same hash). `SnapDiff::Drivers.available` is the canonical read API for the detected list, and since the 3.0-readiness pass the value lives with it, as `SnapDiff::Drivers::AVAILABLE_DRIVERS` — that constant is now the published stubbing point, and the legacy `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` is an eager same-object alias of it. `SnapDiff::Drivers.for` resolves an options hash to a driver instance. See [Custom drivers](snapdiff.md#custom-drivers). +**Registry (ADR-008 step 5b):** `SnapDiff::Drivers.loaded` is the canonical driver-class cache — a `name => class` hash filled lazily by `Utils.find_driver_class_for`, and the registration point for custom drivers (the legacy `Capybara::Screenshot::Diff::LOADED_DRIVERS` is an eager same-object alias, so registrations through either land in the same hash). `SnapDiff::Drivers.available` is the canonical read API for the detected list, and since the 2.1-readiness pass the value lives with it, as `SnapDiff::Drivers::AVAILABLE_DRIVERS` — that constant is now the published stubbing point, and the legacy `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` is an eager same-object alias of it. `SnapDiff::Drivers.for` resolves an options hash to a driver instance. See [Custom drivers](snapdiff.md#custom-drivers). ### 6. Difference Region Detection @@ -223,7 +223,7 @@ Since ADR-008 step 1 the storage ownership is inverted from the original v2 cons The legacy `Capybara::Screenshot.*` / `Capybara::Screenshot::Diff.*` accessors are thin delegators generated from `SnapDiff::LegacyShims::CONFIG_MAPPING` (both singleton and instance methods, matching what `mattr_accessor` used to define) that forward to that one object. One storage, two views — a write through either surface is visible through the other structurally, not by synchronization. -Since the 3.0-readiness pass, `lib/snap_diff/legacy_shims.rb` is the single file that holds the v1 surface as code: the `const_missing` forwarders, `CONFIG_MAPPING` and its generator, the derived forwarders (`Screenshot.active?`, `Diff.configure`, `Diff.default_options`, …) and `SnapDiff.start`. `Config` itself names nothing from the v1 namespaces — it declares its settings in `Config::SETTINGS`, and `LegacyShims::CONFIG_MAPPING` says which legacy holder each one is exposed on (an invariant pinned by `snap_diff_config_test.rb`). `lib/capybara/screenshot/diff/config_legacy.rb` remains at the old path as a pair of requires. +Since the 2.1-readiness pass, `lib/snap_diff/legacy_shims.rb` is the single file that holds the v1 surface as code: the `const_missing` forwarders, `CONFIG_MAPPING` and its generator, the derived forwarders (`Screenshot.active?`, `Diff.configure`, `Diff.default_options`, …) and `SnapDiff.start`. `Config` itself names nothing from the v1 namespaces — it declares its settings in `Config::SETTINGS`, and `LegacyShims::CONFIG_MAPPING` says which legacy holder each one is exposed on (an invariant pinned by `snap_diff_config_test.rb`). `lib/capybara/screenshot/diff/config_legacy.rb` remains at the old path as a pair of requires. The two legacy views are organized into two namespaces: diff --git a/docs/drivers.md b/docs/drivers.md index f4faede0..b9608177 100644 --- a/docs/drivers.md +++ b/docs/drivers.md @@ -9,13 +9,15 @@ ## Removed in 2.1: everything on this page except VIPS 2.1 makes **libvips the only backend**. 2.0 is the transitional release — all of the -following still works, and warns once per process naming 2.1. Silence the warnings with +following still works. Most of it warns once per process naming 2.1; the rows marked +*silent* cannot warn, and this table is their notice. Silence the warnings with `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`. | Removed in 2.1 | What to do in 2.0 | |---|---| | the `:chunky_png` driver | add `gem "ruby-vips"` to your Gemfile and drop `driver: :chunky_png` | -| `driver: :auto` (and the `:auto` default) | with one backend there is nothing to choose; install `ruby-vips` and the default just works | +| the `driver:` setting itself — `SnapDiff.config.driver =`, the legacy `Capybara::Screenshot::Diff.driver =`, and the per-screenshot `driver:` override (**silent**: on 2.1 they raise `NoMethodError`) | delete the line; one backend needs no selection | +| `driver: :auto` (and the `:auto` default) — warns **only when `:auto` actually falls back to ChunkyPNG**, i.e. when `ruby-vips` is missing; **silent** otherwise | with one backend there is nothing to choose; install `ruby-vips` and the default just works | | `shift_distance_limit` | ChunkyPNG-only. Use `median_filter_window_size`, `tolerance` or `color_distance_limit` — see [Configuration](configuration.md#allowed-shift-distance) | | `SnapDiff::Driver` (the custom-driver mixin) | nothing — see below | | `SnapDiff::Drivers.loaded` (the registry) | nothing — see below | diff --git a/docs/snapdiff.md b/docs/snapdiff.md index d431afe6..c74fdccd 100644 --- a/docs/snapdiff.md +++ b/docs/snapdiff.md @@ -104,7 +104,6 @@ All 27 settings live on one flat object, `SnapDiff.config` (a `SnapDiff::Config` SnapDiff.configure do |config| config.window_size = [1280, 1024] config.tolerance = 0.0005 - config.driver = :vips config.save_path = "doc/screenshots" end @@ -270,7 +269,7 @@ SnapDiff::Reporting.finalize! > by name. In 2.0 all of it still works and warns once per process (silence with > `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`). Nothing > here migrates to a 2.1 shape — there is no 2.1 shape. If you maintain a driver, say so on -> [#166](https://github.com/snap-diff/snap_diff-capybara/issues/166) before 2.1 ships. +> [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) before 2.1 ships. A driver is a plain object that does the image work. Include `SnapDiff::Driver` for the shared defaults, then implement the operations the comparison engine calls: diff --git a/test/unit/gemspec_packaging_test.rb b/test/unit/gemspec_packaging_test.rb new file mode 100644 index 00000000..e8bbee26 --- /dev/null +++ b/test/unit/gemspec_packaging_test.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +require "test_helper" + +# The gemspec's `spec.files` allow-list is the one part of the release that no +# other test looks at, and it has shipped broken twice: a missing +# Bundler.require entry file makes `gem "..."` load NOTHING and fail later with +# a confusing NameError, far from the cause. These assertions are about the +# PACKAGED file list, not about the working tree. +class GemspecPackagingTest < ActiveSupport::TestCase + GEMSPEC_PATH = File.expand_path("../../capybara-screenshot-diff.gemspec", __dir__) + + def spec + @spec ||= Gem::Specification.load(GEMSPEC_PATH) + end + + # Bundler requires the gem's OWN NAME. Each published name needs a file that + # matches it, or `Bundler.require` silently no-ops for that name. + test "ships a Bundler.require entry file for both published gem names" do + assert_includes spec.files, "lib/capybara-screenshot-diff.rb" + assert_includes spec.files, "lib/snap_diff-capybara.rb" + end + + test "ships the files rubygems.org and a consumer read" do + ["README.md", "LICENSE.txt", "CHANGELOG.md", "docs/UPGRADING.md", "docs/snapdiff.md"] + .each { |file| assert_includes spec.files, file } + end + + # Contributor-only docs describe releasing this gem and bin/dtest, neither of + # which a consumer has. + test "ships no contributor-only docs" do + refute_includes spec.files, "docs/RELEASE_PREP.md" + refute_includes spec.files, "docs/docker-testing.md" + end + + test "ships no build or development files" do + cruft = spec.files.grep_v(%r{\A(lib/|docs/|README\.md\z|LICENSE\.txt\z|CHANGELOG\.md\z)}) + assert_empty cruft + end + + # Every driver is an optional require. Adding a runtime dependency here is a + # decision, not an accident -- 2.1 makes ruby-vips one deliberately. + test "declares capybara as its only runtime dependency" do + runtime = spec.dependencies.select { |dependency| dependency.type == :runtime } + assert_equal ["capybara"], runtime.map(&:name) + end +end From 92c80a74e67976b85dfdcf3cb0181f50218634ae Mon Sep 17 00:00:00 2001 From: Paul Keen <125715+pftg@users.noreply.github.com> Date: Mon, 24 Aug 2026 07:56:39 +0200 Subject: [PATCH 2/5] docs: SnapDiff::Error is the base class for errors the gem defines, not every error it raises docs/snapdiff.md's object map said "Base class for every error this gem raises". It is not: a missing image backend raises a bare RuntimeError ("Wrong adapter nil. Available adapters: []", reproduced on a bundle with neither ruby-vips nor chunky_png) and StableScreenshoter raises ArgumentError. Verified the four defined errors -- ExpectationNotMet, UnstableImage, WindowSizeMismatchError, DualInstallError -- do all inherit SnapDiff::Error, so the useful half of the promise holds and is now the one being made. --- CHANGELOG.md | 7 +++++-- docs/snapdiff.md | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c39e31f..a9cfbe6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -110,8 +110,11 @@ the line, one backend needs no selection), and the legacy `LOADED_DRIVERS` / - **One consolidated config object** — all 27 settings on `SnapDiff.config`, via `SnapDiff.configure { |config| … }`. Old and new surfaces share one storage, so a write through either is visible through the other -- **`SnapDiff::Error` is the catch-all** the docs promise: every error the gem - raises inherits it, including `WindowSizeMismatchError` and `DualInstallError` +- **`SnapDiff::Error` is the base class for every error the gem defines** — + `ExpectationNotMet`, `UnstableImage`, `WindowSizeMismatchError` and + `DualInstallError` all inherit it, so one `rescue SnapDiff::Error` covers them. + (Misuse still surfaces as plain Ruby: `ArgumentError` for bad arguments, + `RuntimeError` when no image backend is installed.) - **Deprecation warnings name your call site**, so migration is warning-driven rather than grep-driven - **Dual-install guard** — installing both `capybara-screenshot-diff` and diff --git a/docs/snapdiff.md b/docs/snapdiff.md index c74fdccd..57fc812f 100644 --- a/docs/snapdiff.md +++ b/docs/snapdiff.md @@ -160,7 +160,7 @@ integration require; a few objects need their own require, noted below. | `SnapDiff::Region` | Bounding box value object — `from_edge_coordinates`, `to_edge_coordinates` | | `SnapDiff::DSL` | `screenshot`, `assert_matches_screenshot`, `capture_screenshot`, groups/sections | | `SnapDiff::Minitest::Assertions` | Minitest wiring (`snap_diff/integrations/minitest`) | -| `SnapDiff::Error` | Base class for every error this gem raises | +| `SnapDiff::Error` | Base class for every error this gem *defines* — one `rescue SnapDiff::Error` covers all of the rows below. Misuse still surfaces as plain Ruby: bad arguments raise `ArgumentError`, and a missing image backend raises `RuntimeError` | | `SnapDiff::ExpectationNotMet` | A screenshot did not match its baseline | | `SnapDiff::UnstableImage` | No stable capture within `stability_time_limit` / `wait` | | `SnapDiff::WindowSizeMismatchError` | Browser window is not the configured `window_size` | From ed8ee8dde1b497bada993a1f079913813426308b Mon Sep 17 00:00:00 2001 From: Paul Keen <125715+pftg@users.noreply.github.com> Date: Mon, 24 Aug 2026 07:57:29 +0200 Subject: [PATCH 3/5] docs: tag protection, not branch protection, is what gates the release tag push Branch protection rules do not govern tag pushes; tag protection rules (or rulesets) do. The runbook prerequisite now names the right control. --- docs/RELEASE_PREP.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/RELEASE_PREP.md b/docs/RELEASE_PREP.md index 2b4087bc..d05afb4c 100644 --- a/docs/RELEASE_PREP.md +++ b/docs/RELEASE_PREP.md @@ -36,7 +36,8 @@ with a single `version` input (e.g. `2.0.0`, or `2.1.0.beta1`): - **Both** gem names must trust this repo + `release.yml` as a trusted publisher on rubygems.org — `capybara-screenshot-diff` **and** `snap_diff-capybara`. If only one does, step 4 or 5 fails after the tag is already pushed; re-dispatch after fixing. -- Branch protection on `master` must allow the workflow's tag push. +- If tag protection rules are ever added to this repo, `v*` must allow + `github-actions[bot]` to push — that is how step 3 creates the tag. ## Before you dispatch From 0684513b1db9ffcba8ef6a359bdeb8d92aa1816b Mon Sep 17 00:00:00 2001 From: Paul Keen <125715+pftg@users.noreply.github.com> Date: Mon, 24 Aug 2026 07:58:48 +0200 Subject: [PATCH 4/5] docs: the per-screenshot driver: override dies quietly, the config setting dies loudly Both were lumped together as "raises NoMethodError on 2.1". Only the config setting does. Per-screenshot options are a free-form hash, so on 2.1 `screenshot "index", driver: :vips` is inert and nothing tells you the line is dead -- #249's own upgrade note spells out the split. Grep-for-it advice added, since that is the only signal a user gets. --- CHANGELOG.md | 9 +++++---- docs/UPGRADING.md | 19 +++++++++++-------- docs/drivers.md | 3 ++- 3 files changed, 18 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a9cfbe6b..87eb9ef3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -95,10 +95,11 @@ contract one release ahead is the mitigation. 2.0 warns once per process for eac | `include SnapDiff::Driver` in your own driver | nothing — custom drivers have no migration path | Two removals 2.0 cannot warn about, so they are written down instead: **`driver:` -as a setting goes away entirely** (`SnapDiff.config.driver = :vips` and -`Capybara::Screenshot::Diff.driver = :vips` raise `NoMethodError` on 2.1 — delete -the line, one backend needs no selection), and the legacy `LOADED_DRIVERS` / -`AVAILABLE_DRIVERS` constants are plain aliases with nothing to hook. +as a setting goes away entirely** — `SnapDiff.config.driver = :vips` and +`Capybara::Screenshot::Diff.driver = :vips` raise `NoMethodError` on 2.1, and the +per-screenshot `screenshot "x", driver: :vips` is silently ignored there. Delete +both; one backend needs no selection. The legacy `LOADED_DRIVERS` / +`AVAILABLE_DRIVERS` constants are also plain aliases with nothing to hook. ### Added - **`SnapDiff` is the canonical namespace** — the implementation lives in diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index f3137a38..b9fd5f65 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -216,14 +216,17 @@ warns once per process per subject, through the same channel and the same silenc | `include SnapDiff::Driver` in your own driver class | the driver mixin | nothing — see below | > **The one removal on this list that 2.0 cannot warn you about: the `driver:` setting -> itself.** `SnapDiff.config.driver = :vips` (and the legacy -> `Capybara::Screenshot::Diff.driver = :vips`, and the per-screenshot `driver:` override) -> is **silent** in 2.0 and raises `NoMethodError: undefined method 'driver='` in 2.1. -> Warning on it would fire on the recommended configuration, so this note is the warning: -> **delete the line.** With libvips the only backend there is nothing to select, and the -> default just works. The same goes for `driver: :auto` on a machine that *has* -> `ruby-vips` — the `:auto` warning above only fires when `:auto` actually falls back to -> ChunkyPNG, because that is the case where 2.1 stops the process comparing at all. +> itself.** `SnapDiff.config.driver = :vips` and the legacy +> `Capybara::Screenshot::Diff.driver = :vips` are **silent** in 2.0 and raise +> `NoMethodError: undefined method 'driver='` in 2.1, at config time before any test runs. +> The per-screenshot form — `screenshot "index", driver: :vips` — is silent in 2.0 **and** +> in 2.1: per-screenshot options are a free-form hash, so an unknown key is simply inert. +> Warning on any of this would fire on the recommended configuration, so this note is the +> warning: **delete the line, and grep for the per-screenshot one.** With libvips the only +> backend there is nothing to select, and the default just works. The same goes for +> `driver: :auto` on a machine that *has* `ruby-vips` — the `:auto` warning above only +> fires when `:auto` actually falls back to ChunkyPNG, because that is the case where 2.1 +> stops the process comparing at all. ``` [snap_diff deprecation] `driver: :auto` selected chunky_png because libvips is not available in this process. The chunky_png driver is REMOVED in 2.1, when libvips (the `ruby-vips` gem) becomes required -- install it now, or this setup stops comparing on 2.1. See docs/drivers.md. Silence with `SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1. (shown once per process) (called from /app/test/test_helper.rb:12) diff --git a/docs/drivers.md b/docs/drivers.md index b9608177..69792949 100644 --- a/docs/drivers.md +++ b/docs/drivers.md @@ -16,7 +16,8 @@ following still works. Most of it warns once per process naming 2.1; the rows ma | Removed in 2.1 | What to do in 2.0 | |---|---| | the `:chunky_png` driver | add `gem "ruby-vips"` to your Gemfile and drop `driver: :chunky_png` | -| the `driver:` setting itself — `SnapDiff.config.driver =`, the legacy `Capybara::Screenshot::Diff.driver =`, and the per-screenshot `driver:` override (**silent**: on 2.1 they raise `NoMethodError`) | delete the line; one backend needs no selection | +| the `driver:` setting itself — `SnapDiff.config.driver =` and the legacy `Capybara::Screenshot::Diff.driver =` (**silent in 2.0**; on 2.1 they raise `NoMethodError` at config time) | delete the line; one backend needs no selection | +| the per-screenshot `driver:` override — `screenshot "index", driver: :vips` (**silent in 2.0 *and* 2.1**: per-screenshot options are a free-form hash, so an unknown key is simply inert) | delete the option, and grep for it — nothing will tell you the line is dead | | `driver: :auto` (and the `:auto` default) — warns **only when `:auto` actually falls back to ChunkyPNG**, i.e. when `ruby-vips` is missing; **silent** otherwise | with one backend there is nothing to choose; install `ruby-vips` and the default just works | | `shift_distance_limit` | ChunkyPNG-only. Use `median_filter_window_size`, `tolerance` or `color_distance_limit` — see [Configuration](configuration.md#allowed-shift-distance) | | `SnapDiff::Driver` (the custom-driver mixin) | nothing — see below | From 37b1f800bae33626ddcfafeab7c588aca1fcb7b5 Mon Sep 17 00:00:00 2001 From: Paul Keen <125715+pftg@users.noreply.github.com> Date: Mon, 24 Aug 2026 08:13:40 +0200 Subject: [PATCH 5/5] docs: pin versions, and stop telling people to delete baselines Second pass, from customer-persona findings. Each verified here before acting; two of the four reported items turned out to be artifacts of the PUBLISHED beta3 rather than of master, and are handled as such. Baselines (the oldest bug in the tracker: #5 and #6 in 2018, #133 in 2024) - README told users to "delete the baseline and re-run" in two places. It cannot work. `Vcs.checkout_vcs` (lib/snap_diff/vcs.rb:24) resolves every baseline with `git show HEAD:`, and `ScreenshotMatcher#check_base_screenshot` calls it before `need_to_compare?` tests `base_path.exist?` -- so a committed baseline is fetched from HEAD no matter what the working tree says, and `rm` changes nothing. - New first-class "Accepting an intentional change" section: the mechanism, the commit that actually accepts it, and the surprising part -- staging is not enough, so no local run goes green until you commit. The FAQ answer now says the same thing instead of the opposite. - Deliberately does NOT document RECORD_SCREENSHOTS. It is printed by our own error message (screenshot_matcher.rb:73) but read nowhere in lib/; a separate lane is implementing it, and it should be documented once it works, not before. Version pinning - `gem "snap_diff-capybara"` unpinned installs 0.0.1 -- a placeholder whose entire payload is one README and zero Ruby files (verified by fetching and unpacking it), so the user gets an immediate LoadError. And unpinned `gem "capybara-screenshot-diff"` resolves to 1.15.1, not to the 2.0 the surrounding prose is selling. Every install instruction now pins, and the README says plainly that the mirror name is not the one to reach for. CHANGELOG, all verified - Failure messages leaked a libvips pointer struct via the comparison metadata; `to_h` excludes `diff_mask` since #234, which landed after the beta3 tag, so 2.0.0 final is the fix. - Known limitation: fork-parallel runs write no HTML report. Workers accumulate assertions per process; the report is written from `Minitest.after_run` in the parent (integrations/minitest.rb:69), which never sees them. Artifacts and pass/fail are unaffected. - A note for anyone sitting on a prerelease: beta3's deprecation channel was incomplete, so its silence is not evidence of being migrated. Constants - `Capybara::Screenshot::Os` -> `SnapDiff::Os` was in no rename table. Gemspec - rubygems_mfa_required. The four URI fields were added in the first commit. --- CHANGELOG.md | 16 ++++++++++++ README.md | 42 ++++++++++++++++++++++++++++---- capybara-screenshot-diff.gemspec | 1 + docs/UPGRADING.md | 11 +++++++-- docs/migration-guide.md | 2 +- 5 files changed, 64 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 87eb9ef3..5919ea9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,6 +50,11 @@ site: (constant); use `SnapDiff::Comparison` instead. (called from test/test_helper.rb:12) ``` +> **On a 2.0.0 prerelease? Upgrade, do not trust its silence.** In `2.0.0.beta3` the +> deprecation channel was incomplete: a v1-only suite got **no** warnings at all, and +> the driver-half removal warnings did not exist yet. Silence on a beta is not evidence +> that you are migrated. + Requiring the gem, the DSL, settings accessors and the eagerly-defined constants (the error classes, `::VERSION`, `Os`, `Region`, `Reporters::Default`, `LOADED_DRIVERS`, `AVAILABLE_DRIVERS`) are **silent by design** — @@ -146,8 +151,19 @@ both; one backend needs no selection. The legacy `LOADED_DRIVERS` / comparison previously raised `NameError` there - `require "snap_diff/integrations/…"` loads the full `SnapDiff` surface, and `gem "snap_diff-capybara"` works with `Bundler.require` +- **Failure messages no longer dump a libvips pointer struct.** The comparison + metadata carried the raw `diff_mask` image into the error text + (`"diff_mask":{"ptr":{}…}`); it is excluded now, leaving the metrics - Reporter failure warnings use one brand and name the failing reporter class +### Known limitations +- **Fork-based parallel tests produce no HTML report.** Under Minitest's forked + parallel executor (`parallelize(workers: N)`, the Rails default), each worker + accumulates its assertions in its own process, while the report is written from + `Minitest.after_run` in the parent — which never sees them. Pass/fail is correct + and the diff image artifacts are still written; only the HTML report is missing. + Fixed in 2.1 + ### Unchanged - Ruby 3.2+, Capybara `>= 2, < 4`, the `screenshot` / `assert_matches_screenshot` DSL, every capture and comparison option, baseline file names and formats diff --git a/README.md b/README.md index b9757ae0..8c5a0b4e 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ Stop shipping UI bugs. Take screenshots in your Capybara tests, commit baselines > > **2.1 removes what 2.0 warns about**: the legacy namespaces, the ChunkyPNG driver, `shift_distance_limit`, the `driver:` setting and the driver abstraction — libvips becomes the only backend. There is no 3.0. Writing new code? Start from [SnapDiff — the canonical API](docs/snapdiff.md), which uses canonical names only. Migrating an existing suite? See the [upgrade guide](docs/UPGRADING.md). > -> **Two gem names, one gem.** `capybara-screenshot-diff` is the name to install. The same content is also published as [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara) — identical version, forward-looking name matching this repository — so that name is reserved and resolvable. **Install one, never both**: with both in a Gemfile the gem raises `SnapDiff::DualInstallError` at require time. +> **Two gem names, one gem — install `capybara-screenshot-diff`.** From 2.0.0 on, the identical content is also published as [`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara), the forward-looking name matching this repository. Do not reach for it yet: that name's only non-prerelease before 2.0.0 is a `0.0.1` placeholder containing a README and no Ruby files, so an unpinned `gem "snap_diff-capybara"` installs an empty gem and fails with `LoadError`. **Always pin the version**, and **install one name, never both** — with both in a Gemfile the gem raises `SnapDiff::DualInstallError` at require time. ## Quick Start (5 minutes) @@ -23,8 +23,8 @@ Stop shipping UI bugs. Take screenshots in your Capybara tests, commit baselines ```ruby # Gemfile -gem 'capybara-screenshot-diff' -gem 'ruby-vips' # The image backend. Needs libvips — see Installation below +gem 'capybara-screenshot-diff', '~> 2.0' # pin: unpinned resolves to the 1.x line +gem 'ruby-vips' # The image backend. Needs libvips — see Installation below ``` The gem ships no image backend of its own. Add `ruby-vips` (recommended, and the only @@ -107,7 +107,7 @@ Screenshot does not match for 'homepage': ({"area_size":1250,"region":[0,19,199,83],"max_color_distance":42.5}) ``` -Open `doc/screenshots/homepage.diff.png` to see exactly what changed. If the change is intentional, delete the baseline and re-run to update it. +Open `doc/screenshots/homepage.diff.png` to see exactly what changed. If the change is intentional, see [Accepting an intentional change](#accepting-an-intentional-change). | File | Description | |------|-------------| @@ -115,6 +115,38 @@ Open `doc/screenshots/homepage.diff.png` to see exactly what changed. If the cha | `homepage.diff.png` | Visual diff with changes highlighted in red | | `homepage.heatmap.diff.png` | Heatmap of pixel differences | +## Accepting an intentional change + +**Baselines are read from git, not from your working directory.** Every comparison runs +`git show HEAD:` for the baseline, so a screenshot that is committed is the one you +are compared against — no matter what the file on disk says. + +That makes the obvious move the wrong one: **deleting the baseline file does nothing.** The +gem fetches the committed copy from `HEAD` and the test fails exactly as before. + +Accepting a change is therefore a **commit**, not a file operation. The run writes its new +capture to the baseline path, so `git status` shows the baseline as modified — review it and +commit it: + +```bash +git status # doc/screenshots/homepage.png is modified +git diff --stat doc/screenshots/ + +# Look at homepage.diff.png. If the change is what you wanted: +git add doc/screenshots/homepage.png +git commit -m "chore: update homepage baseline" + +bundle exec rake test # now green — HEAD holds the new baseline +``` + +> **Staging is not enough.** `git add` alone does not move `HEAD`, so a staged-but-uncommitted +> baseline is still compared against the old committed one. You cannot get a green local run +> until you commit. That is by design: the baseline under review in a pull request is exactly +> the baseline the suite uses. + +Reviewing the change is what the pull request is for — the updated `.png` shows up as an image +diff next to the code that caused it. + ## Web UI for Reviewing Screenshot Changes Add one line to get an interactive dashboard for reviewing all screenshot differences: @@ -174,7 +206,7 @@ Yes. First run saves baselines and always passes. Run tests again to compare aga
How do I update baselines after intentional UI changes? -Delete the baseline file and re-run tests: `rm doc/screenshots/homepage.png && bundle exec rake test`. Or update all: `rm -rf doc/screenshots/ && bundle exec rake test`. +**Not by deleting the file** — baselines are read from git (`git show HEAD:`), so `rm` has no effect on what you are compared against. Commit the new capture instead: `git add doc/screenshots/homepage.png && git commit`. See [Accepting an intentional change](#accepting-an-intentional-change).
diff --git a/capybara-screenshot-diff.gemspec b/capybara-screenshot-diff.gemspec index efa095d5..01853679 100644 --- a/capybara-screenshot-diff.gemspec +++ b/capybara-screenshot-diff.gemspec @@ -20,6 +20,7 @@ Gem::Specification.new do |spec| spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/master/CHANGELOG.md" spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues" spec.metadata["documentation_uri"] = "#{spec.homepage}/blob/master/docs/snapdiff.md" + spec.metadata["rubygems_mfa_required"] = "true" # Allow-list: everything a consumer needs at runtime plus the shipped docs. # Build/dev files (gems.rb, Rakefile, the gemspec itself, tests, CI) stay out. spec.files = `git ls-files -z`.split("\x0") diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index b9fd5f65..8e777e85 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -25,8 +25,14 @@ with canonical names only, no legacy shapes to unlearn. gem "capybara-screenshot-diff", "~> 2.0" ``` -The same content is also published as `snap_diff-capybara`. Install **one** — with both -in a Gemfile the gem raises `SnapDiff::DualInstallError` at require time. +**Pin the version.** An unpinned `gem "capybara-screenshot-diff"` resolves to the 1.x line, +not to 2.0. + +The same content is also published as `snap_diff-capybara` from 2.0.0 on, but that name's +only earlier non-prerelease is a `0.0.1` placeholder with no Ruby files in it — unpinned, it +installs an empty gem and raises `LoadError`. Stay on `capybara-screenshot-diff`, and install +**one** name: with both in a Gemfile the gem raises `SnapDiff::DualInstallError` at require +time. ```bash bundle install @@ -48,6 +54,7 @@ The implementation now lives in `lib/snap_diff/` under the `SnapDiff` namespace. | `Capybara::Screenshot::Diff::ImageCompare` | `SnapDiff::Comparison` | | `Capybara::Screenshot::Diff::Difference` | `SnapDiff::ComparisonResult` | | `Capybara::Screenshot::Diff::Drivers::BaseDriver` | `SnapDiff::Driver` (now a mixin — see below) | +| `Capybara::Screenshot::Os` | `SnapDiff::Os` | | `CapybaraScreenshotDiff::SnapManager` / `::Snap` | `SnapDiff::SnapManager` / `SnapDiff::Snap` | | `CapybaraScreenshotDiff::RED_RGBA` / `::ORANGE_RGBA` | `SnapDiff::RED_RGBA` / `SnapDiff::ORANGE_RGBA` | | `CapybaraScreenshotDiff::Minitest::Assertions` | `SnapDiff::Minitest::Assertions` | diff --git a/docs/migration-guide.md b/docs/migration-guide.md index 2859aedc..3d56e0a4 100644 --- a/docs/migration-guide.md +++ b/docs/migration-guide.md @@ -36,7 +36,7 @@ end **After (capybara-screenshot-diff):** ```ruby # Gemfile -gem 'capybara-screenshot-diff' +gem 'capybara-screenshot-diff', '~> 2.0' # test helper require 'capybara_screenshot_diff/minitest'