Skip to content

fix(tests): verify the semver cache works, not just that it exists - #52

Merged
oxyc merged 4 commits into
masterfrom
fix/test-semver-cache-integrity
Jul 30, 2026
Merged

fix(tests): verify the semver cache works, not just that it exists#52
oxyc merged 4 commits into
masterfrom
fix/test-semver-cache-integrity

Conversation

@oxyc

@oxyc oxyc commented Jul 30, 2026

Copy link
Copy Markdown
Member

3 of 6 composer-update test files were failing on master. All three had the same cause, and none of the symptoms pointed at it.

The bug

ensure_semver_vendor caches a composer/semver vendor dir and guarded it with:

if [ ! -f "$cache/vendor/autoload.php" ]; then ... build ... fi

Existence is too weak a test. An interrupted composer require leaves autoload.php and vendor/composer/ behind without vendor/composer/ClassLoader.php. The local cache had exactly that shape:

vendor/composer/  ->  autoload_real.php  autoload_static.php  installed.json  semver/
                      (no ClassLoader.php)

composer/semver was there. The autoloader's own class file wasn't.

Why it stayed broken

The guard was satisfied, so the cache was never rebuilt. Every run copied a broken autoloader into the test project, the PHP helpers fatally errored on the missing ClassLoader, and their non-JSON output reached jq as:

jq: parse error: Invalid numeric literal at line 2, column 8

That reads like a bug in the helper's JSON output, not a missing dependency — which is why it sat unfixed. And it's sticky: once the cache is in that state it stays broken for every future run, on that machine, forever.

The fix

The guard now exercises the cache instead of inspecting it:

_semver_cache_ok() {
  local c="$1"
  [ -f "$c/vendor/autoload.php" ] || return 1
  ( cd "$c" && PATH="$_ORIGINAL_PATH" php -r \
      'require "vendor/autoload.php"; exit(class_exists("Composer\\Semver\\Semver") ? 0 : 1);' \
  ) >/dev/null 2>&1
}

If it fails, the cache is deleted and rebuilt from scratch rather than built on top of a partial tree, and the loud-failure path now says a working bootstrap could not be produced.

Result

compute-min-safe-constraints.test.sh   PASS — 13 assertions
is-still-vulnerable.test.sh            PASS —  9 assertions
lib-functions.test.sh                  PASS — 47 assertions
no-widen-still-creates-pr.test.sh      PASS
open-or-refresh-pr.test.sh             PASS — 15 assertions
update-orchestration.test.sh           PASS — 31 assertions

SUMMARY: 6 file(s) passed, 0 failed

No assertion changed and no script under test changed — only the harness.

Why it matters

These are the tests covering compute-min-safe-constraints.php and is-still-vulnerable.php — the code that decides which version the vuln scanner bumps you to, and whether a "fix" actually leaves the affected range. Half the suite being red meant a regression there would not have been caught.

This is the second bug of this class in the same helper; the first was ensure_semver_vendor shelling out to the fake composer shim (#41).

🤖 Generated with Claude Code

test and others added 4 commits July 30, 2026 14:05
npm install is the largest remaining cost in these jobs, and it is almost
entirely the audit — a network round trip over the whole dependency tree on
every run, even when node_modules was restored intact:

  snellmanrecipes  up to date, audited 1716 packages in 48s
  beamex           up to date, audited 1880 packages in 60s
  kaskipuu         up to date, audited 1930 packages in 1m

Locally on a 1250-package tree: 9s with audit, 1s without.

Set via npm config rather than CLI flags so it covers every npm invocation,
including the ones inside projects' composer build scripts, which is where
these installs actually happen.

Dependency vulnerabilities are covered by the nightly vulnerability scan, which
gates and opens PRs. npm's install-time audit summary gates nothing.

The internal setup ref is temporarily pointed at this branch to verify; that is
reverted before merge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three of six composer-update test files were red on master. All three had the
same cause, and none of the symptoms pointed at it.

ensure_semver_vendor caches a composer/semver vendor dir and guarded it with

    if [ ! -f "$cache/vendor/autoload.php" ]; then ... build ... fi

Existence is too weak. An interrupted `composer require` leaves autoload.php
and vendor/composer/ behind WITHOUT vendor/composer/ClassLoader.php — the local
cache here had exactly that shape, with composer/semver present but the
autoloader's own class file missing. The guard was satisfied, so the cache was
never rebuilt: every run copied a broken autoloader into the test project, the
PHP helpers fatally errored on the missing ClassLoader, and their non-JSON
output reached jq as

    jq: parse error: Invalid numeric literal at line 2, column 8

which reads like a bug in the helper's JSON, not a missing dependency. The
partial cache is also sticky — once in that state it stays broken across every
future run.

Now the guard actually exercises the cache: require the autoloader and check
Composer\Semver\Semver resolves. If it does not, the cache is deleted and
rebuilt from scratch rather than built on top of a partial tree, and the
loud-failure path reports a *working* bootstrap could not be produced.

Suite goes from 3 of 6 files failing to 6 of 6 passing, with no change to any
assertion or to the scripts under test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This branch carried

    uses: generoi/github-actions/setup@perf/npm-no-audit

which is #53's verification branch, picked up from master when this branch was
cut. #53 has since merged, so the ref is both unnecessary and wrong: leaving it
would point every consumer of the shared test workflow at a feature branch
instead of the release tag.

Restored from master. Nothing else in this branch touches test.yml — the actual
change here is confined to composer-update/tests/lib.sh.

master and the v1/v2 tags were checked and both already carry @v1, so this
never reached downstream.

Second instance of this pattern; the first was a "tmp: point internal setup ref
at this branch for verification" commit carrying setup@fix/drop-vendor-cache,
caught and reverted before #50 was opened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he-integrity

* origin/master:
  perf(setup): disable npm audit and fund in CI (#53)
@oxyc
oxyc merged commit f73b34b into master Jul 30, 2026
1 check passed
@oxyc
oxyc deleted the fix/test-semver-cache-integrity branch July 30, 2026 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants