fix: the structural checker is found by a binary built somewhere else - #99
Merged
Conversation
oracle.Strict looked for its script beside its own source file, through runtime.Caller, and that path is compiled in. A test binary cross compiled on Windows and run in a Linux container therefore looked for a Windows directory, found nothing, and answered "not available" for every structural check in the suite. Nothing went red for it. An unavailable checker is a skip, and a skip reads like a check that ran, so the container runs had been reporting a green suite in which not one file was ever checked against the specification. It surfaced only when two guards started refusing to pass on zero checks rather than reporting a skip, and the blind spot was older than both of them. There is a second way in now: walk up from the working directory looking for the script at its place in the tree. A test binary runs from inside the tree it tests, which is the case the compiled in path cannot cover. That path is still tried first, so nothing changes on a machine that built its own binary. The first diagnosis was wrong and it is worth recording why. The failure said "0 file(s) decoded strictly by Python", so the container was given an image carrying Python - and nothing changed, because there were two causes and both were needed. The message named the outcome. A probe printing what the function returned named the cause: strictScriptPath answering nothing while LookPath answered /usr/local/bin/python. Three mutations, all caught. The guard asks the second way directly, because the compiled in path is chosen first on any machine that built the binary, so asking through Strict would prove the first way and never reach the second. The control beside it asks the real entry point and is broken by taking the interpreter away rather than either path, since breaking one path leaves the other answering. internal/oracle exists for tests and is linked into neither binary, so no shipped byte moves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
oracle.Strictlooked for its script beside its own source file, throughruntime.Caller, and that path is compiled in. A test binary cross compiledon Windows and run in a Linux container therefore looked for a Windows
directory, found nothing, and answered "not available" for every structural
check in the suite.
Nothing went red for it. An unavailable checker is a skip, and a skip reads like
a check that ran, so the container runs had been reporting a green suite in
which not one file was ever checked against the specification. It surfaced only
when two guards started refusing to pass on zero checks rather than reporting a
skip, and the blind spot was older than both of them.
The fix
A second way in: walk up from the working directory looking for the script at
its place in the tree. A test binary runs from inside the tree it tests, which
is the case the compiled in path cannot cover. The compiled in path is still
tried first, so nothing changes on a machine that built its own binary - and
nothing needs configuring anywhere.
The first diagnosis was wrong, and that is the part worth reading
The failure said
0 file(s) decoded strictly by Python, so the container wasgiven an image carrying Python - and nothing changed, because there were two
causes and both were needed. The message named the outcome. A probe printing
what the function returned named the cause:
Measured
ubuntu:24.04, with the fixpython:3-slim, with the fixpython:3-slim, without the fixThe second half lives in
tools/linux-check.py, outside this repository: theimage now carries an interpreter. Nothing here can hold that, and the regression
row says so rather than implying otherwise.
internal/oracleexists for tests and is linked into neither binary, so noshipped byte moves.
🤖 Generated with Claude Code