fix: the five sinks nothing checked, and the claim that missed them - #222
Merged
Conversation
A verification pass sabotaged all nine redaction call sites one at a time.
Four failed exactly one case each. Five shipped GREEN — including sendJson,
whose own comment calls itself a sink, and the jobs/run.ts catch path added
one commit earlier to fix round eight.
My commit and the CHANGELOG both said removing any one sink's redaction
fails that sink's case. That was false, in the sentence describing the fix
for a test that could not fail. Same overclaim, one level up.
The five are not dead code. No reachable input carries a credential through
them today because something upstream already removed it — the buffered
HTTP path serves result.output read back from result.json, which writeJson
cleaned on the way to disk. That is the design working, and it is why the
guards must stay: scrubbing at sinks rather than sites means a NEW path
reaching one is covered without anyone thinking about it.
So they get the check that fits what they are. A behavioural test cannot
fail for them without a reachable secret, and inventing one would be a test
passing for the wrong reason — the defect that started this. An inventory
now asserts each call is still present, which catches a removal, and names
every sink in one list so a new one has somewhere to go. Verified by
removing three of the previously-invisible guards: each is caught now.
Also from the pass:
- span.recordException and setStatus put raw error text into OTLP spans that
leave the process. The one egress path outside the eight. Opt-in and
localhost by default, which is why it was low rather than nothing.
- warnCredentialInUrlPath exempted every JWT (the `!includes(".")` filter)
and every all-alphabetic token (the digit requirement), so those shapes
got neither redaction nor a warning.
Confirmed unchanged by the pass and left alone: a ${VAR} on a non-credential
field is not redacted, every credential shape still is, and the warning does
not fire for /v1, /v1beta/openai or /openai/deployments/gpt-4-turbo-preview.
npm run check: 1253 passed, 8 skipped, exit 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A verification pass sabotaged all nine redaction call sites one at a time.
mcp/tools.tsjsonTextdispatch-log.tsjobs/store.tswriteJsonhttp/server.tswriteSsehttp/server.tssendJsonmcp/resources.ts(both)jobs/run.ts(three writes)My commit and the CHANGELOG both said "removing the redaction from any one sink fails that sink's case and no other". That was false — in the sentence describing the fix for a test that could not fail. Same overclaim, one level up. One of the green ones is the
jobs/run.tscatch path added a commit earlier to fix round eight.Why the five stay
They are not dead code. No reachable input carries a credential through them today, because something upstream already removed it — the buffered HTTP path serves
result.outputread back fromresult.json, whichwriteJsoncleaned on the way to disk. That is the design working, and it is exactly why the guards must stay: scrubbing at sinks rather than sites means a new path reaching one is covered without anyone thinking about it.The check that fits what they are
A behavioural test cannot fail for them without a reachable secret, and inventing one would be a test that passes for the wrong reason — the defect that started this whole sequence. So an inventory asserts each call is still present. That catches a removal, which is the real risk for a guard, and it names every sink in one list so a new one has somewhere to go. Each entry is marked proven or guard, against the sabotage matrix rather than against intent.
Verified by removing three of the previously-invisible guards: each is caught now.
Also from the pass
span.recordException/setStatusput raw error text into OTLP spans that leave the process — the one egress path outside the eight. Opt-in and localhost by default, which is why it was low rather than nothing.warnCredentialInUrlPathexempted every JWT (via its!includes(".")filter) and every all-alphabetic token (via its digit requirement), so those shapes got neither redaction nor a warning.Confirmed by the pass and left alone
A
${VAR}on a non-credential field is not redacted; every credential shape still is (inline key,${VAR}key,api_keys:entries including ones after the first and ones overridden inline, URL userinfo, credential-named query values); the warning does not fire for/v1,/v1beta/openaior/openai/deployments/gpt-4-turbo-preview.npm run check: 1253 passed, 8 skipped, exit 0.Still open
MCP JSON-RPC error frames bypass every redacting sink on the HTTP transport (structural; the pass could not construct one carrying a credential). The Windows
workspace-resolvegit-spawn flake, measured at roughly 1 run in 11.