diff --git a/BACKLOG.md b/BACKLOG.md index 41f518d..02bc04f 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -48,11 +48,10 @@ records, generated release manifests, or the owning docs named above. | Status | ID | Scope | Completion condition | |---|---|---|---| -| NEXT | COVERAGE-01 | Replace command proof-route candidates with an owner-admitted executable oracle ledger; static route records are explicitly non-semantic. | `CommandCoverageInventory` consumes a separate execution-backed owner ledger whose rows bind `commandRef`, selector, concrete falsification event, assertion oracle, expected public outcome, and owner invariant; an independently authored versioned counterfeit corpus covers every shipped policy, evidence class, identity coordinate, and single or correlated substitution axis with positive controls and exact expected decisions; until then route metadata, prose, legacy source markers, test existence, and failure-capable AST nodes emit only declared routes or `proof_route_candidate`, candidate-route closure remains blocking, and semantic execution evidence remains an explicit non-claim. | | NEXT | COMPACT-01 | Replace compact proof caller labels and synthetic counts with an honest declaration-only schema/profile. | One atomic schema/profile cutover prefixes caller-owned proof and mutation fields with `declared`, removes checked/finding counts without independent evidence owners, rejects collapsed witness roles, updates every compact producer and consumer, and proves exact role and round-trip closure without claiming execution or assurance. | | NEXT | SOURCE-MODEL-01 | Define one representation-neutral typed requirement-source v2 model before selecting a source syntax. | A private bounded model admits atomic requirement identities, grouped authoring, premises, scenarios, definitions, vocabulary, lifecycle, references, and deterministic normalization; an independently authored field/variant completeness manifest plus mutant corpus proves every normative field reaches each required downstream owner, with no production parser or persisted normalized mirror. | | BLOCKED | SOURCE-CODEC-01 | Select at most one compact source codec without creating dual authority. | After `SOURCE-MODEL-01`, one versioned experiment manifest freezes disjoint role sets: the flat-v1 baseline control, grouped-model ablations, and exactly complete grouped-JSON plus at most one complete restricted-DSL codec candidate over the same model. Only codec candidates can win the predeclared replacement relation; controls and ablations measure causality and cannot become production grammars. A newly discovered candidate requires a new manifest version and complete experiment. A frozen corpus and strict `Replace(candidate, grouped-json)` predicate cover grammar completeness, safety, semantic parity, diagnostics, canonical bytes, review accuracy, token cost, diff amplification, parse/format cost, and unknowns. Every metric is classified exactly once by a versioned registry with role, direction, baseline pair, aggregation, material threshold, primary decision requirement, and missing-observation semantics; duplicate or unclassified metrics fail admission, hard constraints cannot trade off, report-only metrics cannot decide replacement, promised byte/token reductions must be materially better, and bounded diff/parse costs must be noninferior. If grouped JSON fails its hard gate, retain the current flat v1 source and perform no v2 cutover; otherwise select the restricted text candidate only when it is the unique strict replacement, while a tie, unknown, incomparability, or non-material improvement selects grouped JSON. The losing parser and formatter are deleted before experiment closeout, and production admits exactly one grammar. | -| BLOCKED | SOURCE-CUTOVER-01 | Migrate self-hosted requirement sources only after one codec, the typed v2 model, nested structural contracts, and the complete evidence counterfeit corpus pass their gates. | `COVERAGE-01` and `SCHEMA-01` are complete; a digest-bound clause ledger proves representation-only equality or owner-reviewed semantic decomposition for every legacy requirement; all bindings/scenarios/contracts/context/diff/graph/browser owners cut over atomically; v1 admission and the losing codec are removed; and active-v1 inventory is zero. | +| BLOCKED | SOURCE-CUTOVER-01 | Migrate self-hosted requirement sources only after one codec, the typed v2 model, nested structural contracts, and the complete evidence counterfeit corpus pass their gates. | The `REQ-PROOFKIT-QUALITY-010` execution-backed command-oracle closure and `SCHEMA-01` are complete; a digest-bound clause ledger proves representation-only equality or owner-reviewed semantic decomposition for every legacy requirement; all bindings/scenarios/contracts/context/diff/graph/browser owners cut over atomically; v1 admission and the losing codec are removed; and active-v1 inventory is zero. | | BLOCKED | SCHEMA-01 | Replace root-shape-only public contracts with one independent complete nested structural-contract owner. | A versioned schema owner covers nested fields, variants, cardinalities, bounds, enums, defaults, duplicate and unknown-field policy, and cross-field constraints; generated artifacts pass parity against an independently authored completeness manifest and mutant corpus without becoming semantic or policy authority. | | BLOCKED | SOURCE-PILOT-01 | Validate the selected source-v2 model and agent routing against heterogeneous external repositories without mutating them. | At least two independent repository classes complete no-push dual runs whose frozen inputs compare incumbent and candidate mapping, diagnostics, token cost, authoring accuracy, proof-route gaps, and rollback; unresolved parity or authority gaps keep incumbent owners active. | | BLOCKED | GOVERNANCE-01 | Evaluate a generic explicit-inventory governance-observation command without promoting one consumer's policy into Proofkit; detailed candidate contract is retained in [issue #64](https://github.com/research-engineering/agentic-proofkit/issues/64). | A sanitized reproducible fixture detects one named failure without classifying its false-positive counterexample, existing owners are proven insufficient, and either a second independent consumer reproduces the predicate or the owner admits recurring first-consumer cost; otherwise retire the candidate. | diff --git a/docs/proofkit-contract-map.md b/docs/proofkit-contract-map.md index 501c154..b565634 100644 --- a/docs/proofkit-contract-map.md +++ b/docs/proofkit-contract-map.md @@ -46,7 +46,7 @@ owner boundaries. It is not a second command-family inventory. | Selective planning | `changed-path-set`, `requirement-impact-input-compose`, `impact`, `selective-gate-plan`, `selective-gate-evidence`, `selective-gate-obligation-decision-input`, `proof-obligation-algebra`, `obligation-decision` | changed paths, base/current requirement sources, base/current single-binding-per-requirement proof contracts, generated-artifact policy, local environment policy, proof-like path policy, scan obligation ownership, planned receipts, obligation routes, obligation algebra records | fail-closed impact input composition, fail-closed planning, receipt comparison, obligation algebra admission, bounded agent packets | git diff truth, repository scanning, command execution, producer trust, final admission | composed impact input, plan, evidence report, obligation algebra report, or obligation input | | Receipts and producers | `proof-receipt-admission`, `receipt-producer-admission`, `receipt-currentness-scope`, `receipt-trust-class`, `producer-policy-self-proof` | receipt sets, producer policy, scope/currentness facts, trust classes | receipt shape, producer/receipt compatibility, self-proof diagnostics | producer authentication, freshness policy, CI trust roots | receipt/provenance report | | Release and deployment | `release-authority`, `external-consumer`, `registry-consumer-proof-input-compose`, `registry-consumer`, `deployment-evidence-admission`, `completion-criteria`, `branch-authority`, `readiness-closeout` | package facts, tarball/registry facts, explicit primitive registry/install/smoke facts, deployment evidence, criteria, branch facts | artifact/channel boundary checks, registry-consumer input composition, release diagnostics, falsifiable criteria shape | package publication, registry fetch, package-manager execution, deployment, rollback, approval | composed input, release/deployment/readiness report | -| Supply-chain and quality | `self-check`, release workflow, `npm run release:sbom`, `npm run self:coverage`, `npm run go:actionlint`, `npm run go:bench` | release artifacts, source workflows, specs, bindings, witness plans, explicit benchmark invocation | deterministic self-check report shape, SBOM candidate evidence, coverage metrics, workflow lint routing, benchmark entrypoints | public-source provenance, vulnerability triage, license approval, CI run admission, release approval | self-check report, SBOM, metrics report, CI signal, or benchmark output | +| Supply-chain and quality | `self-check`, release workflow, `npm run release:sbom`, `npm run self:coverage`, `npm run go:actionlint`, `npm run go:bench` | release artifacts, source workflows, specs, bindings, witness plans, command-oracle candidate inventory, independently authored counterfeit corpus, explicit benchmark invocation | deterministic self-check report shape, SBOM candidate evidence, exact-file materialized-snapshot package-scoped selected-test execution, production-owner counterfeit decisions, strict one-read and current-owner command-oracle diagnostic admission, execution-backed coverage metrics, workflow lint routing, benchmark entrypoints | assertion-branch execution, mutation adequacy, producer authentication, public-source provenance, vulnerability triage, license approval, CI run admission, release approval | self-check report, SBOM, command-oracle diagnostic, metrics report, CI signal, or benchmark output | The `npm run release:sbom`, `npm run self:coverage`, `npm run go:actionlint`, and `npm run go:bench` routes above are maintainer commands for a source diff --git a/docs/release-process.md b/docs/release-process.md index fcf3101..134df82 100644 --- a/docs/release-process.md +++ b/docs/release-process.md @@ -128,7 +128,16 @@ static analysis, workflow linting, vulnerability checks, npm package artifact creation, package artifact verification, Python wheel artifact creation, Python wheel verification, release SBOM, release manifest and checksum generation, outside-consumer binary smoke proof, self-hosting receipt validation, and -coverage metrics generation. +coverage metrics generation. Coverage generation runs the exact owner-selected +Go tests through package-scoped argv vectors from an exact-file materialized +source snapshot, evaluates the independently authored counterfeit corpus +through production admission owners, and emits a command-oracle diagnostic +plus coverage metrics v2 only while the candidate, corpus, runtime, and source +identities remain current. Release closeout re-admits that diagnostic through +its sole strict owner, binds its digest to the same canonical byte read, and +revalidates current producer reachability. This local cooperative execution +does not prove assertion-branch execution, mutation adequacy, producer +authentication, or provider admission. The dry-run package identity proves candidate tarball shape only. It does not prove the bytes served by the registry after publish. diff --git a/docs/specs/proofkit-supply-chain-quality/overview.md b/docs/specs/proofkit-supply-chain-quality/overview.md index 87a81e9..18945f8 100644 --- a/docs/specs/proofkit-supply-chain-quality/overview.md +++ b/docs/specs/proofkit-supply-chain-quality/overview.md @@ -58,15 +58,20 @@ vulnerability absence, or consumer rollout safety by itself. - `REQ-PROOFKIT-QUALITY-009`: performance-sensitive parser and serializer paths expose benchmark entrypoints without making wall-clock budgets a required PR gate before stable baselines exist. -- `REQ-PROOFKIT-QUALITY-010`: coverage metrics report requirement, binding, - witness, CLI inventory linkage, and descriptor-owned command proof-route - candidates from admitted test-evidence-inventory rows, while critical - anti-vacuity scenarios retain exact closed selector inventories. Each linkage and route - conjunct has an independent fail-closed falsifier, and each source-checkout - selector resolves to a valid function in an active Go test file with its - exact executable command; static route metadata, prose, source markers, test - existence, and failure-capable syntax never become semantic falsifier - evidence. +- `REQ-PROOFKIT-QUALITY-010`: coverage metrics keep static proof-route + candidates separate from an execution-backed command-oracle ledger. The + ledger runs exact selected Go tests through package-scoped argv vectors from + one exact-file materialized source snapshot, joins reserved lifecycle + attributes to every candidate identity, terminates bounded subprocesses on + cancellation or output overflow, confines atomic artifact publication and + invalidation to non-symlink repository paths, and fails closed on incomplete + command coverage or event, source, selection, producer-reachability, and + identity drift. A versioned counterfeit corpus owns checked-in expected + decisions for every required policy axis, evidence class, record coordinate, + and substitution axis; its mutations execute the production admission and + lifecycle owners rather than a generated expectation copy. Passing selected + tests does not prove assertion-branch execution, mutation adequacy, or + exhaustive command semantics. - `REQ-PROOFKIT-QUALITY-011`: CI separates the OS-independent full source/package gate from macOS platform smoke, executes the complete Go package set through its owner command, uses explicit hosted runner labels @@ -102,12 +107,11 @@ vulnerability absence, or consumer rollout safety by itself. fields after validation. - `REQ-PROOFKIT-QUALITY-015`: the package gate includes an admitted release closeout completion-criteria report so unit tests alone cannot satisfy - release closeout, and coverage-metric re-admission uses overflow-safe exact - producer relations plus the complete command inventory, and compares both - coverage command projections with the actual `cli-contract.v2.json` command - inventory from the same source snapshot, so neither an impossible count - partition nor a coordinated same-size command substitution can satisfy - closeout. + release closeout. Coverage re-admission rejects candidate-only v1, requires + overflow-safe exact producer relations plus the complete command inventory, + and binds coverage v2 to the current command-oracle diagnostic through the + ledger owner's one-read canonical admission, current-owner revalidation, and + record, candidate-set, corpus, revision, and source-snapshot digests. - `REQ-PROOFKIT-QUALITY-016`: release platform targets use one private owner that projects platform suffixes, Go build targets, npm OS/CPU metadata, package tar entries, Python wheel tags, PyPI candidate completeness, diff --git a/docs/specs/proofkit-supply-chain-quality/requirements.v1.json b/docs/specs/proofkit-supply-chain-quality/requirements.v1.json index 399b71b..7ff411d 100644 --- a/docs/specs/proofkit-supply-chain-quality/requirements.v1.json +++ b/docs/specs/proofkit-supply-chain-quality/requirements.v1.json @@ -129,12 +129,12 @@ { "requirementId": "REQ-PROOFKIT-QUALITY-010", "ownerId": "proofkit.supply-chain-quality", - "invariant": "Coverage metrics report requirement, binding, witness, CLI inventory linkage, and descriptor-owned command proof-route candidates from admitted test-evidence-inventory rows, failing closed independently for each requirement/proof linkage dead-zone class and each missing-candidate, unknown-candidate-ref, unknown-declared-semantic-route-ref, contract-only, or route-only command-route class; source-checkout witness selectors must resolve to valid functions in active Go test files with exact executable commands and a failure-capable assertion candidate outside literal-dead branches, uninvoked closures, and skip-bearing helper paths, critical anti-vacuity scenarios must retain their exact closed selector inventories, and failed command-route inventory admission also fails closed, while static route metadata, prose, source markers, test existence, and failure-capable syntax remain candidate evidence and cannot become execution-backed semantic evidence.", + "invariant": "Coverage metrics report requirement, binding, witness, CLI inventory linkage, descriptor-owned command proof-route candidates, and a separate execution-backed command-oracle ledger. The ledger must bind every candidate commandRef, selector, concrete negative case, falsification event, assertion oracle, expected public outcome, and owner invariant to one immutable exact-file materialized source snapshot; execute exactly the selected active Go tests through canonically ordered package-scoped argv vectors that cannot select same-named non-candidate tests in another package; require unique package start and pass events, unique test run and pass events, owner-reserved cooperative attributes for every candidate, bounded cancellable subprocesses, and immediate process-group termination on output-limit violation; reject missing, unknown, duplicate, skipped, failed, reordered, context-invalid, spoofed-output, cross-test, source-drift, producer-unreachable candidate projections, and correlated identity substitutions; publish and invalidate local diagnostic and metrics artifacts atomically through repository-confined non-symlink destinations; revalidate current candidate, corpus, runtime, and source identities before and after artifact publication; and fail closed when any public command lacks execution-backed evidence. An independently authored versioned counterfeit corpus must bijectively cover every shipped evidence class, required policy axis, record coordinate, and declared substitution axis with positive controls and checked-in exact expected decisions evaluated through the production admission and lifecycle owners rather than a generated expectation registry. Static route metadata, prose, legacy source markers, test existence, and failure-capable syntax remain proof-route candidates and cannot satisfy the execution-backed closure.", "claimLevel": "blocking", "riskClass": "medium", "proofBindingRefs": ["proofkit/requirement-bindings.json"], "nonClaimRefs": ["NC-PROOFKIT-QUALITY-010"], - "nonClaims": ["This requirement does not claim semantic command coverage, line coverage, dead-code absence, exhaustive semantic completeness for any test, native witness execution, receipt freshness, or merge satisfaction."], + "nonClaims": ["Successful selected tests do not prove assertion-branch execution, mutation adequacy, line coverage, dead-code absence, exhaustive command semantics, malicious-test resistance, producer authentication, receipt freshness, merge satisfaction, or production readiness."], "lifecycle": {"state": "active", "replacementRequirementIds": [], "evidenceRefs": []}, "deferral": null, "updatePolicy": {"reviewOwnerId": "proofkit.supply-chain-quality", "requiresImpactDeclaration": true, "requiresProofBindingReview": true} @@ -194,7 +194,7 @@ { "requirementId": "REQ-PROOFKIT-QUALITY-015", "ownerId": "proofkit.supply-chain-quality", - "invariant": "The package gate includes an admitted release closeout completion-criteria report that classifies package artifacts, Python wrappers, release manifest, SBOM, channel scope, and self-hosting evidence instead of allowing unit tests alone to satisfy release closeout; coverage-metric re-admission uses overflow-safe relations to require the exact producer projection between admitted entries, routes, route classes, proof-route candidates, the complete command inventory, and commands missing declaration-only routes, and compares both coverage command projections with the actual cli-contract.v2.json command inventory from the same source snapshot.", + "invariant": "The package gate includes an admitted release closeout completion-criteria report that classifies package artifacts, Python wrappers, release manifest, SBOM, channel scope, and self-hosting evidence instead of allowing unit tests alone to satisfy release closeout; coverage-metric re-admission rejects candidate-only v1 reports, uses overflow-safe relations to require the exact producer projection between admitted entries, routes, route classes, proof-route candidates, execution-backed command-oracle entries, the complete command inventory, and commands missing declaration-only routes, compares both coverage command projections with the actual cli-contract.v2.json command inventory, and binds the coverage report to a command-oracle diagnostic admitted by its sole owner from one bounded byte read, with the digest computed from those same canonical bytes and matched by candidate-set digest, counterfeit-corpus digest, source revision, and source-snapshot digest. Release closeout additionally revalidates that the diagnostic remains reachable from the current candidate, corpus, runtime, and source owners before admitting it.", "claimLevel": "blocking", "riskClass": "high", "proofBindingRefs": ["proofkit/requirement-bindings.json"], diff --git a/internal/app/app_test.go b/internal/app/app_test.go index aedd03f..6476b72 100644 --- a/internal/app/app_test.go +++ b/internal/app/app_test.go @@ -249,7 +249,7 @@ func TestDuplicateFormatIsRejectedBeforeInputRead(t *testing.T) { } func TestSelfCheckRejectsDuplicateKeys(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.061049109061347524269448772857617649849822202469664158122537165529475398131547") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.031584524343444160231600312979302740662395207275659075147050938676784401609593") var stdout bytes.Buffer var stderr bytes.Buffer status := Run(t.Context(), []string{"self-check", "--input", "-"}, strings.NewReader(`{"schemaVersion":1,"schemaVersion":2}`), &stdout, &stderr) diff --git a/internal/app/cli_abi_test.go b/internal/app/cli_abi_test.go index 1266896..920efb5 100644 --- a/internal/app/cli_abi_test.go +++ b/internal/app/cli_abi_test.go @@ -21,7 +21,7 @@ import ( ) func TestCLIABIGoldenCorpus(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.072652008964654291511814074625765716851058391430332230195872382093981424771505") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.019844608890608007981435607465632554695607593958274064855839894961505266938135") specPath := filepath.Join(repoRoot(t), "docs/specs/proofkit-package-boundary/requirements.v1.json") specContent, err := os.ReadFile(specPath) if err != nil { @@ -1071,7 +1071,7 @@ func TestOutputWriterRejectsDeterministicParentSwap(t *testing.T) { } func TestRequirementBrowserServerSpecTreeCLIABI(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.006501090000492450297502395866607916441745680520470690190439229497945905166304") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.053429100218927507111827924045562630374131695751215621621358073482708475349353") var input any if err := json.Unmarshal([]byte(cliRequirementSpecTreeInput()), &input); err != nil { t.Fatalf("root-oracle input must be JSON: %v", err) @@ -1225,7 +1225,7 @@ func waitForBrowserLauncherURL(t *testing.T, urlFile string, result <-chan int, } func TestAdoptionDoctorCLIABI(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.018061704360080936767110902235369680299779598229946624164723476412925521966788") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.069327963569220983248235596502245621103210642467353785655205406140014773697960") cases := []struct { name string args []string diff --git a/internal/app/cli_contract_test.go b/internal/app/cli_contract_test.go index 33abc5c..a773abb 100644 --- a/internal/app/cli_contract_test.go +++ b/internal/app/cli_contract_test.go @@ -22,7 +22,7 @@ import ( ) const ( - cliContractPublicABISHA256 = "6a3b6504546419e9198af89ea90af19d543dac26af975aeef1075b4ff7d88491" + cliContractPublicABISHA256 = "440f53b6433deef5e254d0d48665e79eda10cf4c0bceafe8acfedb42a6f46df6" maxAggregateFileReadBytesForContractTest = 64 << 20 maxPackageManifestBytesForContractTest = 256 << 10 maxSourceFileBytesForContractTest = 8 << 20 @@ -1335,7 +1335,7 @@ func stringsAsAny(values []string) []any { } func TestHelpCommandContractForms(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.012212946147973847974188673193955565304078130183905790171739464374424221304025") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.079695171929558035833123525411377372802577641283912049521939698994087384245655") for _, args := range [][]string{{"help"}, {"help", "--help"}, {"help", "-h"}, {"--help"}, {"-h"}, {"help", "repo-profile-admission"}, {"repo-profile-admission", "--help"}} { t.Run(strings.Join(args, " "), func(t *testing.T) { var stdout bytes.Buffer diff --git a/internal/app/command_contract_generated.go b/internal/app/command_contract_generated.go index 5aa3f98..3849f3b 100644 --- a/internal/app/command_contract_generated.go +++ b/internal/app/command_contract_generated.go @@ -1,7 +1,7 @@ // Code generated by internal/tools/commandcontractgen; DO NOT EDIT. package app -const commandContractSourceSHA256 = "3d2fab7d4b8da24fb1c105ce8814b531926c50dee9a594eacad953f8af897132" +const commandContractSourceSHA256 = "f73bd8e28f0855963568866141d1e67283b2b8cd2e931658e73e704db2cf19c0" type generatedCommandContractMetadata struct { InputContractSHA256 string @@ -38,7 +38,7 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "migration-plan": {InputContractSHA256: "sha256:58a62759a634101ce2ca9218184175134bbe5633328e1b23797b94c19fc9b11a", InputSchemaSummary: []string{"schemaVersion=1", "migrationId", "sourceProofOwners[]", "targetProofkitRefs[]", "parityEvidenceRefs[]", "retainedOwners[]", "retirementCandidates[]", "followUpCommands[]", "nonClaims[]", "root-shape-only definition proofkit.migration-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f14f0381e9dc241357c346315b95b03ef5b23f1d1bbc3b00f111fbe1515ed3ff", FlagChoices: map[string][]string{}}, "obligation-decision": {InputContractSHA256: "sha256:1dea2ed5c5066451d6d49b815cea99df2cdae2ef05d42fed16c8aeb45eb7f445", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.obligation-decision.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:96dc074f611bcc12e511bc803c548e4df623e2de869d3add29a3ea6386e04330", FlagChoices: map[string][]string{}}, "package-runtime-dependency-admission": {InputContractSHA256: "sha256:fc85887af9b8fcd899d245f0db30b2f2f68609822fc268126bf999082bb4115f", InputSchemaSummary: []string{"schemaVersion=1", "reportId", "expectedDependencySpec", "expectedLockfileIntegrity", "expectedPackageName", "expectedPackageVersion", "admissibleLocations{}", "packageResolution{}", "nonClaims[]", "root-shape-only definition proofkit.package-runtime-dependency-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:c012032e8c8212fd50bc2e85669cc610609ca2124ebc992c9e88f44a1ad2d5fc", FlagChoices: map[string][]string{}}, - "pilot-admission": {InputContractSHA256: "sha256:6c4d6fb7ba99cb8da806826584807a723891955c83c04d0054a848e303e5ca9a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.pilot-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:1603d385db8a493287b4394a3bdad99e027cb87cc043c6c42b132c2616b1a804", FlagChoices: map[string][]string{}}, + "pilot-admission": {InputContractSHA256: "sha256:6c4d6fb7ba99cb8da806826584807a723891955c83c04d0054a848e303e5ca9a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.pilot-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:1b69c3999d5af63ec99a73105525c90c2c9694f29217f80423dd7a35e989e4a8", FlagChoices: map[string][]string{}}, "producer-policy-self-proof": {InputContractSHA256: "sha256:d48e18826000c8d415f3c44b6c686e1da6ed962ef7ca36c9f705de8c68d034f9", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.producer-policy-self-proof.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e82a3989a743f8babc6069f7af82b1dd1ea62bad8dbb18d95e105b36f74e4276", FlagChoices: map[string][]string{}}, "proof-obligation-algebra": {InputContractSHA256: "sha256:4f176b6bc9bdbd0d96d65c071d66447d246665bda7a23269e7927f1d0b80b043", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.proof-obligation-algebra.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f9ee9e56b349756c55856a2dab198e1ad85db70a468c38e3aeca73cfe2ed66f6", FlagChoices: map[string][]string{}}, "proof-receipt-admission": {InputContractSHA256: "sha256:7cb4c4fb60c8b5a37109bbd8c00d567749f7d181bbc905d8bc58155f139c44cb", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.proof-receipt-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:3f802ac3fac6762ede51f0e0a151f16dc10b4a20344a3887b3ee8bae43ce94f2", FlagChoices: map[string][]string{}}, @@ -76,7 +76,7 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "selective-gate-evidence": {InputContractSHA256: "sha256:8aa178ab7ca7c475c23707bc4e15fd3f9f8d57acf6f6dcf279677e7769a45586", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.selective-gate-evidence.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:723569262bb85d9674b2a78d3bcb6e9f4cab229b71e8c784ff1b804a7fcade71", FlagChoices: map[string][]string{}}, "selective-gate-obligation-decision-input": {InputContractSHA256: "sha256:85761fcbc0ea94239d55bf379d0592a6ca814e6612a2d609a651f6cdaf8ca10a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.selective-gate-obligation-decision-input.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:ab9dddabe975238d7019266c43350afa2df1a61d4c2eb7bc23afd520b588a2da", FlagChoices: map[string][]string{}}, "selective-gate-plan": {InputContractSHA256: "sha256:5293a5a4c7d8426cf637e6f8d252095ca0eb1714365bb89bec83307b778c678a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.selective-gate-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:d7bffed853af5595af08b03859be01c283a3bdff1b3502d94ddc190889977647", FlagChoices: map[string][]string{}}, - "self-check": {InputContractSHA256: "sha256:dee7c7a0b11b427553814e3bc1925391118f079018c6c97b940cf9fc2ae8450f", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.self-check.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:7062eb2e46470c0bbcbdea1239e747dc3e3de76b782d3ea509caafe2d0653f29", FlagChoices: map[string][]string{}}, + "self-check": {InputContractSHA256: "sha256:347a12b80bfd182fced99f02d49358645aa1c839ac5c6603810d2d6a0ebbf40f", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.self-check.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:6d732343f74ddce1fd9f926fda52ca36378736af2b2bdf464fd87f4054c17433", FlagChoices: map[string][]string{}}, "spec-overview-claims": {InputContractSHA256: "sha256:2490dcd34ba7485e13f8f33e8a288a0463c4c52cc6b0d82c57777466927e49a4", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.spec-overview-claims.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:554f3a7020e9820ccb90672629fd769c52b2f298f356040aa3b0a817666cbfbf", FlagChoices: map[string][]string{}}, "spec-proof-bundle-admission": {InputContractSHA256: "sha256:6b6c2875b6476e63a1911e7d6112d9999df2babbee969f84abc4c9e4b470c933", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.spec-proof-bundle-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e9e0eb66cebca3b99fe5036fb2e7327a9284934ed76f58818d18094d0546fc52", FlagChoices: map[string][]string{}}, "stack-preset": {InputContractSHA256: "", InputSchemaSummary: []string(nil), OutputContractSHA256: "sha256:f495e9ade4e1e7af7a8f8b2059f7611cc016e6080363afccf76d8dfc2dbc6d2d", FlagChoices: map[string][]string{"--preset": []string{"agentic_runtime_repo", "generated_docs_contract_repo", "python_service", "python_typescript_service", "typescript_monorepo", "typescript_workspace"}}}, diff --git a/internal/app/command_coverage_oracle.go b/internal/app/command_coverage_oracle.go new file mode 100644 index 0000000..6819b1f --- /dev/null +++ b/internal/app/command_coverage_oracle.go @@ -0,0 +1,109 @@ +package app + +import ( + "fmt" + "path/filepath" + "sort" + "strings" +) + +type CommandCoverageOracleCandidate struct { + AssertionOracleID string `json:"assertionOracleId"` + CommandRef string `json:"commandRef"` + ExpectedPublicOutcome string `json:"expectedPublicOutcome"` + FalsificationEventID string `json:"falsificationEventId"` + NegativeCaseID string `json:"negativeCaseId"` + OracleKind string `json:"oracleKind"` + OwnerInvariantID string `json:"ownerInvariantId"` + PackagePath string `json:"packagePath"` + Selector string `json:"selector"` + SourceMarker string `json:"sourceMarker"` + SourcePath string `json:"sourcePath"` + TestID string `json:"testId"` + TestName string `json:"testName"` + WrongImplementationClass string `json:"wrongImplementationClassId"` +} + +func CommandCoverageOracleCandidates() ([]CommandCoverageOracleCandidate, error) { + root, err := repositoryRootFromWorkingDirectory() + if err != nil { + return nil, err + } + return CommandCoverageOracleCandidatesAtRoot(root) +} + +func CommandCoverageOracleCandidatesAtRoot(root string) ([]CommandCoverageOracleCandidate, error) { + commands := make([]string, 0, len(commandCoverageRoutes)) + for command := range commandCoverageRoutes { + commands = append(commands, command) + } + sort.Strings(commands) + candidates := []CommandCoverageOracleCandidate{} + seenTestIDs := map[string]struct{}{} + seenMarkers := map[string]struct{}{} + for _, command := range commands { + for _, route := range commandCoverageRoutes[command] { + if !route.isSemanticCandidate() { + continue + } + if problem := route.semanticProofProblem(); problem != "" { + return nil, fmt.Errorf("%s coverage route %s has invalid semantic proof metadata: %s", command, route.testName, problem) + } + if problem := routeSemanticOwnerProblem(command, route); problem != "" { + return nil, fmt.Errorf("%s coverage route %s has invalid semantic owner scope: %s", command, route.testName, problem) + } + if problem := routeSemanticSourceProblemAtRoot(command, route, root); problem != "" { + return nil, fmt.Errorf("%s coverage route %s has invalid source oracle: %s", command, route.testName, problem) + } + proof := route.semanticProof + candidate := CommandCoverageOracleCandidate{ + AssertionOracleID: proof.oracleID(), + CommandRef: CommandCoverageCommandRef(command), + ExpectedPublicOutcome: route.rationale, + FalsificationEventID: proof.falsifierID(), + NegativeCaseID: proof.negativeCaseID(), + OracleKind: "semantic_route_falsifier", + OwnerInvariantID: proof.semanticRouteInvariantID(), + PackagePath: "./" + filepath.ToSlash(filepath.Dir(route.file)), + Selector: route.file + "::" + route.testName, + SourceMarker: route.sourceOracleMarker(command), + SourcePath: route.file, + TestID: proof.routeTestID(), + TestName: route.testName, + WrongImplementationClass: proof.wrongImplementationClassID(), + } + if _, exists := seenTestIDs[candidate.TestID]; exists { + return nil, fmt.Errorf("command coverage oracle candidate has duplicate test identity") + } + if _, exists := seenMarkers[candidate.SourceMarker]; exists { + return nil, fmt.Errorf("command coverage oracle candidate has duplicate source marker") + } + seenTestIDs[candidate.TestID] = struct{}{} + seenMarkers[candidate.SourceMarker] = struct{}{} + candidates = append(candidates, candidate) + } + } + sort.Slice(candidates, func(left, right int) bool { + return strings.Join(candidateIdentity(candidates[left]), "\x00") < strings.Join(candidateIdentity(candidates[right]), "\x00") + }) + return candidates, nil +} + +func candidateIdentity(candidate CommandCoverageOracleCandidate) []string { + return []string{ + candidate.CommandRef, + candidate.Selector, + candidate.TestID, + candidate.OwnerInvariantID, + candidate.FalsificationEventID, + candidate.NegativeCaseID, + candidate.WrongImplementationClass, + candidate.AssertionOracleID, + candidate.OracleKind, + candidate.ExpectedPublicOutcome, + candidate.SourceMarker, + candidate.SourcePath, + candidate.PackagePath, + candidate.TestName, + } +} diff --git a/internal/app/command_coverage_routes.go b/internal/app/command_coverage_routes.go index a382c42..2e05673 100644 --- a/internal/app/command_coverage_routes.go +++ b/internal/app/command_coverage_routes.go @@ -9,9 +9,8 @@ import ( "strings" ) -// These private marker names predate the declaration-only public contract. -// Their projection remains proof_route_candidate until COVERAGE-01 supplies an -// independently admitted execution-backed oracle ledger. +// Static routes remain declaration-only candidates. The command-oracle owner +// separately joins them to source-bound cooperative runtime evidence. type commandCoverageRoute struct { file string kind string @@ -21,8 +20,7 @@ type commandCoverageRoute struct { } type commandCoverageSemanticProof struct { - ref string - expectedPublicOutcome string + ref string } type commandCoverageSourceOracleBinding struct { @@ -40,8 +38,6 @@ type commandCoverageSourceOracleBinding struct { ExpectedPublicOutcome string `json:"expectedPublicOutcome"` } -const commandCoverageExpectedPublicOutcome = "referenced owner test asserts the bound command's public pass/fail outcome, diagnostics, or emitted packet contract" - type CommandCoverageSummary struct { Command string CommandRef string @@ -58,106 +54,103 @@ var requiredInputAdmissionRoute = commandCoverageRoute{ } var commandCoverageRoutes = map[string][]commandCoverageRoute{ - "adoption-checklist": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/adoptionchecklist/adoptionchecklist_test.go", "TestBuildClassifiesRequiredChecklistItemsAndPreservesOptionalNonFailures", semanticRouteProof("adoptionchecklist.build_classifies_required_checklist_items_and_preserves_optional_non_failures", commandCoverageExpectedPublicOutcome), "Adoption checklist reports must fail missing, blocked, and not-applicable required items while preserving optional non-failures.")}, - "adoption-contract-envelope": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/adoptioncontract/adoptioncontract_test.go", "TestBuildDelegatesModesWithParity", semanticRouteProof("adoptioncontract.build_delegates_modes_with_parity", commandCoverageExpectedPublicOutcome), "Adoption contract envelope admission must prove aggregate-root admission while delegating selected modes to existing child command outputs without drift.")}, - "adoption-doctor": {requiredInputAdmissionRoute, directCLIRoute("internal/app/cli_abi_test.go", "TestAdoptionDoctorCLIABI", semanticRouteProof("cli_abi.adoption_doctor_cliabi", commandCoverageExpectedPublicOutcome), "Adoption doctor CLI ABI must emit stable report and agent-envelope JSON for admitted caller records."), packageFalsifierRoute("internal/command/adoptiondoctor/adoptiondoctor_test.go", "TestBuildFailsEnforcementForCandidateBoundaryAndMissingRoutes", semanticRouteProof("adoptiondoctor.build_fails_enforcement_for_candidate_boundary_and_missing_routes", commandCoverageExpectedPublicOutcome), "Adoption doctor reports must fail closed for enforcement modes when caller-provided owner routes or candidate boundaries are not admitted.")}, - "adoption-workflow-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/adoptionworkflow/adoptionworkflow_test.go", "TestBuildGeneratesBoundedCommandArgv", semanticRouteProof("adoptionworkflow.build_generates_bounded_command_argv", commandCoverageExpectedPublicOutcome), "Adoption workflow plans must generate bounded argv commands from admitted route refs.")}, - "agent-route": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/agentroute/agentroute_test.go", "TestBuildRoutesRequirementSourceAndBlocksUnknownGoal", semanticRouteProof("agentroute.build_routes_requirement_source_and_blocks_unknown_goal", commandCoverageExpectedPublicOutcome), "Agent route reports must select a deterministic command family from explicit caller-owned input and fail closed for unknown goals."), packageFalsifierRoute("internal/command/agentroute/agentroute_test.go", "TestBuildEnvelopeKeepsBlockedRoutesAsStopSignals", semanticRouteProof("agentroute.build_envelope_keeps_blocked_routes_as_stop_signals", commandCoverageExpectedPublicOutcome), "Agent route envelopes must preserve missing-input route states as stop signals instead of executable guidance."), packageFalsifierRoute("internal/command/agentroute/agentroute_test.go", "TestBuildEnvelopeCarriesBlockedObservedReportPreconditions", semanticRouteProof("agentroute.build_envelope_carries_blocked_observed_report_preconditions", commandCoverageExpectedPublicOutcome), "Agent route envelopes must preserve non-passed observed reports as blocked preconditions instead of executable guidance.")}, - "binding-partition": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/bindingpartition/bindingpartition_test.go", "TestBuildRejectsCrossSurfaceRouteReferenceWithoutDelegation", semanticRouteProof("bindingpartition.build_rejects_cross_surface_route_reference_without_delegation", commandCoverageExpectedPublicOutcome), "Binding partition admission must reject undelegated cross-surface proof route references.")}, - "branch-authority": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/branchauthority/branchauthority_test.go", "TestBuildAdmitsAlignedRequiredBranchAndRejectsRequiredDrift", semanticRouteProof("branchauthority.build_admits_aligned_required_branch_and_rejects_required_drift", commandCoverageExpectedPublicOutcome), "Branch authority must reject required branch drift.")}, - "capability-map-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/capabilitymapadmission/capability_map_admission_test.go", "TestBuildCodeBaselineFailsMissingCandidateRequirementAndAnchor", semanticRouteProof("capabilitymapadmission.build_code_baseline_rejects_missing_candidate_or_anchor", commandCoverageExpectedPublicOutcome), "Capability map admission must fail code_baseline mode when candidate requirement ids or active scenario anchors are missing, while keeping outputs candidate-only.")}, - "changed-path-set": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/changedpathset/changedpathset_test.go", "TestBuildDeduplicatesAndFailsClosedOnInvalidPaths", semanticRouteProof("changedpathset.build_deduplicates_and_fails_closed_on_invalid_paths", commandCoverageExpectedPublicOutcome), "Changed path set must deduplicate caller path sources and fail closed with redacted invalid-path diagnostics.")}, - "completion-criteria": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/completioncriteria/completioncriteria_test.go", "TestBuildBlocksUnsatisfiedBlockingCriterion", semanticRouteProof("completioncriteria.build_blocks_unsatisfied_blocking_criterion", commandCoverageExpectedPublicOutcome), "Completion criteria must fail when a blocking criterion is not satisfied.")}, + "adoption-checklist": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/adoptionchecklist/adoptionchecklist_test.go", "TestBuildClassifiesRequiredChecklistItemsAndPreservesOptionalNonFailures", semanticRouteProof("adoptionchecklist.build_classifies_required_checklist_items_and_preserves_optional_non_failures"), "Adoption checklist reports must fail missing, blocked, and not-applicable required items while preserving optional non-failures.")}, + "adoption-contract-envelope": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/adoptioncontract/adoptioncontract_test.go", "TestBuildDelegatesModesWithParity", semanticRouteProof("adoptioncontract.build_delegates_modes_with_parity"), "Adoption contract envelope admission must prove aggregate-root admission while delegating selected modes to existing child command outputs without drift.")}, + "adoption-doctor": {requiredInputAdmissionRoute, directCLIRoute("internal/app/cli_abi_test.go", "TestAdoptionDoctorCLIABI", semanticRouteProof("cli_abi.adoption_doctor_cliabi"), "Adoption doctor CLI ABI must emit stable report and agent-envelope JSON for admitted caller records."), packageFalsifierRoute("internal/command/adoptiondoctor/adoptiondoctor_test.go", "TestBuildFailsEnforcementForCandidateBoundaryAndMissingRoutes", semanticRouteProof("adoptiondoctor.build_fails_enforcement_for_candidate_boundary_and_missing_routes"), "Adoption doctor reports must fail closed for enforcement modes when caller-provided owner routes or candidate boundaries are not admitted.")}, + "adoption-workflow-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/adoptionworkflow/adoptionworkflow_test.go", "TestBuildGeneratesBoundedCommandArgv", semanticRouteProof("adoptionworkflow.build_generates_bounded_command_argv"), "Adoption workflow plans must generate bounded argv commands from admitted route refs.")}, + "agent-route": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/agentroute/agentroute_test.go", "TestBuildRoutesRequirementSourceAndBlocksUnknownGoal", semanticRouteProof("agentroute.build_routes_requirement_source_and_blocks_unknown_goal"), "Agent route reports must select a deterministic command family from explicit caller-owned input and fail closed for unknown goals."), packageFalsifierRoute("internal/command/agentroute/agentroute_test.go", "TestBuildEnvelopeKeepsBlockedRoutesAsStopSignals", semanticRouteProof("agentroute.build_envelope_keeps_blocked_routes_as_stop_signals"), "Agent route envelopes must preserve missing-input route states as stop signals instead of executable guidance."), packageFalsifierRoute("internal/command/agentroute/agentroute_test.go", "TestBuildEnvelopeCarriesBlockedObservedReportPreconditions", semanticRouteProof("agentroute.build_envelope_carries_blocked_observed_report_preconditions"), "Agent route envelopes must preserve non-passed observed reports as blocked preconditions instead of executable guidance.")}, + "binding-partition": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/bindingpartition/bindingpartition_test.go", "TestBuildRejectsCrossSurfaceRouteReferenceWithoutDelegation", semanticRouteProof("bindingpartition.build_rejects_cross_surface_route_reference_without_delegation"), "Binding partition admission must reject undelegated cross-surface proof route references.")}, + "branch-authority": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/branchauthority/branchauthority_test.go", "TestBuildAdmitsAlignedRequiredBranchAndRejectsRequiredDrift", semanticRouteProof("branchauthority.build_admits_aligned_required_branch_and_rejects_required_drift"), "Branch authority must reject required branch drift.")}, + "capability-map-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/capabilitymapadmission/capability_map_admission_test.go", "TestBuildCodeBaselineFailsMissingCandidateRequirementAndAnchor", semanticRouteProof("capabilitymapadmission.build_code_baseline_rejects_missing_candidate_or_anchor"), "Capability map admission must fail code_baseline mode when candidate requirement ids or active scenario anchors are missing, while keeping outputs candidate-only.")}, + "changed-path-set": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/changedpathset/changedpathset_test.go", "TestBuildDeduplicatesAndFailsClosedOnInvalidPaths", semanticRouteProof("changedpathset.build_deduplicates_and_fails_closed_on_invalid_paths"), "Changed path set must deduplicate caller path sources and fail closed with redacted invalid-path diagnostics.")}, + "completion-criteria": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/completioncriteria/completioncriteria_test.go", "TestBuildBlocksUnsatisfiedBlockingCriterion", semanticRouteProof("completioncriteria.build_blocks_unsatisfied_blocking_criterion"), "Completion criteria must fail when a blocking criterion is not satisfied.")}, "conformance-profile": { requiredInputAdmissionRoute, - packageFalsifierRoute("internal/command/conformanceprofile/conformanceprofile_test.go", "TestBuildProfileResolvesRequiredSurfaceAndRejectsMissingSurface", semanticRouteProof("conformanceprofile.build_profile_resolves_required_surface_and_rejects_missing_surface", commandCoverageExpectedPublicOutcome), "Conformance profile resolution must reject required surfaces absent from the proof contract."), - packageFalsifierRoute("internal/command/conformanceprofile/conformanceprofile_test.go", "TestBuildVerificationRejectsDuplicateProfiles", semanticRouteProof("conformanceprofile.build_verification_rejects_duplicate_profiles", commandCoverageExpectedPublicOutcome), "Conformance profile verification must reject duplicate profile identities."), - packageFalsifierRoute("internal/command/conformanceprofile/conformanceprofile_test.go", "TestListReturnsSortedProfileIDsAndRejectsInvalidInput", semanticRouteProof("conformanceprofile.list_returns_sorted_profile_ids_and_rejects_invalid_input", commandCoverageExpectedPublicOutcome), "Conformance profile listing must preserve deterministic public ids and reject invalid input."), + packageFalsifierRoute("internal/command/conformanceprofile/conformanceprofile_test.go", "TestBuildProfileResolvesRequiredSurfaceAndRejectsMissingSurface", semanticRouteProof("conformanceprofile.build_profile_resolves_required_surface_and_rejects_missing_surface"), "Conformance profile resolution must reject required surfaces absent from the proof contract."), + packageFalsifierRoute("internal/command/conformanceprofile/conformanceprofile_test.go", "TestBuildVerificationRejectsDuplicateProfiles", semanticRouteProof("conformanceprofile.build_verification_rejects_duplicate_profiles"), "Conformance profile verification must reject duplicate profile identities."), + packageFalsifierRoute("internal/command/conformanceprofile/conformanceprofile_test.go", "TestListReturnsSortedProfileIDsAndRejectsInvalidInput", semanticRouteProof("conformanceprofile.list_returns_sorted_profile_ids_and_rejects_invalid_input"), "Conformance profile listing must preserve deterministic public ids and reject invalid input."), }, - "custom-rule-boundary": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/customruleboundary/customruleboundary_test.go", "TestBuildAdmitsBoundedCustomRuleAndRejectsUnsafeEffects", semanticRouteProof("customruleboundary.build_admits_bounded_custom_rule_and_rejects_unsafe_effects", commandCoverageExpectedPublicOutcome), "Custom-rule boundary reports must reject unsafe custom-rule effects while keeping custom rules local and non-authoritative.")}, - "deployment-evidence-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/deploymentevidenceadmission/deployment_evidence_admission_test.go", "TestBuildAdmitsCandidateEvidenceAndRejectsUnpinnedImages", semanticRouteProof("deployment_evidence_admission.build_admits_candidate_evidence_and_rejects_unpinned_images", commandCoverageExpectedPublicOutcome), "Deployment evidence admission must reject unpinned image references while admitting explicit candidate evidence.")}, - "document-lifecycle-boundary": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/documentlifecycle/documentlifecycle_test.go", "TestBuildAdmitsCurrentDurableDocumentAndRejectsAuthorityDrift", semanticRouteProof("documentlifecycle.build_admits_current_durable_document_and_rejects_authority_drift", commandCoverageExpectedPublicOutcome), "Document lifecycle boundary reports must reject active authority drift across current, generated, rendered, temporary, and archived surfaces.")}, - "evidence-graph": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/projections_test.go", "TestBuildEvidenceGraphBuildsGraphAndRejectsFailedReport", semanticRouteProof("projections.build_evidence_graph_builds_graph_and_rejects_failed_report", commandCoverageExpectedPublicOutcome), "Evidence graph projection must emit graph output only from passed requirement bindings.")}, - "external-consumer": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/externalconsumer/externalconsumer_test.go", "TestBuildAdmitsExternalConsumerProofAndRejectsWorkspaceLock", semanticRouteProof("externalconsumer.build_admits_external_consumer_proof_and_rejects_workspace_lock", commandCoverageExpectedPublicOutcome), "External consumer evidence must reject lockfiles that resolve through the local workspace.")}, - "gradual-adoption": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/gradualadoption/gradualadoption_test.go", "TestBuildRejectsRollbackShellControlCommand", semanticRouteProof("gradualadoption.build_rejects_rollback_shell_control_command", commandCoverageExpectedPublicOutcome), "Gradual adoption reports must reject shell-control rollback commands.")}, - "gradual-adoption-bootstrap": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/gradualadoption/gradualadoption_test.go", "TestBootstrapRejectsUnknownRootAndNestedFields", semanticRouteProof("gradualadoption.bootstrap_rejects_unknown_root_and_nested_fields", commandCoverageExpectedPublicOutcome), "Gradual adoption bootstrap must reject unknown root and nested input fields.")}, - "gradual-adoption-guidance": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/gradualadoption/guidance_test.go", "TestGuidanceEnforcementFailsClosedForCandidateBoundaries", semanticRouteProof("guidance.guidance_enforcement_fails_closed_for_candidate_boundaries", commandCoverageExpectedPublicOutcome), "Gradual adoption guidance must fail closed for candidate boundaries in enforcement modes.")}, - "help": {directCLIRoute("internal/app/cli_contract_test.go", "TestHelpCommandContractForms", semanticRouteProof("cli_contract.help_command_contract_forms", commandCoverageExpectedPublicOutcome), "Help command forms must emit the documented usage contract.")}, - "impact": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/impact/impact_test.go", "TestBuildRoutesChangedRecordToObligationAndRejectsUnboundProofChange", semanticRouteProof("impact.build_routes_changed_record_to_obligation_and_rejects_unbound_proof_change", commandCoverageExpectedPublicOutcome), "Impact analysis must route changed requirement records to obligations and reject unbound proof-like changes.")}, - "init": {directCLIRoute("internal/app/cli_abi_test.go", "TestCLIABIGoldenCorpus", semanticRouteProof("cli_abi.init_golden_corpus", commandCoverageExpectedPublicOutcome), "Init CLI ABI must emit dry-run route guidance without reading stdin, scanning, writing, or promoting repository facts.")}, - "json-report-cli-adapter-source": {packageFalsifierRoute("internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go", "TestGeneratedTypeScriptAdapterExecutesCoreSemantics", semanticRouteProof("json_report_cli_adapter_source.generated_type_script_adapter_executes_core_semantics", commandCoverageExpectedPublicOutcome), "JSON report CLI adapter source generation must emit executable TypeScript that preserves parser, stable JSON, subprocess exit-code, stdout, stderr, and redacted direct-main semantics.")}, - "migration-parity-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationparityadmission/migrationparityadmission_test.go", "TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift", semanticRouteProof("migrationparityadmission.build_admits_caller_declared_match_and_rejects_digest_drift", commandCoverageExpectedPublicOutcome), "Migration parity admission must reject caller-declared matches whose supplied digests differ without claiming native digest verification.")}, - "migration-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationplan/migrationplan_test.go", "TestSortedFollowUpCommandsRejectsShellControlTokens", semanticRouteProof("migrationplan.sorted_follow_up_commands_rejects_shell_control_tokens", commandCoverageExpectedPublicOutcome), "Migration plans must reject shell-control follow-up commands.")}, - "obligation-decision": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/obligationdecision/obligationdecision_test.go", "TestBuildAdmitsSatisfiedBlockingObligationsAndRejectsMissingReceipt", semanticRouteProof("obligationdecision.build_admits_satisfied_blocking_obligations_and_rejects_missing_receipt", commandCoverageExpectedPublicOutcome), "Obligation decision must fail blocking obligations that lack satisfying evidence states.")}, - "package-runtime-dependency-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/packageruntimedependency/package_runtime_dependency_test.go", "TestBuildAdmitsExternalRuntimeDependencyAndRejectsWorkspaceResolution", semanticRouteProof("package_runtime_dependency.build_admits_external_runtime_dependency_and_rejects_workspace_resolution", commandCoverageExpectedPublicOutcome), "Package runtime dependency admission must reject local workspace resolution.")}, - "pilot-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/pilotadmission/pilotadmission_test.go", "TestBuildRejectsUnknownPilotContractField", semanticRouteProof("pilotadmission.build_rejects_unknown_pilot_contract_field", commandCoverageExpectedPublicOutcome), "Pilot admission must reject malformed pilot contract records instead of silently accepting unknown policy fields.")}, - "producer-policy-self-proof": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/producerpolicyselfproof/producerpolicyselfproof_test.go", "TestBuildRejectsPolicyChangeProvedByNewlyAdmittedProducerTuple", semanticRouteProof("producerpolicyselfproof.build_rejects_policy_change_proved_by_newly_admitted_producer_tuple", commandCoverageExpectedPublicOutcome), "Producer policy self-proof must reject merge evidence from the producer tuple admitted by the same policy change.")}, - "proof-obligation-algebra": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/proofobligationalgebra/proof_obligation_algebra_test.go", "TestBuildAdmitsAtomicObligationAndRejectsMissingRoute", semanticRouteProof("proof_obligation_algebra.build_admits_atomic_obligation_and_rejects_missing_route", commandCoverageExpectedPublicOutcome), "Proof obligation algebra must reject atomic obligations with no proof route.")}, - "proof-receipt-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/proofreceiptadmission/proofreceiptadmission_test.go", "TestBuildAdmitsAdvisoryReceiptAndRejectsMergeSatisfyingWithoutProvenance", semanticRouteProof("proofreceiptadmission.build_admits_advisory_receipt_and_rejects_merge_satisfying_without_provenance", commandCoverageExpectedPublicOutcome), "Proof receipt admission must reject merge-satisfying receipt class without provenance evidence.")}, - "proof-slice": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/projections_test.go", "TestBuildProofSliceSelectsRequirementsAndRejectsFailedReport", semanticRouteProof("projections.build_proof_slice_selects_requirements_and_rejects_failed_report", commandCoverageExpectedPublicOutcome), "Proof slice projection must select scoped requirements and reject failed requirement bindings.")}, - "readiness-closeout": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/readinesscloseout/readinesscloseout_test.go", "TestBuildRejectsBroadNegationAndFrontierOverclaim", semanticRouteProof("readinesscloseout.build_rejects_broad_negation_and_frontier_overclaim", commandCoverageExpectedPublicOutcome), "Readiness closeout must reject broad negation suppressors and still detect frontier overclaim grammar.")}, - "receipt-currentness-scope": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/receiptcurrentnessscope/receipt_currentness_scope_test.go", "TestBuildAdmitsCurrentScopedReceiptAndRejectsStaleDigest", semanticRouteProof("receipt_currentness_scope.build_admits_current_scoped_receipt_and_rejects_stale_digest", commandCoverageExpectedPublicOutcome), "Receipt currentness-scope admission must reject stale recorded/current digest pairs.")}, - "receipt-producer-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/receiptproduceradmission/receiptproduceradmission_test.go", "TestBuildRejectsAdvisoryProducerForMergeSatisfyingReceipt", semanticRouteProof("receiptproduceradmission.build_rejects_advisory_producer_for_merge_satisfying_receipt", commandCoverageExpectedPublicOutcome), "Receipt producer admission must reject advisory producers for merge-satisfying receipt claims.")}, - "receipt-trust-class": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/receipttrustclass/receipt_trust_class_test.go", "TestBuildAdmitsTrustedReceiptAndRejectsMissingProvenance", semanticRouteProof("receipt_trust_class.build_admits_trusted_receipt_and_rejects_missing_provenance", commandCoverageExpectedPublicOutcome), "Receipt trust-class admission must reject missing provenance for trust classes that require it.")}, - "registry-consumer": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", semanticRouteProof("registryconsumer.registry_consumer_accepts_registry_release_proof", commandCoverageExpectedPublicOutcome), "Registry consumer reports must accept registry-release proof only when release authority, registry pack facts, lockfiles, smoke output, and release-authority digest all align."), packageFalsifierRoute("internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerRejectsLegacyRootImportProof", semanticRouteProof("registryconsumer.registry_consumer_rejects_legacy_root_import_proof", commandCoverageExpectedPublicOutcome), "Registry consumer reports must reject legacy root import proof shape.")}, - "registry-consumer-proof-input-compose": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go", "TestBuildComposesInputAcceptedByRegistryConsumer", semanticRouteProof("registry_consumer_input_compose.build_composes_input_accepted_by_registry_consumer", commandCoverageExpectedPublicOutcome), "Registry consumer proof input composition must project explicit primitive registry and toolchain facts into an input accepted by the existing registry-consumer validator without executing registry or toolchain work."), packageFalsifierRoute("internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go", "TestBuildBlocksUnavailableRequiredPreconditionsWithoutAcceptedInput", semanticRouteProof("registry_consumer_input_compose.build_blocks_unavailable_required_preconditions_without_accepted_input", commandCoverageExpectedPublicOutcome), "Unavailable registry, install, smoke, or rollback preconditions must produce blocked composition output instead of accepted registry-consumer input.")}, - "release-authority": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/releaseauthority/releaseauthority_test.go", "TestBuildRejectsPrivateSourceNPMProvenanceClaim", semanticRouteProof("releaseauthority.build_rejects_private_source_npmprovenance_claim", commandCoverageExpectedPublicOutcome), "Release authority must reject npm provenance claims without public source repository proof.")}, - "rendered-artifact-freshness": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/renderedartifactfreshness/rendered_artifact_freshness_test.go", "TestBuildAdmitsFreshRenderedArtifactAndRejectsDigestDrift", semanticRouteProof("rendered_artifact_freshness.build_admits_fresh_rendered_artifact_and_rejects_digest_drift", commandCoverageExpectedPublicOutcome), "Rendered artifact freshness must reject recorded/current digest drift.")}, - "repo-profile-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/repoprofileadmission/repo_profile_admission_test.go", "TestBuildAdmitsValidRepoProfileAndRejectsRootPackageMismatch", semanticRouteProof("repo_profile_admission.build_admits_valid_repo_profile_and_rejects_root_package_mismatch", commandCoverageExpectedPublicOutcome), "Repo profile admission must reject mismatch between profile root package and observed package facts.")}, - "requirement-bindings": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/projections_test.go", "TestBuildReportFailsUnknownRequirementBinding", semanticRouteProof("projections.build_report_fails_unknown_requirement_binding", commandCoverageExpectedPublicOutcome), "Requirement binding reports must fail closed when bindings reference unknown requirements.")}, - "requirement-browser-server": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbrowser/server_test.go", "TestStartServerFailsClosedForNonLoopbackHosts", semanticRouteProof("server.start_server_fails_closed_for_non_loopback_hosts", commandCoverageExpectedPublicOutcome), "Requirement browser server must reject non-loopback host binding."), directCLIRoute("internal/app/cli_abi_test.go", "TestRequirementBrowserServerSpecTreeCLIABI", semanticRouteProof("cli_abi.requirement_browser_server_spec_tree_cliabi", commandCoverageExpectedPublicOutcome), "Requirement browser server CLI ABI must admit explicit spec-tree view routing and emit a presentation-only browser plan.")}, - "requirement-context-compose": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.compose_through_whole_cli", commandCoverageExpectedPublicOutcome), "Requirement context composition must honor its explicit repository root and emit an owner-admitted snapshot through the public CLI."), packageFalsifierRoute("internal/command/requirementcontext/requirementcontext_test.go", "TestComposeAndSliceRoundTrip", semanticRouteProof("requirementcontext.compose_and_slice_round_trip", commandCoverageExpectedPublicOutcome), "Requirement context composition must read only an explicit catalog and produce a content-bound snapshot accepted unchanged by the slice owner.")}, - "requirement-context-slice": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.slice_through_whole_cli", commandCoverageExpectedPublicOutcome), "Requirement context slicing must consume the composed snapshot and emit the selected semantic fragment through the public CLI."), packageFalsifierRoute("internal/command/requirementcontext/requirementcontext_test.go", "TestSliceRejectsTamperedSnapshotAndUnknownNode", semanticRouteProof("requirementcontext.slice_rejects_tampered_snapshot_and_unknown_node", commandCoverageExpectedPublicOutcome), "Requirement context slicing must reject stale snapshot identity and unknown explicit semantic targets.")}, + "custom-rule-boundary": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/customruleboundary/customruleboundary_test.go", "TestBuildAdmitsBoundedCustomRuleAndRejectsUnsafeEffects", semanticRouteProof("customruleboundary.build_admits_bounded_custom_rule_and_rejects_unsafe_effects"), "Custom-rule boundary reports must reject unsafe custom-rule effects while keeping custom rules local and non-authoritative.")}, + "deployment-evidence-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/deploymentevidenceadmission/deployment_evidence_admission_test.go", "TestBuildAdmitsCandidateEvidenceAndRejectsUnpinnedImages", semanticRouteProof("deployment_evidence_admission.build_admits_candidate_evidence_and_rejects_unpinned_images"), "Deployment evidence admission must reject unpinned image references while admitting explicit candidate evidence.")}, + "document-lifecycle-boundary": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/documentlifecycle/documentlifecycle_test.go", "TestBuildAdmitsCurrentDurableDocumentAndRejectsAuthorityDrift", semanticRouteProof("documentlifecycle.build_admits_current_durable_document_and_rejects_authority_drift"), "Document lifecycle boundary reports must reject active authority drift across current, generated, rendered, temporary, and archived surfaces.")}, + "evidence-graph": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/projections_test.go", "TestBuildEvidenceGraphBuildsGraphAndRejectsFailedReport", semanticRouteProof("projections.build_evidence_graph_builds_graph_and_rejects_failed_report"), "Evidence graph projection must emit graph output only from passed requirement bindings.")}, + "external-consumer": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/externalconsumer/externalconsumer_test.go", "TestBuildAdmitsExternalConsumerProofAndRejectsWorkspaceLock", semanticRouteProof("externalconsumer.build_admits_external_consumer_proof_and_rejects_workspace_lock"), "External consumer evidence must reject lockfiles that resolve through the local workspace.")}, + "gradual-adoption": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/gradualadoption/gradualadoption_test.go", "TestBuildRejectsRollbackShellControlCommand", semanticRouteProof("gradualadoption.build_rejects_rollback_shell_control_command"), "Gradual adoption reports must reject shell-control rollback commands.")}, + "gradual-adoption-bootstrap": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/gradualadoption/gradualadoption_test.go", "TestBootstrapRejectsUnknownRootAndNestedFields", semanticRouteProof("gradualadoption.bootstrap_rejects_unknown_root_and_nested_fields"), "Gradual adoption bootstrap must reject unknown root and nested input fields.")}, + "gradual-adoption-guidance": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/gradualadoption/guidance_test.go", "TestGuidanceEnforcementFailsClosedForCandidateBoundaries", semanticRouteProof("guidance.guidance_enforcement_fails_closed_for_candidate_boundaries"), "Gradual adoption guidance must fail closed for candidate boundaries in enforcement modes.")}, + "help": {directCLIRoute("internal/app/cli_contract_test.go", "TestHelpCommandContractForms", semanticRouteProof("cli_contract.help_command_contract_forms"), "Help command forms must emit the documented usage contract.")}, + "impact": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/impact/impact_test.go", "TestBuildRoutesChangedRecordToObligationAndRejectsUnboundProofChange", semanticRouteProof("impact.build_routes_changed_record_to_obligation_and_rejects_unbound_proof_change"), "Impact analysis must route changed requirement records to obligations and reject unbound proof-like changes.")}, + "init": {directCLIRoute("internal/app/cli_abi_test.go", "TestCLIABIGoldenCorpus", semanticRouteProof("cli_abi.init_golden_corpus"), "Init CLI ABI must emit dry-run route guidance without reading stdin, scanning, writing, or promoting repository facts.")}, + "json-report-cli-adapter-source": {packageFalsifierRoute("internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go", "TestGeneratedTypeScriptAdapterExecutesCoreSemantics", semanticRouteProof("json_report_cli_adapter_source.generated_type_script_adapter_executes_core_semantics"), "JSON report CLI adapter source generation must emit executable TypeScript that preserves parser, stable JSON, subprocess exit-code, stdout, stderr, and redacted direct-main semantics.")}, + "migration-parity-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationparityadmission/migrationparityadmission_test.go", "TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift", semanticRouteProof("migrationparityadmission.build_admits_caller_declared_match_and_rejects_digest_drift"), "Migration parity admission must reject caller-declared matches whose supplied digests differ without claiming native digest verification.")}, + "migration-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationplan/migrationplan_test.go", "TestSortedFollowUpCommandsRejectsShellControlTokens", semanticRouteProof("migrationplan.sorted_follow_up_commands_rejects_shell_control_tokens"), "Migration plans must reject shell-control follow-up commands.")}, + "obligation-decision": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/obligationdecision/obligationdecision_test.go", "TestBuildAdmitsSatisfiedBlockingObligationsAndRejectsMissingReceipt", semanticRouteProof("obligationdecision.build_admits_satisfied_blocking_obligations_and_rejects_missing_receipt"), "Obligation decision must fail blocking obligations that lack satisfying evidence states.")}, + "package-runtime-dependency-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/packageruntimedependency/package_runtime_dependency_test.go", "TestBuildAdmitsExternalRuntimeDependencyAndRejectsWorkspaceResolution", semanticRouteProof("package_runtime_dependency.build_admits_external_runtime_dependency_and_rejects_workspace_resolution"), "Package runtime dependency admission must reject local workspace resolution.")}, + "pilot-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/pilotadmission/pilotadmission_test.go", "TestBuildRejectsUnknownPilotContractField", semanticRouteProof("pilotadmission.build_rejects_unknown_pilot_contract_field"), "Pilot admission must reject malformed pilot contract records instead of silently accepting unknown policy fields.")}, + "producer-policy-self-proof": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/producerpolicyselfproof/producerpolicyselfproof_test.go", "TestBuildRejectsPolicyChangeProvedByNewlyAdmittedProducerTuple", semanticRouteProof("producerpolicyselfproof.build_rejects_policy_change_proved_by_newly_admitted_producer_tuple"), "Producer policy self-proof must reject merge evidence from the producer tuple admitted by the same policy change.")}, + "proof-obligation-algebra": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/proofobligationalgebra/proof_obligation_algebra_test.go", "TestBuildAdmitsAtomicObligationAndRejectsMissingRoute", semanticRouteProof("proof_obligation_algebra.build_admits_atomic_obligation_and_rejects_missing_route"), "Proof obligation algebra must reject atomic obligations with no proof route.")}, + "proof-receipt-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/proofreceiptadmission/proofreceiptadmission_test.go", "TestBuildAdmitsAdvisoryReceiptAndRejectsMergeSatisfyingWithoutProvenance", semanticRouteProof("proofreceiptadmission.build_admits_advisory_receipt_and_rejects_merge_satisfying_without_provenance"), "Proof receipt admission must reject merge-satisfying receipt class without provenance evidence.")}, + "proof-slice": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/projections_test.go", "TestBuildProofSliceSelectsRequirementsAndRejectsFailedReport", semanticRouteProof("projections.build_proof_slice_selects_requirements_and_rejects_failed_report"), "Proof slice projection must select scoped requirements and reject failed requirement bindings.")}, + "readiness-closeout": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/readinesscloseout/readinesscloseout_test.go", "TestBuildRejectsBroadNegationAndFrontierOverclaim", semanticRouteProof("readinesscloseout.build_rejects_broad_negation_and_frontier_overclaim"), "Readiness closeout must reject broad negation suppressors and still detect frontier overclaim grammar.")}, + "receipt-currentness-scope": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/receiptcurrentnessscope/receipt_currentness_scope_test.go", "TestBuildAdmitsCurrentScopedReceiptAndRejectsStaleDigest", semanticRouteProof("receipt_currentness_scope.build_admits_current_scoped_receipt_and_rejects_stale_digest"), "Receipt currentness-scope admission must reject stale recorded/current digest pairs.")}, + "receipt-producer-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/receiptproduceradmission/receiptproduceradmission_test.go", "TestBuildRejectsAdvisoryProducerForMergeSatisfyingReceipt", semanticRouteProof("receiptproduceradmission.build_rejects_advisory_producer_for_merge_satisfying_receipt"), "Receipt producer admission must reject advisory producers for merge-satisfying receipt claims.")}, + "receipt-trust-class": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/receipttrustclass/receipt_trust_class_test.go", "TestBuildAdmitsTrustedReceiptAndRejectsMissingProvenance", semanticRouteProof("receipt_trust_class.build_admits_trusted_receipt_and_rejects_missing_provenance"), "Receipt trust-class admission must reject missing provenance for trust classes that require it.")}, + "registry-consumer": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", semanticRouteProof("registryconsumer.registry_consumer_accepts_registry_release_proof"), "Registry consumer reports must accept registry-release proof only when release authority, registry pack facts, lockfiles, smoke output, and release-authority digest all align."), packageFalsifierRoute("internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerRejectsLegacyRootImportProof", semanticRouteProof("registryconsumer.registry_consumer_rejects_legacy_root_import_proof"), "Registry consumer reports must reject legacy root import proof shape.")}, + "registry-consumer-proof-input-compose": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go", "TestBuildComposesInputAcceptedByRegistryConsumer", semanticRouteProof("registry_consumer_input_compose.build_composes_input_accepted_by_registry_consumer"), "Registry consumer proof input composition must project explicit primitive registry and toolchain facts into an input accepted by the existing registry-consumer validator without executing registry or toolchain work."), packageFalsifierRoute("internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go", "TestBuildBlocksUnavailableRequiredPreconditionsWithoutAcceptedInput", semanticRouteProof("registry_consumer_input_compose.build_blocks_unavailable_required_preconditions_without_accepted_input"), "Unavailable registry, install, smoke, or rollback preconditions must produce blocked composition output instead of accepted registry-consumer input.")}, + "release-authority": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/releaseauthority/releaseauthority_test.go", "TestBuildRejectsPrivateSourceNPMProvenanceClaim", semanticRouteProof("releaseauthority.build_rejects_private_source_npmprovenance_claim"), "Release authority must reject npm provenance claims without public source repository proof.")}, + "rendered-artifact-freshness": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/renderedartifactfreshness/rendered_artifact_freshness_test.go", "TestBuildAdmitsFreshRenderedArtifactAndRejectsDigestDrift", semanticRouteProof("rendered_artifact_freshness.build_admits_fresh_rendered_artifact_and_rejects_digest_drift"), "Rendered artifact freshness must reject recorded/current digest drift.")}, + "repo-profile-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/repoprofileadmission/repo_profile_admission_test.go", "TestBuildAdmitsValidRepoProfileAndRejectsRootPackageMismatch", semanticRouteProof("repo_profile_admission.build_admits_valid_repo_profile_and_rejects_root_package_mismatch"), "Repo profile admission must reject mismatch between profile root package and observed package facts.")}, + "requirement-bindings": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/projections_test.go", "TestBuildReportFailsUnknownRequirementBinding", semanticRouteProof("projections.build_report_fails_unknown_requirement_binding"), "Requirement binding reports must fail closed when bindings reference unknown requirements.")}, + "requirement-browser-server": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbrowser/server_test.go", "TestStartServerFailsClosedForNonLoopbackHosts", semanticRouteProof("server.start_server_fails_closed_for_non_loopback_hosts"), "Requirement browser server must reject non-loopback host binding."), directCLIRoute("internal/app/cli_abi_test.go", "TestRequirementBrowserServerSpecTreeCLIABI", semanticRouteProof("cli_abi.requirement_browser_server_spec_tree_cliabi"), "Requirement browser server CLI ABI must admit explicit spec-tree view routing and emit a presentation-only browser plan.")}, + "requirement-context-compose": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.compose_through_whole_cli"), "Requirement context composition must honor its explicit repository root and emit an owner-admitted snapshot through the public CLI."), packageFalsifierRoute("internal/command/requirementcontext/requirementcontext_test.go", "TestComposeAndSliceRoundTrip", semanticRouteProof("requirementcontext.compose_and_slice_round_trip"), "Requirement context composition must read only an explicit catalog and produce a content-bound snapshot accepted unchanged by the slice owner.")}, + "requirement-context-slice": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.slice_through_whole_cli"), "Requirement context slicing must consume the composed snapshot and emit the selected semantic fragment through the public CLI."), packageFalsifierRoute("internal/command/requirementcontext/requirementcontext_test.go", "TestSliceRejectsTamperedSnapshotAndUnknownNode", semanticRouteProof("requirementcontext.slice_rejects_tampered_snapshot_and_unknown_node"), "Requirement context slicing must reject stale snapshot identity and unknown explicit semantic targets.")}, "requirement-coverage-input-compose": { requiredInputAdmissionRoute, - packageFalsifierRoute("internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "TestBuildComposesInputPreservesDeclaredUniverseAndAllowsDownstreamFailures", semanticRouteProof("requirementcoverageinput.build_composes_input_preserves_declared_universe_and_allows_downstream_failures", commandCoverageExpectedPublicOutcome), "Requirement coverage input composition must preserve declared universe facts while keeping downstream coverage failures separate from composition admission."), - packageFalsifierRoute("internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "TestBuildRejectsFabricatedDirectEnvelopeWithSourceMetadata", semanticRouteProof("requirementcoverageinput.build_rejects_fabricated_direct_envelope_with_source_metadata", commandCoverageExpectedPublicOutcome), "Requirement coverage input composition must reject fabricated normalized inventory envelopes before composing a coverage view input."), - packageFalsifierRoute("internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "TestBuildComposesDirectRequirementProofBindingAndInventory", semanticRouteProof("requirementcoverageinput.build_composes_direct_requirement_proof_binding_and_inventory", commandCoverageExpectedPublicOutcome), "Requirement coverage input composition must admit direct proof-binding and test-inventory child reports before composing the coverage-view input."), + packageFalsifierRoute("internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "TestBuildComposesInputPreservesDeclaredUniverseAndAllowsDownstreamFailures", semanticRouteProof("requirementcoverageinput.build_composes_input_preserves_declared_universe_and_allows_downstream_failures"), "Requirement coverage input composition must preserve declared universe facts while keeping downstream coverage failures separate from composition admission."), + packageFalsifierRoute("internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "TestBuildRejectsFabricatedDirectEnvelopeWithSourceMetadata", semanticRouteProof("requirementcoverageinput.build_rejects_fabricated_direct_envelope_with_source_metadata"), "Requirement coverage input composition must reject fabricated normalized inventory envelopes before composing a coverage view input."), + packageFalsifierRoute("internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "TestBuildComposesDirectRequirementProofBindingAndInventory", semanticRouteProof("requirementcoverageinput.build_composes_direct_requirement_proof_binding_and_inventory"), "Requirement coverage input composition must admit direct proof-binding and test-inventory child reports before composing the coverage-view input."), }, - "requirement-coverage-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementcoverageview/requirementcoverageview_test.go", "TestBuildJSONRejectsRouteOnlyCoverageForBlockingRequirement", semanticRouteProof("requirementcoverageview.build_jsonrejects_route_only_coverage_for_blocking_requirement", commandCoverageExpectedPublicOutcome), "Requirement coverage views must not treat route-only smoke evidence as semantic requirement coverage.")}, - "requirement-impact-input-compose": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementimpactinput/requirementimpactinput_test.go", "TestBuildComposesInputAndRoutesChangedBlockingRequirement", semanticRouteProof("requirementimpactinput.build_composes_input_and_routes_changed_blocking_requirement", commandCoverageExpectedPublicOutcome), "Requirement impact input composition must emit direct impact inputs from admitted caller-owned sources while preserving downstream impact semantics.")}, - "requirement-proof-resolver": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/compact_contract_test.go", "TestBuildResolverRejectsUnscopedCompactIdentity", semanticRouteProof("compact_contract.build_resolver_rejects_unscoped_compact_identity", commandCoverageExpectedPublicOutcome), "Requirement proof resolver must fail closed on unscoped scenario ids and unadmitted witness selector identities."), packageFalsifierRoute("internal/command/requirementbinding/compact_contract_test.go", "TestBuildResolverEmitsNamedLookupFacts", semanticRouteProof("compact_contract.build_resolver_emits_named_lookup_facts", commandCoverageExpectedPublicOutcome), "Requirement proof resolver must emit deterministic named lookup facts for commands, environment classes, surfaces, scenarios, and witness selectors.")}, - "requirement-proof-source-set": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementproofsourceset/requirementproofsourceset_test.go", "TestBuildSelectsSourceSetRowsAndEmitsResolverInput", semanticRouteProof("requirementproofsourceset.build_selects_source_set_rows_and_emits_resolver_input", commandCoverageExpectedPublicOutcome), "Requirement proof source-set normalization must select caller-owned source rows and emit resolver-compatible projections without scanning repositories.")}, - "requirement-semantic-diff": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.diff_through_whole_cli", commandCoverageExpectedPublicOutcome), "Requirement semantic diff must emit an owner-admitted change set through the public CLI."), packageFalsifierRoute("internal/command/requirementdiff/requirementdiff_test.go", "TestBuildCoversCompleteRequirementChangeAlgebra", semanticRouteProof("requirementdiff.build_covers_complete_requirement_change_algebra", commandCoverageExpectedPublicOutcome), "Requirement semantic diff must cover entity, scalar, set, map, and lifecycle changes and remain closed under output admission.")}, - "requirement-proof-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementproofview/requirementproofview_test.go", "TestBuildMarkdownEscapesCallerControlledCompactFields", semanticRouteProof("requirementproofview.build_markdown_escapes_caller_controlled_compact_fields", commandCoverageExpectedPublicOutcome), "Requirement proof view must escape caller-controlled compact binding fields.")}, - "requirement-authoring-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementauthoringplan/requirement_authoring_plan_test.go", "TestBuildRejectsCandidateSourceAdmissionFailure", semanticRouteProof("requirement_authoring_plan.build_rejects_candidate_source_admission_failure", commandCoverageExpectedPublicOutcome), "Requirement authoring plans must keep candidate source previews candidate-only and fail closed when the composed source cannot pass requirement-source admission.")}, - "requirement-source-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementsourceadmission/requirementsourceadmission_test.go", "TestEvaluateRejectsBlockingRequirementWithoutProofRoute", semanticRouteProof("requirementsourceadmission.evaluate_rejects_blocking_requirement_without_proof_route", commandCoverageExpectedPublicOutcome), "Requirement source admission must reject blocking active requirements without proof binding routes.")}, - "requirement-source-transition": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementsourcetransition/requirementsourcetransition_test.go", "TestBuildRejectsRequirementSourceTransitionContractViolations", semanticRouteProof("requirementsourcetransition.build_rejects_requirement_source_transition_contract_violations", commandCoverageExpectedPublicOutcome), "Requirement source transition must reject previous and next source admission, source identity, package boundary, durable identity, terminal state, evidence-delta, active replacement, and stable-ref contract violations.")}, - "requirement-source-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementsourceview/requirementsourceview_test.go", "TestBuildMarkdownEscapesCallerControlledText", semanticRouteProof("requirementsourceview.build_markdown_escapes_caller_controlled_text", commandCoverageExpectedPublicOutcome), "Requirement source view must escape caller-controlled requirement text.")}, - "requirement-spec-tree": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementspectree/requirementspectree_test.go", "TestBuildRejectsDAGAndStaleDigest", semanticRouteProof("requirementspectree.build_rejects_dagand_stale_digest", commandCoverageExpectedPublicOutcome), "Requirement spec tree admission must reject DAG topology and stale caller-provided source digest facts.")}, - "requirement-spec-tree-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementspectree/requirementspectree_test.go", "TestBuildViewMarkdownAndHTMLAreDeterministicAndEscaped", semanticRouteProof("requirementspectree.build_view_markdown_and_htmlare_deterministic_and_escaped", commandCoverageExpectedPublicOutcome), "Requirement spec tree views must reuse admitted spec-tree input and escape caller-controlled text in deterministic HTML and Markdown projections.")}, - "requirement-traceability-graph": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.graph_through_whole_cli", commandCoverageExpectedPublicOutcome), "Requirement traceability graph must emit an owner-admitted graph through the public CLI."), packageFalsifierRoute("internal/command/requirementgraph/requirementgraph_test.go", "TestBuildKeepsTraceabilityEvidencePlanesDistinct", semanticRouteProof("requirementgraph.build_keeps_traceability_evidence_planes_distinct", commandCoverageExpectedPublicOutcome), "Requirement traceability graph must keep specification, proof, code traceability, and native execution evidence planes distinct."), packageFalsifierRoute("internal/command/requirementgraph/requirementgraph_test.go", "TestAdmitOutputRejectsDanglingAndIncoherentCodeParents", semanticRouteProof("requirementgraph.admit_output_rejects_dangling_and_incoherent_code_parents", commandCoverageExpectedPublicOutcome), "Requirement traceability graph output admission must reject dangling and incoherent code parent relations.")}, - "scaffold-profile-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/scaffoldprofileplan/scaffoldprofileplan_test.go", "TestBuildAcceptsCommandMatcherHints", semanticRouteProof("scaffoldprofileplan.build_accepts_command_matcher_hints", commandCoverageExpectedPublicOutcome), "Scaffold profile planning must preserve caller-reviewed command matcher hints as deterministic profile draft data.")}, - "scaffold-project-structure": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/projectstructure/projectstructure_test.go", "TestBuildAdmitsProjectStructureScaffoldAndEmitsBoundedEnvelope", semanticRouteProof("projectstructure.build_admits_project_structure_scaffold_and_emits_bounded_envelope", commandCoverageExpectedPublicOutcome), "Project structure scaffold must emit deterministic source-report identity and bounded agent guidance without writing files."), packageFalsifierRoute("internal/command/projectstructure/projectstructure_test.go", "TestBuildRejectsProjectStructurePathDriftAndUnsafePaths", semanticRouteProof("projectstructure.build_rejects_project_structure_path_drift_and_unsafe_paths", commandCoverageExpectedPublicOutcome), "Project structure scaffold must reject unsafe paths and inconsistent bootstrap/profile proof paths.")}, - "selective-gate-evidence": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/selectivegateevidence/selectivegateevidence_test.go", "TestBuildRejectsMergeSatisfyingEvidenceWithoutProducerAdmission", semanticRouteProof("selectivegateevidence.build_rejects_merge_satisfying_evidence_without_producer_admission", commandCoverageExpectedPublicOutcome), "Selective gate evidence must reject merge-satisfying evidence without producer admission."), packageFalsifierRoute("internal/command/selectivegateevidence/selectivegateevidence_test.go", "TestBuildReportsMergeEvidenceWithoutApprovingMerge", semanticRouteProof("selectivegateevidence.build_reports_merge_evidence_without_approving_merge", commandCoverageExpectedPublicOutcome), "Selective gate evidence must report merge evidence facts without approving consumer-owned merge admission.")}, - "selective-gate-obligation-decision-input": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/selectivegateevidence/selectivegateevidence_test.go", "TestProjectObligationDecisionBuildsInputAndRejectsUnroutedCommand", semanticRouteProof("selectivegateevidence.project_obligation_decision_builds_input_and_rejects_unrouted_command", commandCoverageExpectedPublicOutcome), "Selective evidence projection must reject receipts that cannot be routed to planned commands.")}, - "selective-gate-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/selectivegateplan/selectivegateplan_test.go", "TestBuildFailsClosedForUncoveredUnknownEdge", semanticRouteProof("selectivegateplan.build_fails_closed_for_uncovered_unknown_edge", commandCoverageExpectedPublicOutcome), "Selective gate planning must fail closed for uncovered unknown dependency edges.")}, - "secret-scan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/secretscan/secretscan_test.go", "TestBuildFindsSecretLikeTextWithoutLeakingValue", semanticRouteProof("secretscan.build_finds_secret_like_text_without_leaking_value", commandCoverageExpectedPublicOutcome), "Secret scan must detect secret-like text in explicit caller inventory without leaking matched values or scanning repository state.")}, - "self-check": {directCLIRoute("internal/app/app_test.go", "TestSelfCheckRejectsDuplicateKeys", semanticRouteProof("app.self_check_rejects_duplicate_keys", commandCoverageExpectedPublicOutcome), "Self-check must reject ambiguous duplicate-key JSON without echoing the duplicated key.")}, - "spec-overview-claims": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/specoverviewclaims/specoverviewclaims_test.go", "TestBuildRejectsInvalidOverviewClaimBoundaryFacts", semanticRouteProof("specoverviewclaims.build_rejects_invalid_overview_claim_boundary_facts", commandCoverageExpectedPublicOutcome), "Spec overview claim admission must reject invalid path, extraction, digest, marker, rationale, and non-claim boundary facts."), packageFalsifierRoute("internal/command/specoverviewclaims/specoverviewclaims_test.go", "TestBuildRejectsNonDurableRequirementCitationsForEveryNonDurableKind", semanticRouteProof("specoverviewclaims.build_rejects_non_durable_requirement_citations_for_every_non_durable_kind", commandCoverageExpectedPublicOutcome), "Spec overview claim admission must reject every non-durable claim kind when it carries requirement citations.")}, - "spec-proof-bundle-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/specproofbundleadmission/specproofbundleadmission_test.go", "TestBuildRejectsForgedReceiptAdmissionChild", semanticRouteProof("specproofbundleadmission.build_rejects_forged_receipt_admission_child", commandCoverageExpectedPublicOutcome), "Spec proof bundle admission must reject forged child receipt admission reports.")}, - "stack-preset": {directCLIRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("command_coverage.no_input_commands_have_command_specific_behavior", commandCoverageExpectedPublicOutcome), "Stack preset CLI route must emit JSON and reject unknown preset flags."), packageFalsifierRoute("internal/command/stackpreset/stackpreset_test.go", "TestPresetInventoryIsCompleteDeterministicAndDefensivelyCopied", semanticRouteProof("stackpreset.preset_inventory_is_complete_deterministic_and_defensively_copied", commandCoverageExpectedPublicOutcome), "Stack preset inventory must keep preset ids aligned with complete non-empty profile records and defensive copies."), packageFalsifierRoute("internal/command/stackpreset/stackpreset_test.go", "TestUnknownPresetIsRejected", semanticRouteProof("stackpreset.unknown_preset_is_rejected", commandCoverageExpectedPublicOutcome), "Stack preset package API must reject unknown preset ids.")}, + "requirement-coverage-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementcoverageview/requirementcoverageview_test.go", "TestBuildJSONRejectsRouteOnlyCoverageForBlockingRequirement", semanticRouteProof("requirementcoverageview.build_jsonrejects_route_only_coverage_for_blocking_requirement"), "Requirement coverage views must not treat route-only smoke evidence as semantic requirement coverage.")}, + "requirement-impact-input-compose": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementimpactinput/requirementimpactinput_test.go", "TestBuildComposesInputAndRoutesChangedBlockingRequirement", semanticRouteProof("requirementimpactinput.build_composes_input_and_routes_changed_blocking_requirement"), "Requirement impact input composition must emit direct impact inputs from admitted caller-owned sources while preserving downstream impact semantics.")}, + "requirement-proof-resolver": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementbinding/compact_contract_test.go", "TestBuildResolverRejectsUnscopedCompactIdentity", semanticRouteProof("compact_contract.build_resolver_rejects_unscoped_compact_identity"), "Requirement proof resolver must fail closed on unscoped scenario ids and unadmitted witness selector identities."), packageFalsifierRoute("internal/command/requirementbinding/compact_contract_test.go", "TestBuildResolverEmitsNamedLookupFacts", semanticRouteProof("compact_contract.build_resolver_emits_named_lookup_facts"), "Requirement proof resolver must emit deterministic named lookup facts for commands, environment classes, surfaces, scenarios, and witness selectors.")}, + "requirement-proof-source-set": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementproofsourceset/requirementproofsourceset_test.go", "TestBuildSelectsSourceSetRowsAndEmitsResolverInput", semanticRouteProof("requirementproofsourceset.build_selects_source_set_rows_and_emits_resolver_input"), "Requirement proof source-set normalization must select caller-owned source rows and emit resolver-compatible projections without scanning repositories.")}, + "requirement-semantic-diff": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.diff_through_whole_cli"), "Requirement semantic diff must emit an owner-admitted change set through the public CLI."), packageFalsifierRoute("internal/command/requirementdiff/requirementdiff_test.go", "TestBuildCoversCompleteRequirementChangeAlgebra", semanticRouteProof("requirementdiff.build_covers_complete_requirement_change_algebra"), "Requirement semantic diff must cover entity, scalar, set, map, and lifecycle changes and remain closed under output admission.")}, + "requirement-proof-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementproofview/requirementproofview_test.go", "TestBuildMarkdownEscapesCallerControlledCompactFields", semanticRouteProof("requirementproofview.build_markdown_escapes_caller_controlled_compact_fields"), "Requirement proof view must escape caller-controlled compact binding fields.")}, + "requirement-authoring-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementauthoringplan/requirement_authoring_plan_test.go", "TestBuildRejectsCandidateSourceAdmissionFailure", semanticRouteProof("requirement_authoring_plan.build_rejects_candidate_source_admission_failure"), "Requirement authoring plans must keep candidate source previews candidate-only and fail closed when the composed source cannot pass requirement-source admission.")}, + "requirement-source-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementsourceadmission/requirementsourceadmission_test.go", "TestEvaluateRejectsBlockingRequirementWithoutProofRoute", semanticRouteProof("requirementsourceadmission.evaluate_rejects_blocking_requirement_without_proof_route"), "Requirement source admission must reject blocking active requirements without proof binding routes.")}, + "requirement-source-transition": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementsourcetransition/requirementsourcetransition_test.go", "TestBuildRejectsRequirementSourceTransitionContractViolations", semanticRouteProof("requirementsourcetransition.build_rejects_requirement_source_transition_contract_violations"), "Requirement source transition must reject previous and next source admission, source identity, package boundary, durable identity, terminal state, evidence-delta, active replacement, and stable-ref contract violations.")}, + "requirement-source-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementsourceview/requirementsourceview_test.go", "TestBuildMarkdownEscapesCallerControlledText", semanticRouteProof("requirementsourceview.build_markdown_escapes_caller_controlled_text"), "Requirement source view must escape caller-controlled requirement text.")}, + "requirement-spec-tree": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementspectree/requirementspectree_test.go", "TestBuildRejectsDAGAndStaleDigest", semanticRouteProof("requirementspectree.build_rejects_dagand_stale_digest"), "Requirement spec tree admission must reject DAG topology and stale caller-provided source digest facts.")}, + "requirement-spec-tree-view": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/requirementspectree/requirementspectree_test.go", "TestBuildViewMarkdownAndHTMLAreDeterministicAndEscaped", semanticRouteProof("requirementspectree.build_view_markdown_and_htmlare_deterministic_and_escaped"), "Requirement spec tree views must reuse admitted spec-tree input and escape caller-controlled text in deterministic HTML and Markdown projections.")}, + "requirement-traceability-graph": {requiredInputAdmissionRoute, directCLIRoute("internal/app/requirement_context_cli_test.go", "TestRequirementContextCommandsComposeThroughWholeCLI", semanticRouteProof("requirement_context_cli.graph_through_whole_cli"), "Requirement traceability graph must emit an owner-admitted graph through the public CLI."), packageFalsifierRoute("internal/command/requirementgraph/requirementgraph_test.go", "TestBuildKeepsTraceabilityEvidencePlanesDistinct", semanticRouteProof("requirementgraph.build_keeps_traceability_evidence_planes_distinct"), "Requirement traceability graph must keep specification, proof, code traceability, and native execution evidence planes distinct."), packageFalsifierRoute("internal/command/requirementgraph/requirementgraph_test.go", "TestAdmitOutputRejectsDanglingAndIncoherentCodeParents", semanticRouteProof("requirementgraph.admit_output_rejects_dangling_and_incoherent_code_parents"), "Requirement traceability graph output admission must reject dangling and incoherent code parent relations.")}, + "scaffold-profile-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/scaffoldprofileplan/scaffoldprofileplan_test.go", "TestBuildAcceptsCommandMatcherHints", semanticRouteProof("scaffoldprofileplan.build_accepts_command_matcher_hints"), "Scaffold profile planning must preserve caller-reviewed command matcher hints as deterministic profile draft data.")}, + "scaffold-project-structure": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/projectstructure/projectstructure_test.go", "TestBuildAdmitsProjectStructureScaffoldAndEmitsBoundedEnvelope", semanticRouteProof("projectstructure.build_admits_project_structure_scaffold_and_emits_bounded_envelope"), "Project structure scaffold must emit deterministic source-report identity and bounded agent guidance without writing files."), packageFalsifierRoute("internal/command/projectstructure/projectstructure_test.go", "TestBuildRejectsProjectStructurePathDriftAndUnsafePaths", semanticRouteProof("projectstructure.build_rejects_project_structure_path_drift_and_unsafe_paths"), "Project structure scaffold must reject unsafe paths and inconsistent bootstrap/profile proof paths.")}, + "selective-gate-evidence": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/selectivegateevidence/selectivegateevidence_test.go", "TestBuildRejectsMergeSatisfyingEvidenceWithoutProducerAdmission", semanticRouteProof("selectivegateevidence.build_rejects_merge_satisfying_evidence_without_producer_admission"), "Selective gate evidence must reject merge-satisfying evidence without producer admission."), packageFalsifierRoute("internal/command/selectivegateevidence/selectivegateevidence_test.go", "TestBuildReportsMergeEvidenceWithoutApprovingMerge", semanticRouteProof("selectivegateevidence.build_reports_merge_evidence_without_approving_merge"), "Selective gate evidence must report merge evidence facts without approving consumer-owned merge admission.")}, + "selective-gate-obligation-decision-input": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/selectivegateevidence/selectivegateevidence_test.go", "TestProjectObligationDecisionBuildsInputAndRejectsUnroutedCommand", semanticRouteProof("selectivegateevidence.project_obligation_decision_builds_input_and_rejects_unrouted_command"), "Selective evidence projection must reject receipts that cannot be routed to planned commands.")}, + "selective-gate-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/selectivegateplan/selectivegateplan_test.go", "TestBuildFailsClosedForUncoveredUnknownEdge", semanticRouteProof("selectivegateplan.build_fails_closed_for_uncovered_unknown_edge"), "Selective gate planning must fail closed for uncovered unknown dependency edges.")}, + "secret-scan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/secretscan/secretscan_test.go", "TestBuildFindsSecretLikeTextWithoutLeakingValue", semanticRouteProof("secretscan.build_finds_secret_like_text_without_leaking_value"), "Secret scan must detect secret-like text in explicit caller inventory without leaking matched values or scanning repository state.")}, + "self-check": {directCLIRoute("internal/app/app_test.go", "TestSelfCheckRejectsDuplicateKeys", semanticRouteProof("app.self_check_rejects_duplicate_keys"), "Self-check must reject ambiguous duplicate-key JSON without echoing the duplicated key.")}, + "spec-overview-claims": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/specoverviewclaims/specoverviewclaims_test.go", "TestBuildRejectsInvalidOverviewClaimBoundaryFacts", semanticRouteProof("specoverviewclaims.build_rejects_invalid_overview_claim_boundary_facts"), "Spec overview claim admission must reject invalid path, extraction, digest, marker, rationale, and non-claim boundary facts."), packageFalsifierRoute("internal/command/specoverviewclaims/specoverviewclaims_test.go", "TestBuildRejectsNonDurableRequirementCitationsForEveryNonDurableKind", semanticRouteProof("specoverviewclaims.build_rejects_non_durable_requirement_citations_for_every_non_durable_kind"), "Spec overview claim admission must reject every non-durable claim kind when it carries requirement citations.")}, + "spec-proof-bundle-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/specproofbundleadmission/specproofbundleadmission_test.go", "TestBuildRejectsForgedReceiptAdmissionChild", semanticRouteProof("specproofbundleadmission.build_rejects_forged_receipt_admission_child"), "Spec proof bundle admission must reject forged child receipt admission reports.")}, + "stack-preset": {directCLIRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("command_coverage.no_input_commands_have_command_specific_behavior"), "Stack preset CLI route must emit JSON and reject unknown preset flags."), packageFalsifierRoute("internal/command/stackpreset/stackpreset_test.go", "TestPresetInventoryIsCompleteDeterministicAndDefensivelyCopied", semanticRouteProof("stackpreset.preset_inventory_is_complete_deterministic_and_defensively_copied"), "Stack preset inventory must keep preset ids aligned with complete non-empty profile records and defensive copies."), packageFalsifierRoute("internal/command/stackpreset/stackpreset_test.go", "TestUnknownPresetIsRejected", semanticRouteProof("stackpreset.unknown_preset_is_rejected"), "Stack preset package API must reject unknown preset ids.")}, "test-evidence-inventory": { requiredInputAdmissionRoute, - packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier", semanticRouteProof("testevidenceinventory.build_rejects_incomplete_declared_oracle_metadata_and_duplicate_falsifier", commandCoverageExpectedPublicOutcome), "Test evidence inventory must reject incomplete caller-declared oracle metadata and duplicate falsifier equivalence claims."), - packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildDiscoveryDraftEmitsCandidateOnlyInventory", semanticRouteProof("testevidenceinventory.build_discovery_draft_emits_candidate_only_inventory", commandCoverageExpectedPublicOutcome), "Test discovery draft projection must emit candidate-only inventory guidance without closing semantic coverage."), - packageFalsifierRoute("internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go", "TestBuildRejectsDerivedCommandRefCollision", semanticRouteProof("proofbindingtestinventory.build_rejects_derived_command_ref_collision", commandCoverageExpectedPublicOutcome), "Proof-binding-derived inventory projection must reject command-ref collisions before emitting normalized inventory."), + packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier", semanticRouteProof("testevidenceinventory.build_rejects_incomplete_declared_oracle_metadata_and_duplicate_falsifier"), "Test evidence inventory must reject incomplete caller-declared oracle metadata and duplicate falsifier equivalence claims."), + packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildDiscoveryDraftEmitsCandidateOnlyInventory", semanticRouteProof("testevidenceinventory.build_discovery_draft_emits_candidate_only_inventory"), "Test discovery draft projection must emit candidate-only inventory guidance without closing semantic coverage."), + packageFalsifierRoute("internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go", "TestBuildRejectsDerivedCommandRefCollision", semanticRouteProof("proofbindingtestinventory.build_rejects_derived_command_ref_collision"), "Proof-binding-derived inventory projection must reject command-ref collisions before emitting normalized inventory."), }, - "text-policy": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/textpolicy/textpolicy_test.go", "TestEvaluatePreservesUTF8ASCIIWhitespaceAndBinaryFalsifiers", semanticRouteProof("textpolicy.evaluate_preserves_utf8_asciiwhitespace_and_binary_falsifiers", commandCoverageExpectedPublicOutcome), "Text policy must preserve UTF-8, ASCII, final-newline, trailing-whitespace, binary-suffix, missing-file, and explicit-inventory falsifiers without scanning repository state.")}, - "typescript-public-api-surfaces": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/publicapi/public_api_test.go", "TestVerifyTypeScriptPackagePublicAPIRejectsExportStar", semanticRouteProof("public_api.verify_type_script_package_public_apirejects_export_star", commandCoverageExpectedPublicOutcome), "TypeScript public API verifier must reject export-star surfaces that hide public contract drift."), packageFalsifierRoute("internal/command/publicapi/public_api_test.go", "TestVerifyTypeScriptPackagePublicAPIRejectsExportsFromDifferentDeclaredSource", semanticRouteProof("public_api.verify_type_script_package_public_apirejects_exports_from_different_declared_source", commandCoverageExpectedPublicOutcome), "TypeScript public API verifier must compare declared public exports against each explicitly referenced source file.")}, - "witness-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/witnessplan/witnessplan_test.go", "TestBuildAdmitsSafeCommandAndRejectsShellCommand", semanticRouteProof("witnessplan.build_admits_safe_command_and_rejects_shell_command", commandCoverageExpectedPublicOutcome), "Witness plan must preserve witness command safety policy and reject shell command execution."), packageFalsifierRoute("internal/command/witnessplan/witnessplan_test.go", "TestBuildProjectsRequirementBindingsToWitnessPlan", semanticRouteProof("witnessplan.build_projects_requirement_bindings_to_witness_plan", commandCoverageExpectedPublicOutcome), "Witness plan projection must derive witness commands from admitted requirement proof bindings without duplicating command identity.")}, - "witness-scheduler-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/witnessschedulerplan/witnessschedulerplan_test.go", "TestBuildRejectsUnsafeParallelWriteCollision", semanticRouteProof("witnessschedulerplan.build_rejects_unsafe_parallel_write_collision", commandCoverageExpectedPublicOutcome), "Witness scheduler planning must reject unsafe parallel write collisions.")}, - "workspace-changed-package-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspaceplanning/workspaceplanning_test.go", "TestChangedPackagePlanAdmitsPackagesRootAndSchema", semanticRouteProof("workspaceplanning.changed_package_plan_admits_packages_root_and_schema", commandCoverageExpectedPublicOutcome), "Workspace changed-package planning must admit packagesRoot only through explicit schema-versioned input.")}, - "workspace-manifest-facts": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go", "TestBuildProjectsManifestFactsAndPlanningInputs", semanticRouteProof("workspace_manifest_facts.build_projects_manifest_facts_and_planning_inputs", commandCoverageExpectedPublicOutcome), "Workspace manifest fact projection must emit registry-compatible facts and workspace planning inputs from explicit caller-owned manifests."), packageFalsifierRoute("internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go", "TestBuildRejectsUnsafeManifestPathAndDuplicatePackageIdentity", semanticRouteProof("workspace_manifest_facts.build_rejects_unsafe_manifest_path_and_duplicate_package_identity", commandCoverageExpectedPublicOutcome), "Workspace manifest fact projection must reject unsafe paths and duplicate package identities.")}, - "workspace-registry": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspaceregistry/workspaceregistry_test.go", "TestBuildAdmitsWorkspaceRegistryAndRejectsMissingScriptTarget", semanticRouteProof("workspaceregistry.build_admits_workspace_registry_and_rejects_missing_script_target", commandCoverageExpectedPublicOutcome), "Workspace registry admission must reject scripts targeting missing workspace packages.")}, - "workspace-shard-partition": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspaceplanning/workspaceplanning_test.go", "TestShardPartitionAdmitsCoveredRootsAndRejectsMissingDependency", semanticRouteProof("workspaceplanning.shard_partition_admits_covered_roots_and_rejects_missing_dependency", commandCoverageExpectedPublicOutcome), "Workspace shard partitioning must reject roots that depend on missing workspace packages.")}, + "text-policy": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/textpolicy/textpolicy_test.go", "TestEvaluatePreservesUTF8ASCIIWhitespaceAndBinaryFalsifiers", semanticRouteProof("textpolicy.evaluate_preserves_utf8_asciiwhitespace_and_binary_falsifiers"), "Text policy must preserve UTF-8, ASCII, final-newline, trailing-whitespace, binary-suffix, missing-file, and explicit-inventory falsifiers without scanning repository state.")}, + "typescript-public-api-surfaces": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/publicapi/public_api_test.go", "TestVerifyTypeScriptPackagePublicAPIRejectsExportStar", semanticRouteProof("public_api.verify_type_script_package_public_apirejects_export_star"), "TypeScript public API verifier must reject export-star surfaces that hide public contract drift."), packageFalsifierRoute("internal/command/publicapi/public_api_test.go", "TestVerifyTypeScriptPackagePublicAPIRejectsExportsFromDifferentDeclaredSource", semanticRouteProof("public_api.verify_type_script_package_public_apirejects_exports_from_different_declared_source"), "TypeScript public API verifier must compare declared public exports against each explicitly referenced source file.")}, + "witness-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/witnessplan/witnessplan_test.go", "TestBuildAdmitsSafeCommandAndRejectsShellCommand", semanticRouteProof("witnessplan.build_admits_safe_command_and_rejects_shell_command"), "Witness plan must preserve witness command safety policy and reject shell command execution."), packageFalsifierRoute("internal/command/witnessplan/witnessplan_test.go", "TestBuildProjectsRequirementBindingsToWitnessPlan", semanticRouteProof("witnessplan.build_projects_requirement_bindings_to_witness_plan"), "Witness plan projection must derive witness commands from admitted requirement proof bindings without duplicating command identity.")}, + "witness-scheduler-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/witnessschedulerplan/witnessschedulerplan_test.go", "TestBuildRejectsUnsafeParallelWriteCollision", semanticRouteProof("witnessschedulerplan.build_rejects_unsafe_parallel_write_collision"), "Witness scheduler planning must reject unsafe parallel write collisions.")}, + "workspace-changed-package-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspaceplanning/workspaceplanning_test.go", "TestChangedPackagePlanAdmitsPackagesRootAndSchema", semanticRouteProof("workspaceplanning.changed_package_plan_admits_packages_root_and_schema"), "Workspace changed-package planning must admit packagesRoot only through explicit schema-versioned input.")}, + "workspace-manifest-facts": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go", "TestBuildProjectsManifestFactsAndPlanningInputs", semanticRouteProof("workspace_manifest_facts.build_projects_manifest_facts_and_planning_inputs"), "Workspace manifest fact projection must emit registry-compatible facts and workspace planning inputs from explicit caller-owned manifests."), packageFalsifierRoute("internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go", "TestBuildRejectsUnsafeManifestPathAndDuplicatePackageIdentity", semanticRouteProof("workspace_manifest_facts.build_rejects_unsafe_manifest_path_and_duplicate_package_identity"), "Workspace manifest fact projection must reject unsafe paths and duplicate package identities.")}, + "workspace-registry": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspaceregistry/workspaceregistry_test.go", "TestBuildAdmitsWorkspaceRegistryAndRejectsMissingScriptTarget", semanticRouteProof("workspaceregistry.build_admits_workspace_registry_and_rejects_missing_script_target"), "Workspace registry admission must reject scripts targeting missing workspace packages.")}, + "workspace-shard-partition": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/workspaceplanning/workspaceplanning_test.go", "TestShardPartitionAdmitsCoveredRootsAndRejectsMissingDependency", semanticRouteProof("workspaceplanning.shard_partition_admits_covered_roots_and_rejects_missing_dependency"), "Workspace shard partitioning must reject roots that depend on missing workspace packages.")}, } -func semanticRouteProof(ref string, expectedPublicOutcome string) commandCoverageSemanticProof { - return commandCoverageSemanticProof{ - ref: ref, - expectedPublicOutcome: expectedPublicOutcome, - } +func semanticRouteProof(ref string) commandCoverageSemanticProof { + return commandCoverageSemanticProof{ref: ref} } func directCLIRoute(file string, testName string, proof commandCoverageSemanticProof, rationale string) commandCoverageRoute { @@ -286,7 +279,7 @@ func (route commandCoverageRoute) sourceOracleBinding(command string) commandCov WrongImplementationClass: proof.wrongImplementationClassID(), OracleID: proof.oracleID(), OracleKind: "semantic_route_falsifier", - ExpectedPublicOutcome: proof.expectedPublicOutcome, + ExpectedPublicOutcome: route.rationale, } } @@ -336,8 +329,8 @@ func (route commandCoverageRoute) semanticProofProblem() string { if ref == "" { return "semantic coverage route requires owner-declared proof metadata" } - if strings.TrimSpace(route.semanticProof.expectedPublicOutcome) == "" { - return "semantic coverage route requires owner-declared expected public outcome" + if strings.TrimSpace(route.rationale) == "" { + return "semantic coverage route requires a route-specific expected public outcome" } if containsRouteIndexToken(ref) { return "semantic proof metadata must not be derived from route index" diff --git a/internal/app/command_coverage_source.go b/internal/app/command_coverage_source.go index 3ef9459..033caab 100644 --- a/internal/app/command_coverage_source.go +++ b/internal/app/command_coverage_source.go @@ -24,6 +24,13 @@ func routeSemanticSourceProblem(command string, route commandCoverageRoute) stri return goTestSemanticOracleProblem(filePath, route.testName, route.sourceOracleMarker(command)) } +func routeSemanticSourceProblemAtRoot(command string, route commandCoverageRoute, root string) string { + if !route.isSemanticCandidate() { + return "" + } + return goTestSemanticOracleProblem(filepath.Join(root, filepath.FromSlash(route.file)), route.testName, route.sourceOracleMarker(command)) +} + func commandCoverageRouteFilePath(routeFile string) (string, error) { if filepath.IsAbs(routeFile) { return routeFile, nil diff --git a/internal/app/command_coverage_test.go b/internal/app/command_coverage_test.go index daf1583..05add1d 100644 --- a/internal/app/command_coverage_test.go +++ b/internal/app/command_coverage_test.go @@ -39,14 +39,14 @@ func TestSupportedCommandsHaveExplicitCoverageRoutes(t *testing.T) { } func TestCommandCoverageRejectsPackageRouteToAppSmokeTest(t *testing.T) { - route := packageFalsifierRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("test.unrelated_app_smoke", commandCoverageExpectedPublicOutcome), "Unrelated app smoke must not satisfy package-level semantic coverage.") + route := packageFalsifierRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("test.unrelated_app_smoke"), "Unrelated app smoke must not satisfy package-level semantic coverage.") if problem := routeSemanticOwnerProblem("registry-consumer", route); problem == "" { t.Fatal("package-level semantic route to app smoke test was admitted") } } func TestCommandCoverageInventoryRejectsSemanticRouteOutsideOwnerScope(t *testing.T) { - route := packageFalsifierRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("test.unrelated_app_smoke", commandCoverageExpectedPublicOutcome), "Unrelated app smoke must not satisfy package-level semantic coverage.") + route := packageFalsifierRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("test.unrelated_app_smoke"), "Unrelated app smoke must not satisfy package-level semantic coverage.") if _, err := commandCoverageInventoryFrom(map[string][]commandCoverageRoute{"registry-consumer": {route}}); err == nil { t.Fatal("production command coverage inventory builder admitted a semantic route outside the command owner scope") } @@ -56,7 +56,7 @@ func TestCommandCoverageInventoryRejectsSameOwnerUnrelatedNonEmptyTest(t *testin route := packageFalsifierRoute( "internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAddsMandatoryBoundaryNonClaims", - semanticRouteProof("registryconsumer.registry_consumer_accepts_registry_release_proof", commandCoverageExpectedPublicOutcome), + semanticRouteProof("registryconsumer.registry_consumer_accepts_registry_release_proof"), "Same-owner unrelated assertion must not satisfy registry-consumer release-proof semantic coverage.", ) _, err := commandCoverageInventoryFrom(map[string][]commandCoverageRoute{"registry-consumer": {route}}) @@ -66,21 +66,21 @@ func TestCommandCoverageInventoryRejectsSameOwnerUnrelatedNonEmptyTest(t *testin } func TestCommandCoverageRejectsPackageRouteToDifferentCommandPackage(t *testing.T) { - route := packageFalsifierRoute("internal/command/externalconsumer/externalconsumer_test.go", "TestBuildAdmitsExternalConsumerProofAndRejectsWorkspaceLock", semanticRouteProof("test.unrelated_command_package", commandCoverageExpectedPublicOutcome), "Unrelated command package must not satisfy registry-consumer semantic coverage.") + route := packageFalsifierRoute("internal/command/externalconsumer/externalconsumer_test.go", "TestBuildAdmitsExternalConsumerProofAndRejectsWorkspaceLock", semanticRouteProof("test.unrelated_command_package"), "Unrelated command package must not satisfy registry-consumer semantic coverage.") if problem := routeSemanticOwnerProblem("registry-consumer", route); problem == "" { t.Fatal("package-level semantic route to unrelated command package was admitted") } } func TestCommandCoverageRejectsDirectRouteToDifferentAppCommand(t *testing.T) { - route := directCLIRoute("internal/app/cli_abi_test.go", "TestRequirementBrowserServerSpecTreeCLIABI", semanticRouteProof("test.unrelated_app_cli_abi", commandCoverageExpectedPublicOutcome), "Unrelated app CLI ABI test must not satisfy adoption-doctor semantic coverage.") + route := directCLIRoute("internal/app/cli_abi_test.go", "TestRequirementBrowserServerSpecTreeCLIABI", semanticRouteProof("test.unrelated_app_cli_abi"), "Unrelated app CLI ABI test must not satisfy adoption-doctor semantic coverage.") if problem := routeSemanticOwnerProblem("adoption-doctor", route); problem == "" { t.Fatal("direct app semantic route to unrelated command was admitted") } } func TestCommandCoverageRejectsRouteWithoutDescriptorOwner(t *testing.T) { - route := packageFalsifierRoute("internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", semanticRouteProof("test.unsupported_descriptor_owner", commandCoverageExpectedPublicOutcome), "Unsupported command must not satisfy package-level semantic coverage.") + route := packageFalsifierRoute("internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", semanticRouteProof("test.unsupported_descriptor_owner"), "Unsupported command must not satisfy package-level semantic coverage.") if problem := routeSemanticOwnerProblem("unsupported-command", route); problem == "" { t.Fatal("package-level semantic route without descriptor owner was admitted") } @@ -99,12 +99,12 @@ func TestCommandCoverageRejectsSemanticRouteWithoutProofMetadata(t *testing.T) { } func TestCommandCoverageRejectsSemanticRouteWithoutExpectedOutcome(t *testing.T) { - route := packageFalsifierRoute( - "internal/command/registryconsumer/registryconsumer_test.go", - "TestRegistryConsumerAcceptsRegistryReleaseProof", - commandCoverageSemanticProof{ref: "registryconsumer.accepts_registry_release_proof"}, - "Semantic command route must bind an expected public outcome.", - ) + route := commandCoverageRoute{ + file: "internal/command/registryconsumer/registryconsumer_test.go", + kind: "package_level_falsifier", + semanticProof: semanticRouteProof("registryconsumer.accepts_registry_release_proof"), + testName: "TestRegistryConsumerAcceptsRegistryReleaseProof", + } if _, err := commandCoverageInventoryFrom(map[string][]commandCoverageRoute{"registry-consumer": {route}}); err == nil { t.Fatal("semantic route without expected public outcome was admitted by production inventory builder") } @@ -114,7 +114,7 @@ func TestCommandCoverageRejectsRouteIndexDerivedSemanticProofID(t *testing.T) { route := packageFalsifierRoute( "internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", - semanticRouteProof("registryconsumer.route_7", commandCoverageExpectedPublicOutcome), + semanticRouteProof("registryconsumer.route_7"), "Semantic command route must not derive proof identity from route order.", ) if _, err := commandCoverageInventoryFrom(map[string][]commandCoverageRoute{"registry-consumer": {route}}); err == nil { @@ -126,7 +126,7 @@ func TestCommandCoverageRejectsProseDerivedSemanticProofID(t *testing.T) { route := packageFalsifierRoute( "internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", - semanticRouteProof("Registry consumer accepts release proof", commandCoverageExpectedPublicOutcome), + semanticRouteProof("Registry consumer accepts release proof"), "Semantic command route must not derive proof identity from prose.", ) if _, err := commandCoverageInventoryFrom(map[string][]commandCoverageRoute{"registry-consumer": {route}}); err == nil { @@ -136,7 +136,7 @@ func TestCommandCoverageRejectsProseDerivedSemanticProofID(t *testing.T) { func TestCommandCoverageRejectsRouteOnlyWithSemanticProofMetadata(t *testing.T) { route := requiredInputAdmissionRoute - route.semanticProof = semanticRouteProof("test.route_only_with_semantic_proof", commandCoverageExpectedPublicOutcome) + route.semanticProof = semanticRouteProof("test.route_only_with_semantic_proof") if _, err := commandCoverageInventoryFrom(map[string][]commandCoverageRoute{"registry-consumer": {route}}); err == nil { t.Fatal("route-only smoke accepted semantic proof metadata in production inventory builder") } @@ -184,11 +184,58 @@ func TestCommandCoverageInventoryIsAdmittedAndBindsProofRouteCandidates(t *testi } } +func TestCommandCoverageOracleCandidatesAreExactAndRouteSpecific(t *testing.T) { + candidates, err := CommandCoverageOracleCandidates() + if err != nil { + t.Fatalf("CommandCoverageOracleCandidates() error = %v", err) + } + wantCount := 0 + for _, summary := range CommandCoverageSummaries() { + wantCount += summary.ProofRouteCandidateCount + } + if len(candidates) != wantCount || len(candidates) == 0 { + t.Fatalf("candidate count = %d, want %d", len(candidates), wantCount) + } + seenMarkers := map[string]struct{}{} + seenOutcomes := map[string]struct{}{} + for index, candidate := range candidates { + if strings.TrimSpace(candidate.ExpectedPublicOutcome) == "" { + t.Fatalf("candidate has no route-specific outcome: %#v", candidate) + } + if _, exists := seenMarkers[candidate.SourceMarker]; exists { + t.Fatalf("duplicate candidate marker %s", candidate.SourceMarker) + } + seenMarkers[candidate.SourceMarker] = struct{}{} + seenOutcomes[candidate.ExpectedPublicOutcome] = struct{}{} + if index > 0 && strings.Join(candidateIdentity(candidates[index-1]), "\x00") >= strings.Join(candidateIdentity(candidate), "\x00") { + t.Fatalf("candidates are not in strict canonical order at %d", index) + } + } + if len(seenOutcomes) < 2 { + t.Fatalf("route-specific outcomes collapsed: %#v", seenOutcomes) + } +} + +func TestCommandCoverageSourceMarkerBindsRouteSpecificOutcome(t *testing.T) { + route := packageFalsifierRoute( + "internal/command/registryconsumer/registryconsumer_test.go", + "TestRegistryConsumerAcceptsRegistryReleaseProof", + semanticRouteProof("registryconsumer.marker_binds_outcome"), + "Registry consumer accepts only an admitted registry release proof.", + ) + first := route.sourceOracleMarker("registry-consumer") + route.rationale = "Registry consumer rejects a registry release proof with mismatched identity." + second := route.sourceOracleMarker("registry-consumer") + if first == second { + t.Fatalf("route-specific outcome did not change source marker: %s", first) + } +} + func TestCommandCoverageInventoryProjectsStableCandidateRefWithoutSemanticProof(t *testing.T) { route := packageFalsifierRoute( "internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", - semanticRouteProof("registryconsumer.accepts_registry_release_proof", commandCoverageExpectedPublicOutcome), + semanticRouteProof("registryconsumer.accepts_registry_release_proof"), "Registry consumer release proof must be tied to an owner-declared semantic proof identity.", ) entry := route.inventoryEntry("registry-consumer", 99) @@ -327,7 +374,7 @@ func TestGoTestSemanticOracleProblemRejectsMismatchedBindingFact(t *testing.T) { route := packageFalsifierRoute( "internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", - semanticRouteProof("registryconsumer.registry_consumer_accepts_registry_release_proof", commandCoverageExpectedPublicOutcome), + semanticRouteProof("registryconsumer.registry_consumer_accepts_registry_release_proof"), "Registry consumer release proof must be tied to an owner-declared semantic proof identity.", ) filePath := filepath.Join(repoRoot(t), route.file) @@ -362,7 +409,7 @@ func TestUnreachableFatalCannotBecomeSemanticEvidence(t *testing.T) { route := packageFalsifierRoute( "internal/command/registryconsumer/registryconsumer_test.go", "TestRegistryConsumerAcceptsRegistryReleaseProof", - semanticRouteProof("registryconsumer.unreachable_fatal_regression", commandCoverageExpectedPublicOutcome), + semanticRouteProof("registryconsumer.unreachable_fatal_regression"), "An unreachable assertion must remain candidate-only.", ) marker := route.sourceOracleMarker("registry-consumer") @@ -419,7 +466,7 @@ func TestRequiredInputCommandsRejectMalformedCallerRecords(t *testing.T) { } func TestNoInputCommandsHaveCommandSpecificBehavior(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.069735802754139690645953016388349571937723357477618679005916661691281309389599") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.102282482911729620466899441592793892640151459429551384318004264301951934933567") t.Run("stack-preset", func(t *testing.T) { var stdout bytes.Buffer var stderr bytes.Buffer diff --git a/internal/app/requirement_context_cli_test.go b/internal/app/requirement_context_cli_test.go index 54a3e71..97d412d 100644 --- a/internal/app/requirement_context_cli_test.go +++ b/internal/app/requirement_context_cli_test.go @@ -21,10 +21,10 @@ import ( ) func TestRequirementContextCommandsComposeThroughWholeCLI(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.078417255856588541640488533337296521071324425566921898314006295059346651375053") - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.026918779924949500817883735459777435749569395463462098851750112168717349371591") - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.037106612382996619981621102911104931496378404893868406197919247815645123874441") - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.023947615033231006584057272018250251823714544315759467896512469485975523531624") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.015005723751943372102052485735054220409079175497974852772238314306849909704926") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.049495764657602855686625469543032083639688769989400372107516147133048789250421") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.057489097167034237791564307746461892229693375121512105387246196474554343257848") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.011976964944325039688029974963654018529660938760094284739788591455188270918568") root := t.TempDir() tree := map[string]any{ "schemaVersion": json.Number("2"), "treeId": "consumer.spec-tree", "rootNodeId": "consumer.root", diff --git a/internal/command/adoptionchecklist/adoptionchecklist_test.go b/internal/command/adoptionchecklist/adoptionchecklist_test.go index 1bc5c9e..20c3038 100644 --- a/internal/command/adoptionchecklist/adoptionchecklist_test.go +++ b/internal/command/adoptionchecklist/adoptionchecklist_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildClassifiesRequiredChecklistItemsAndPreservesOptionalNonFailures(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.029885892657284911864832253608444920506806901698647310573488696226218711680289") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.069749879516188799356142053767399217128942865655819702242129484856747289540132") record, exitCode, err := Build(validChecklistInput()) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/command/adoptioncontract/adoptioncontract_test.go b/internal/command/adoptioncontract/adoptioncontract_test.go index 06c6fca..7670a25 100644 --- a/internal/command/adoptioncontract/adoptioncontract_test.go +++ b/internal/command/adoptioncontract/adoptioncontract_test.go @@ -156,7 +156,7 @@ func TestValidateOptionsOwnsModePilotAndCompatibilityPolicy(t *testing.T) { } func TestBuildDelegatesModesWithParity(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.064840772979561884532437014395024443929767634378814122316392969426222714882392") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.073115937715906245510885594108709053081835764670748024087132020466101700061433") cases := []struct { name string options Options diff --git a/internal/command/adoptiondoctor/adoptiondoctor_test.go b/internal/command/adoptiondoctor/adoptiondoctor_test.go index 834cd5a..c08a4e0 100644 --- a/internal/command/adoptiondoctor/adoptiondoctor_test.go +++ b/internal/command/adoptiondoctor/adoptiondoctor_test.go @@ -77,7 +77,7 @@ func TestBuildEnforceTouchedSkipsGapsOutsideTouchedSelection(t *testing.T) { } func TestBuildFailsEnforcementForCandidateBoundaryAndMissingRoutes(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.007421917124002404912211613315239198743059540465995158650367800592497462368119") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.009316750311738530526586138484790439891684089174716743044696722390531958762956") input := baseInput() input["mode"] = "enforce-all" input["checkedScope"] = "all" diff --git a/internal/command/adoptionworkflow/adoptionworkflow_test.go b/internal/command/adoptionworkflow/adoptionworkflow_test.go index 0eccae5..1521301 100644 --- a/internal/command/adoptionworkflow/adoptionworkflow_test.go +++ b/internal/command/adoptionworkflow/adoptionworkflow_test.go @@ -18,7 +18,7 @@ func TestBuildRejectsUnknownInputRefField(t *testing.T) { } func TestBuildGeneratesBoundedCommandArgv(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.037862658707003067560912417742828567812344707083894145862800768783923813459857") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.071485626206261445985975684684155922995969280410048467184237861241913337881344") result, err := BuildResult(validWorkflowInput()) if err != nil { t.Fatalf("BuildResult() error=%v", err) diff --git a/internal/command/agentroute/agentroute_test.go b/internal/command/agentroute/agentroute_test.go index 3aad0f6..79dfe75 100644 --- a/internal/command/agentroute/agentroute_test.go +++ b/internal/command/agentroute/agentroute_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildRoutesRequirementSourceAndBlocksUnknownGoal(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.108125484608845771285185958596730048384317499814813362270908806756918268107297") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.052032679560954279991581640663936146133283843978254811517731144801933075103271") t.Parallel() report, exitCode, err := Build(map[string]any{ @@ -1318,7 +1318,7 @@ func TestBuildEnvelopeKeepsMachineRefIDsBoundedForLongInputRefs(t *testing.T) { } func TestBuildEnvelopeKeepsBlockedRoutesAsStopSignals(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.022893294120171709584005915658023363866013069336456249164413421169696538215866") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.039089309512279322876166263378306645627644382088544574407694362092047953400284") t.Parallel() envelope, exitCode, err := BuildEnvelope(map[string]any{ @@ -1350,7 +1350,7 @@ func TestBuildEnvelopeKeepsBlockedRoutesAsStopSignals(t *testing.T) { } func TestBuildEnvelopeCarriesBlockedObservedReportPreconditions(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.070004323406011051259833433994481535056161810177395705776549338061243536668298") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.112527168675876041372970118427108526301468859106766054318947512648421264085367") t.Parallel() envelope, exitCode, err := BuildEnvelope(map[string]any{ diff --git a/internal/command/bindingpartition/bindingpartition_test.go b/internal/command/bindingpartition/bindingpartition_test.go index 7055dd9..b441319 100644 --- a/internal/command/bindingpartition/bindingpartition_test.go +++ b/internal/command/bindingpartition/bindingpartition_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildRejectsCrossSurfaceRouteReferenceWithoutDelegation(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.007367819136484564623482005833685691497213272711615520825513165365000892708767") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.052171025929495006005566277017011611265246234017302535159618229629097907645234") record, exitCode, err := Build(validBindingPartitionInput(false)) if err != nil { t.Fatalf("Build() error = %v", err) diff --git a/internal/command/branchauthority/branchauthority_test.go b/internal/command/branchauthority/branchauthority_test.go index 5978804..941eeb4 100644 --- a/internal/command/branchauthority/branchauthority_test.go +++ b/internal/command/branchauthority/branchauthority_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAdmitsAlignedRequiredBranchAndRejectsRequiredDrift(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.059237477459693884008799222027390418858342772482064960101818015827014505791733") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.071179729500816627799208554853464300619289513317734324260194435111694229088770") record, exitCode, err := Build(validBranchAuthorityInput("main")) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/command/capabilitymapadmission/capability_map_admission_test.go b/internal/command/capabilitymapadmission/capability_map_admission_test.go index 144fb9c..c8c86ac 100644 --- a/internal/command/capabilitymapadmission/capability_map_admission_test.go +++ b/internal/command/capabilitymapadmission/capability_map_admission_test.go @@ -36,7 +36,7 @@ func TestBuildCodeBaselineEmitsCandidateRequirementsAndBindings(t *testing.T) { } func TestBuildCodeBaselineFailsMissingCandidateRequirementAndAnchor(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.059300390373196834575756311871568010867803598706625444326212661238632859496824") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.112196325409848896148926885908046594150757046298431637036021245352716060102765") t.Parallel() input := validCapabilityMapInput("code_baseline") diff --git a/internal/command/changedpathset/changedpathset_test.go b/internal/command/changedpathset/changedpathset_test.go index 9dcc9af..b122860 100644 --- a/internal/command/changedpathset/changedpathset_test.go +++ b/internal/command/changedpathset/changedpathset_test.go @@ -66,7 +66,7 @@ func validateContextJSONPointers(output any, contextRefs []any) (int, error) { } func TestBuildDeduplicatesAndFailsClosedOnInvalidPaths(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.020996495977209692976965486603091189537178619363476268911749132800454063351641") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.060535997331386831986160802626392669912987256775670711883112001232124039341072") result, err := Build(map[string]any{ "schemaVersion": json.Number("1"), "reportId": "proofkit.test.changed-path-set", diff --git a/internal/command/completioncriteria/completioncriteria_test.go b/internal/command/completioncriteria/completioncriteria_test.go index a646075..861053d 100644 --- a/internal/command/completioncriteria/completioncriteria_test.go +++ b/internal/command/completioncriteria/completioncriteria_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildBlocksUnsatisfiedBlockingCriterion(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.080845831970041108238055543444258074502099931881803185088012985084173569192906") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.013180397593748965161681129039250936414813610855632261268268126314868701130095") record, exitCode, err := Build(validCompletionCriteriaInput("satisfied")) if err != nil { t.Fatalf("Build() error = %v", err) diff --git a/internal/command/conformanceprofile/conformanceprofile_test.go b/internal/command/conformanceprofile/conformanceprofile_test.go index 40d3b0f..a987e7e 100644 --- a/internal/command/conformanceprofile/conformanceprofile_test.go +++ b/internal/command/conformanceprofile/conformanceprofile_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildProfileResolvesRequiredSurfaceAndRejectsMissingSurface(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.058560543430386671975862540474631351828433682994553568983923953081174649721029") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.018828880652012560180983294316467397502747610605037816609704340855648617186412") result, err := BuildProfile(validConformanceProfileInput(), "local") if err != nil { t.Fatalf("BuildProfile() error=%v", err) @@ -38,7 +38,7 @@ func TestBuildProfileResolvesRequiredSurfaceAndRejectsMissingSurface(t *testing. } func TestBuildVerificationRejectsDuplicateProfiles(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.059839663405963145056826817506435791735368004114196862540174073017127636205209") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.049942093994206845167684609448858504077053804318310280433394779107021396059359") input := validConformanceProfileInput() profiles := input["manifest"].(map[string]any)["profiles"].([]any) input["manifest"].(map[string]any)["profiles"] = append(profiles, profiles[0]) @@ -76,7 +76,7 @@ func TestBuildVerificationRejectsSecretLikeReportVisibleText(t *testing.T) { } func TestListReturnsSortedProfileIDsAndRejectsInvalidInput(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.087092535541262698904962014459883891376365242892684917393349609831025338348503") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.051339077909994076367504988840992439039749956644468897314149305148259581754885") input := validConformanceProfileInput() manifest := input["manifest"].(map[string]any) profile := manifest["profiles"].([]any)[0].(map[string]any) diff --git a/internal/command/customruleboundary/customruleboundary_test.go b/internal/command/customruleboundary/customruleboundary_test.go index 0d7016c..6b7cb97 100644 --- a/internal/command/customruleboundary/customruleboundary_test.go +++ b/internal/command/customruleboundary/customruleboundary_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildAdmitsBoundedCustomRuleAndRejectsUnsafeEffects(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.106806151242803002240171487302316779109328290727609414333938322871488003612890") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.025087881226507484990874681833161710116470838935948976618942966583527556385445") record, exitCode, err := Build(validCustomRuleBoundaryInput()) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/command/deploymentevidenceadmission/deployment_evidence_admission_test.go b/internal/command/deploymentevidenceadmission/deployment_evidence_admission_test.go index 9642e83..bfe0e1f 100644 --- a/internal/command/deploymentevidenceadmission/deployment_evidence_admission_test.go +++ b/internal/command/deploymentevidenceadmission/deployment_evidence_admission_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAdmitsCandidateEvidenceAndRejectsUnpinnedImages(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.079783886865424080333074209821707019130265643884720844463730606151075706943505") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.069418897389528177020444235635879439930452356193962278098825404413216668484505") record, exitCode, err := Build(validDeploymentEvidenceInput()) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/command/documentlifecycle/documentlifecycle_test.go b/internal/command/documentlifecycle/documentlifecycle_test.go index 6e5ee71..307601d 100644 --- a/internal/command/documentlifecycle/documentlifecycle_test.go +++ b/internal/command/documentlifecycle/documentlifecycle_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildAdmitsCurrentDurableDocumentAndRejectsAuthorityDrift(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.062727261506172699203502066235963181372003705299463204256348067594015852290549") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.002977780838547275321268818409231101675805953197916588621169638796768812025486") record, exitCode, err := Build(validDocumentLifecycleInput()) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/command/externalconsumer/externalconsumer_test.go b/internal/command/externalconsumer/externalconsumer_test.go index c8cb3a6..ff24bc9 100644 --- a/internal/command/externalconsumer/externalconsumer_test.go +++ b/internal/command/externalconsumer/externalconsumer_test.go @@ -11,7 +11,7 @@ import ( ) func TestBuildAdmitsExternalConsumerProofAndRejectsWorkspaceLock(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.012498104838553757060272339493203313996229216776537383465605902427034061329443") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.030059518418876335628059921154117881105808166179626900465875767444982225149190") input := validExternalConsumerInput(t) record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/gradualadoption/gradualadoption_test.go b/internal/command/gradualadoption/gradualadoption_test.go index 789a68f..005808c 100644 --- a/internal/command/gradualadoption/gradualadoption_test.go +++ b/internal/command/gradualadoption/gradualadoption_test.go @@ -37,7 +37,7 @@ func TestBuildReportsUnknownNestedGradualAdoptionFields(t *testing.T) { } func TestBuildRejectsRollbackShellControlCommand(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.000552315326374361122976245190825208897136607688049300862777999079678067750852") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.035266010797425408239887120588026500100025591003049779646439503189706679453150") input := validAdoptionInput() input["rollback"].(map[string]any)["disableCommand"] = "remove proofkit report && curl example.test" @@ -169,7 +169,7 @@ func assertMaterializedCallerCommandPrefix(t *testing.T, manifest map[string]any } func TestBootstrapRejectsUnknownRootAndNestedFields(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.051261371983756429229417344656679944334685270955078781853250259863842037139662") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.111414333073228644561471369108062338215611265991082424538014863654404666653540") cases := []struct { name string mutate func(map[string]any) diff --git a/internal/command/gradualadoption/guidance_test.go b/internal/command/gradualadoption/guidance_test.go index 097ae0b..fc0b174 100644 --- a/internal/command/gradualadoption/guidance_test.go +++ b/internal/command/gradualadoption/guidance_test.go @@ -62,7 +62,7 @@ func TestGuidanceKeepsCommandOwnedNonClaims(t *testing.T) { } func TestGuidanceEnforcementFailsClosedForCandidateBoundaries(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.014122077686373699171401312512146978765072555275124478915795819268469905855861") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.021210643414670754890972585342688421403224755441264357584078647180199403727299") cases := []struct { name string mode string diff --git a/internal/command/impact/impact_test.go b/internal/command/impact/impact_test.go index 3221e7d..b4dd072 100644 --- a/internal/command/impact/impact_test.go +++ b/internal/command/impact/impact_test.go @@ -85,7 +85,7 @@ func TestBuildRejectsShellControlTokensInObligationCommands(t *testing.T) { } func TestBuildRoutesChangedRecordToObligationAndRejectsUnboundProofChange(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.015680759158531317531499583930144280843140752606638002248368777967073648841036") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.004126818763873109094701885631231832544658829793890758986188782288995889572211") input := validImpactInput() input["changedRecordIds"] = []any{"REQ-PROOFKIT-001"} input["obligationCatalog"] = []any{ diff --git a/internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go b/internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go index e02507b..205c46c 100644 --- a/internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go +++ b/internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go @@ -163,7 +163,7 @@ func TestGeneratedSourceAvoidsGenericIndexedAssignmentDrift(t *testing.T) { } func TestGeneratedTypeScriptAdapterExecutesCoreSemantics(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.113810164796311271953578736163663817358813962387525070369353049827898529598997") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.053185673671850315997459957102069176684420983647605639631428960314206272025809") nodePath, err := exec.LookPath("node") if err != nil { t.Fatalf("node is required to prove generated TypeScript semantics: %v", err) diff --git a/internal/command/migrationparityadmission/migrationparityadmission_test.go b/internal/command/migrationparityadmission/migrationparityadmission_test.go index 3c73afd..eec86a5 100644 --- a/internal/command/migrationparityadmission/migrationparityadmission_test.go +++ b/internal/command/migrationparityadmission/migrationparityadmission_test.go @@ -14,7 +14,7 @@ const testDigest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa const otherDigest = "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" func TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.087858357304530014219351528044382591532264544185224319857347811230381754931058") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.031283583526771024463426071371708324157925862677375589108846795108748604685801") input := validMigrationParityInput() record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/migrationplan/migrationplan_test.go b/internal/command/migrationplan/migrationplan_test.go index efdd332..eab59bb 100644 --- a/internal/command/migrationplan/migrationplan_test.go +++ b/internal/command/migrationplan/migrationplan_test.go @@ -8,7 +8,7 @@ import ( ) func TestSortedFollowUpCommandsRejectsShellControlTokens(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.082298967063760496779599754309164283301628231265151485187770019052237054259416") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.068167811965828868524495721027219601964718026488781175287330661603079334133111") _, err := sortedFollowUpCommands([]any{map[string]any{ "command": "go test ./... && curl example.test", "commandId": "proofkit.followup", diff --git a/internal/command/obligationdecision/obligationdecision_test.go b/internal/command/obligationdecision/obligationdecision_test.go index a3eec51..621880a 100644 --- a/internal/command/obligationdecision/obligationdecision_test.go +++ b/internal/command/obligationdecision/obligationdecision_test.go @@ -9,7 +9,7 @@ import ( ) func TestBuildAdmitsSatisfiedBlockingObligationsAndRejectsMissingReceipt(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.050614412851824410481957717232202238484677067491629728235402344049400709677602") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.001377148545194323854964413239799966083945326665458969842188702848644583442833") result, err := Build(validObligationDecisionInput("satisfied", "not_applicable")) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/command/packageruntimedependency/package_runtime_dependency_test.go b/internal/command/packageruntimedependency/package_runtime_dependency_test.go index 2cb41c1..96a0d31 100644 --- a/internal/command/packageruntimedependency/package_runtime_dependency_test.go +++ b/internal/command/packageruntimedependency/package_runtime_dependency_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAdmitsExternalRuntimeDependencyAndRejectsWorkspaceResolution(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.027617011738725950397530096345580960352335933116627715389463795934691621119686") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.022262543555161298264303898252120278311624176226118336531091301958798480297007") input := validPackageRuntimeDependencyInput() record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/pilotadmission/pilotadmission_test.go b/internal/command/pilotadmission/pilotadmission_test.go index cf78df4..7d6949a 100644 --- a/internal/command/pilotadmission/pilotadmission_test.go +++ b/internal/command/pilotadmission/pilotadmission_test.go @@ -18,7 +18,7 @@ func TestBuildAcceptsCompletePilotContract(t *testing.T) { } func TestBuildRejectsUnknownPilotContractField(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.050451179684877268396400075572263131598063826486749434897893969628542100806612") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.057912462745542653837414866608340350115558004021771379883888595693744581144285") input := validPilotInput() input["ignoredPolicy"] = true diff --git a/internal/command/producerpolicyselfproof/producerpolicyselfproof_test.go b/internal/command/producerpolicyselfproof/producerpolicyselfproof_test.go index 86b9929..2d97b05 100644 --- a/internal/command/producerpolicyselfproof/producerpolicyselfproof_test.go +++ b/internal/command/producerpolicyselfproof/producerpolicyselfproof_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildRejectsPolicyChangeProvedByNewlyAdmittedProducerTuple(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.051973025282364233365946493539636279354221690599943144823854134459401810936427") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.023031113506101401680342008180057163880726687061377420204317138240585982404187") record, exitCode, err := Build(validProducerPolicySelfProofInput()) if err != nil { t.Fatalf("Build() error = %v", err) diff --git a/internal/command/projectstructure/projectstructure_test.go b/internal/command/projectstructure/projectstructure_test.go index 56e2ae4..6b6e2c7 100644 --- a/internal/command/projectstructure/projectstructure_test.go +++ b/internal/command/projectstructure/projectstructure_test.go @@ -14,7 +14,7 @@ import ( ) func TestBuildAdmitsProjectStructureScaffoldAndEmitsBoundedEnvelope(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.059212367917167148851747700357371540728657804008449562600146109586760618366690") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.077605368601324151625037603028023832943347653489996031097831539301304492489683") result, err := BuildResult(validProjectStructureInput()) if err != nil { t.Fatalf("BuildResult() error=%v", err) @@ -110,7 +110,7 @@ func TestBuildRejectsSecretLikeProjectStructureNonClaims(t *testing.T) { } func TestBuildRejectsProjectStructurePathDriftAndUnsafePaths(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.036326342746928185666570492555700463770379515182536792074657611113385319861032") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.101379439615551172730953420664832137049480746789667740612911039267803529974896") cases := []struct { name string mutate func(map[string]any) diff --git a/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go b/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go index 6073553..e08e0f0 100644 --- a/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go +++ b/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go @@ -111,7 +111,7 @@ func TestBuildRejectsUnsafeStructuredFalsificationSelector(t *testing.T) { } func TestBuildRejectsDerivedCommandRefCollision(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.045998791748895126484384867727425029606933562065763381226241226336418593975574") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.015764554175908530517697174697685677859930004803180746537891295020670383915624") input := validInput() binding := input["compactProofContract"].(map[string]any)["bindings"].([]any)[0].([]any) falsification := binding[9].([]any) diff --git a/internal/command/proofobligationalgebra/proof_obligation_algebra_test.go b/internal/command/proofobligationalgebra/proof_obligation_algebra_test.go index e47c3c7..d9f5b86 100644 --- a/internal/command/proofobligationalgebra/proof_obligation_algebra_test.go +++ b/internal/command/proofobligationalgebra/proof_obligation_algebra_test.go @@ -176,7 +176,7 @@ func equalStrings(left []string, right []string) bool { } func TestBuildAdmitsAtomicObligationAndRejectsMissingRoute(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.102661548092694621144975813646090608936469525015523708393935225916992472621290") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.007102825729733194306222688479474522145093834474110859044235069588057227816672") input := validProofObligationAlgebraInput() record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/proofreceiptadmission/proofreceiptadmission_test.go b/internal/command/proofreceiptadmission/proofreceiptadmission_test.go index 3ded7ea..5f1a0ff 100644 --- a/internal/command/proofreceiptadmission/proofreceiptadmission_test.go +++ b/internal/command/proofreceiptadmission/proofreceiptadmission_test.go @@ -10,7 +10,7 @@ import ( ) func TestBuildAdmitsAdvisoryReceiptAndRejectsMergeSatisfyingWithoutProvenance(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.076311225906412383451940890026527143214204102348303993661137717946500939722995") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.046833331227659402056123694228405967213258150200955697512043665799069784736285") record, exitCode, err := Build(validProofReceiptInput()) if err != nil { t.Fatalf("Build() error = %v", err) diff --git a/internal/command/publicapi/public_api_test.go b/internal/command/publicapi/public_api_test.go index 3d07f4b..31a7869 100644 --- a/internal/command/publicapi/public_api_test.go +++ b/internal/command/publicapi/public_api_test.go @@ -124,7 +124,7 @@ func TestVerifyTypeScriptPackagePublicAPIRejectsUnsupportedSourceSyntaxExtension } func TestVerifyTypeScriptPackagePublicAPIRejectsExportsFromDifferentDeclaredSource(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.045293983925342815526031937349730030851244261191887869042173792304502947213298") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.004792030078784317588665526148252597261312386806264755602821268098843148288440") repoRoot := writeTypeScriptPackageFixture(t) packageRoot := filepath.Join(repoRoot, "packages", "alpha") if err := os.WriteFile(filepath.Join(packageRoot, "src", "other.ts"), []byte(`export const OTHER = 1;`), 0o600); err != nil { @@ -160,7 +160,7 @@ func TestVerifyTypeScriptPackagePublicAPIRejectsExportsFromDifferentDeclaredSour } func TestVerifyTypeScriptPackagePublicAPIRejectsExportStar(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.115680445627116622094001089796073200022717487136984640379014309614998296843761") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.060836943116930548342946569118531229790270856128186493541932275964533781302400") repoRoot := writeTypeScriptPackageFixture(t) sourcePath := filepath.Join(repoRoot, "packages", "alpha", "src", "index.ts") if err := os.WriteFile(sourcePath, []byte(`export * from "./internal";`), 0o600); err != nil { diff --git a/internal/command/readinesscloseout/readinesscloseout_test.go b/internal/command/readinesscloseout/readinesscloseout_test.go index baeb259..2049bb7 100644 --- a/internal/command/readinesscloseout/readinesscloseout_test.go +++ b/internal/command/readinesscloseout/readinesscloseout_test.go @@ -160,7 +160,7 @@ func TestBuildAddsMandatoryBoundaryNonClaims(t *testing.T) { } func TestBuildRejectsBroadNegationAndFrontierOverclaim(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.066064424951322431464180586591196427645864429207324866346989790552124866941997") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.007267731246771664992521702627347665138317306947213538578926686143640960390811") markdown := closedFrontierMarkdown( "Future authoring rows must not require separate owner proof before merge authority is established.", ) diff --git a/internal/command/receiptcurrentnessscope/receipt_currentness_scope_test.go b/internal/command/receiptcurrentnessscope/receipt_currentness_scope_test.go index 377a97c..8f8ec59 100644 --- a/internal/command/receiptcurrentnessscope/receipt_currentness_scope_test.go +++ b/internal/command/receiptcurrentnessscope/receipt_currentness_scope_test.go @@ -10,7 +10,7 @@ const digestA = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa const digestB = "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" func TestBuildAdmitsCurrentScopedReceiptAndRejectsStaleDigest(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.086966962808690097804613007060988808190045007706377086217228560067889861834248") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.022236487108238931965127542889597957904695079808409845972751946122899435814532") input := validReceiptCurrentnessScopeInput() record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/receiptproduceradmission/receiptproduceradmission_test.go b/internal/command/receiptproduceradmission/receiptproduceradmission_test.go index 44667bf..3c8d2bd 100644 --- a/internal/command/receiptproduceradmission/receiptproduceradmission_test.go +++ b/internal/command/receiptproduceradmission/receiptproduceradmission_test.go @@ -20,7 +20,7 @@ func TestBuildAcceptsMergeSatisfyingProducerReceipt(t *testing.T) { } func TestBuildRejectsAdvisoryProducerForMergeSatisfyingReceipt(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.016058221490407979556173242304898277468063189448256779462558039961547903134881") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.053308755633390310507952668654187701868931366930577643796604589279359393321955") input := validAdmission() producer := input["producers"].([]any)[0].(map[string]any) producer["admissionLevel"] = "advisory" diff --git a/internal/command/receipttrustclass/receipt_trust_class_test.go b/internal/command/receipttrustclass/receipt_trust_class_test.go index 86b63b5..bf33322 100644 --- a/internal/command/receipttrustclass/receipt_trust_class_test.go +++ b/internal/command/receipttrustclass/receipt_trust_class_test.go @@ -9,7 +9,7 @@ import ( ) func TestBuildAdmitsTrustedReceiptAndRejectsMissingProvenance(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.105335073905306761462059630478323546719954632613931373419064614644871938990973") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.014682110212614408688669170070931583072458934622263058290594375355449658410585") input := validReceiptTrustClassInput() record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/registryconsumer/registryconsumer_test.go b/internal/command/registryconsumer/registryconsumer_test.go index 25d28dd..cc69b30 100644 --- a/internal/command/registryconsumer/registryconsumer_test.go +++ b/internal/command/registryconsumer/registryconsumer_test.go @@ -12,7 +12,7 @@ import ( ) func TestRegistryConsumerAcceptsRegistryReleaseProof(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.078570628200989177884060697900986161263859706648466286899141113980107918828824") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.033718356853491749326048145854675346348034022276684730993169590765794228506662") record, exitCode, err := Build(validRegistryConsumerInput(t)) if err != nil { t.Fatalf("Build() error = %v", err) @@ -178,7 +178,7 @@ func TestRegistryExpectedReleaseAuthorityOutputUsesOwnerDigest(t *testing.T) { } func TestRegistryConsumerRejectsLegacyRootImportProof(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.039930770239161369599321884905894984519351217077585815477648288469771222069580") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.014303427678413549037642209464102836025458053002613511670388392919823523961849") input := validRegistryConsumerInput(t) proof := input["proof"].(map[string]any) proof["rootImportOutputSha256"] = sha256Hex() diff --git a/internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go b/internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go index 739055c..4547ea4 100644 --- a/internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go +++ b/internal/command/registryconsumerinputcompose/registry_consumer_input_compose_test.go @@ -23,7 +23,7 @@ var testRequiredPreconditionIDs = []string{ } func TestBuildComposesInputAcceptedByRegistryConsumer(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.013286022796687848436579723494873862332245131771566454685280914078856550358560") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.061624550493343445773564486888927869310676729537690954962361875225762539722512") input := validComposeInput(t) output, exitCode, err := Build(input) if err != nil { @@ -44,7 +44,7 @@ func TestBuildComposesInputAcceptedByRegistryConsumer(t *testing.T) { } func TestBuildBlocksUnavailableRequiredPreconditionsWithoutAcceptedInput(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.014640206555303265838366804019420074169369289515355674136620238515916737123108") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.074432687060893820701506224431203998502881951879706934067424442564366527870298") for _, preconditionID := range testRequiredPreconditionIDs { t.Run(preconditionID, func(t *testing.T) { input := validComposeInput(t) diff --git a/internal/command/releaseauthority/releaseauthority_test.go b/internal/command/releaseauthority/releaseauthority_test.go index 9067888..c0ccf81 100644 --- a/internal/command/releaseauthority/releaseauthority_test.go +++ b/internal/command/releaseauthority/releaseauthority_test.go @@ -97,7 +97,7 @@ func TestAdmitConsumerProjectionInputJSONRoundTripsThroughOwner(t *testing.T) { } func TestBuildRejectsPrivateSourceNPMProvenanceClaim(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.040417136410336177429000613084048687335976405838137408436642719228824047433829") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.105439509272579211435567089194759430024590675934332783957047489125678984707707") input := validRegistryReleaseInput("npm_provenance", "private") record, exitCode, err := Build(input) diff --git a/internal/command/renderedartifactfreshness/rendered_artifact_freshness_test.go b/internal/command/renderedartifactfreshness/rendered_artifact_freshness_test.go index 3f4de40..871b45f 100644 --- a/internal/command/renderedartifactfreshness/rendered_artifact_freshness_test.go +++ b/internal/command/renderedartifactfreshness/rendered_artifact_freshness_test.go @@ -11,7 +11,7 @@ const digestA = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa const digestB = "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" func TestBuildAdmitsFreshRenderedArtifactAndRejectsDigestDrift(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.094607798783002554961030625996597379975534494166124917670946505313810563955150") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.068616017028884434484440707243549301670259924558596491643742977971172281661381") input := validRenderedArtifactFreshnessInput() record, exitCode, err := Build(input) if err != nil { diff --git a/internal/command/repoprofileadmission/repo_profile_admission_test.go b/internal/command/repoprofileadmission/repo_profile_admission_test.go index 1cea6ef..f8392d8 100644 --- a/internal/command/repoprofileadmission/repo_profile_admission_test.go +++ b/internal/command/repoprofileadmission/repo_profile_admission_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAdmitsValidRepoProfileAndRejectsRootPackageMismatch(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.043504337756050392884249498699771665633165590624042065689844493508169838701202") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.042220601016185803105807340743508604421080385922243495458605947963572735177886") record, exitCode, err := Build(validRepoProfileInput()) if err != nil { t.Fatalf("Build() error = %v", err) diff --git a/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go b/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go index d986898..4afab68 100644 --- a/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go +++ b/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go @@ -142,7 +142,7 @@ func TestBuildRejectsCandidateMissingRequiredAuthoringFields(t *testing.T) { } func TestBuildRejectsCandidateSourceAdmissionFailure(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.085104032906462850481973845525599803854236430725230974151578780375965864330575") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.001844210971300640912460155205486111610590080014844769281330826828830013003484") input := validInput() candidate := firstUpdate(input)["candidateRequirement"].(map[string]any) candidate["proofBindingRefs"] = []any{} diff --git a/internal/command/requirementbinding/compact_contract_test.go b/internal/command/requirementbinding/compact_contract_test.go index 3126437..817804e 100644 --- a/internal/command/requirementbinding/compact_contract_test.go +++ b/internal/command/requirementbinding/compact_contract_test.go @@ -23,7 +23,7 @@ func TestBuildResolverPreservesCompactMutationResistanceState(t *testing.T) { } func TestBuildResolverEmitsNamedLookupFacts(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.079097784231569243123760864431497247802974951490040482853947549382894609207552") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.111196201832829118735064910982698751650497890612272762431668463661114699885279") output, exitCode, err := BuildResolver(validCompactContract(), ResolverOptions{LocalEnvironmentClasses: []string{"local-go"}}) if err != nil { t.Fatalf("BuildResolver() error = %v", err) @@ -157,7 +157,7 @@ func TestBuildResolverRejectsCompactShellControlCommandText(t *testing.T) { } func TestBuildResolverRejectsUnscopedCompactIdentity(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.069762921155265534833897611463817586909954409189497461749409846690639299144534") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.076041686007458666270617161270722640013102244122307015251002781061987907765701") type selectorCase struct { name string mutate func(map[string]any) diff --git a/internal/command/requirementbinding/projections_test.go b/internal/command/requirementbinding/projections_test.go index d73d628..93ec93d 100644 --- a/internal/command/requirementbinding/projections_test.go +++ b/internal/command/requirementbinding/projections_test.go @@ -72,7 +72,7 @@ func readOwnerJSON(t *testing.T, path string) any { } func TestBuildReportFailsUnknownRequirementBinding(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.000376648692213538787264617164821843651034132903736657977168045531003514534050") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.104544593850875418975815490237416802070339444461856063303404358287592673340470") input := validRequirementBindingInput() input["bindings"].([]any)[0].(map[string]any)["requirementId"] = "REQ-PROOFKIT-MISSING" @@ -155,7 +155,7 @@ func TestBuildReportRejectsEmptyWitnessCommandEnvironmentClasses(t *testing.T) { } func TestBuildEvidenceGraphBuildsGraphAndRejectsFailedReport(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.107818454717962585697455608706257308689211271216669906752699392232450827006825") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.103879939339912119536842874285920100618442305235744692163114660613220220633744") graph, exitCode, err := BuildEvidenceGraph(validRequirementBindingInput()) if err != nil { t.Fatalf("BuildEvidenceGraph() error=%v", err) @@ -177,7 +177,7 @@ func TestBuildEvidenceGraphBuildsGraphAndRejectsFailedReport(t *testing.T) { } func TestBuildProofSliceSelectsRequirementsAndRejectsFailedReport(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.057814514043290257871128107874680755541616706843803515635545866062242868890363") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.108834399078649179642042963327326693613324047941888185048583794094499189388965") input := validRequirementBindingInput() input["selection"] = map[string]any{ "changedPaths": []any{}, diff --git a/internal/command/requirementbrowser/server_test.go b/internal/command/requirementbrowser/server_test.go index 2306f7e..551780f 100644 --- a/internal/command/requirementbrowser/server_test.go +++ b/internal/command/requirementbrowser/server_test.go @@ -603,7 +603,7 @@ func (writer readyWriter) Write(bytes []byte) (int, error) { } func TestStartServerFailsClosedForNonLoopbackHosts(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.094034782477634282784990120509101846539349621662349085355544682336212659794052") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.023640023374539652452651315983351191617142195079844369423488208926061999224113") for _, host := range []string{"0.0.0.0", "localhost"} { t.Run(host, func(t *testing.T) { _, err := StartServer(sourceInput(t), Options{ diff --git a/internal/command/requirementcontext/requirementcontext_test.go b/internal/command/requirementcontext/requirementcontext_test.go index 199b255..0b8335e 100644 --- a/internal/command/requirementcontext/requirementcontext_test.go +++ b/internal/command/requirementcontext/requirementcontext_test.go @@ -16,7 +16,7 @@ import ( ) func TestComposeAndSliceRoundTrip(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.039174333591545173112362713528481218186528989446819372039588644025024411317142") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.073105413718212685720467657880735003544056444223060540529926611328658669412386") root := fixtureRepository(t) contextValue, err := Compose(root, fixtureCatalog()) if err != nil { @@ -315,7 +315,7 @@ func resignSnapshot(t *testing.T, value map[string]any) { } func TestSliceRejectsTamperedSnapshotAndUnknownNode(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.085802294599012556735496328767610257842830634122573088743085183818381349380300") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.070746173522414644645062396948230600718686825876357257505266577982734819530024") root := fixtureRepository(t) contextValue, err := Compose(root, fixtureCatalog()) if err != nil { diff --git a/internal/command/requirementcoverageinput/requirementcoverageinput_test.go b/internal/command/requirementcoverageinput/requirementcoverageinput_test.go index 21ce13d..856fc26 100644 --- a/internal/command/requirementcoverageinput/requirementcoverageinput_test.go +++ b/internal/command/requirementcoverageinput/requirementcoverageinput_test.go @@ -15,7 +15,7 @@ import ( ) func TestBuildComposesInputPreservesDeclaredUniverseAndAllowsDownstreamFailures(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.113418411841251725235421480333392205904743124021673839349632925543087313742674") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.077615860756811261841718576176256104140210383495891744406155830486424738479038") output, exitCode, err := Build(validComposeInput(t, baseInventoryEntries())) if err != nil { t.Fatalf("Build() error = %v", err) @@ -45,7 +45,7 @@ func TestBuildComposesInputPreservesDeclaredUniverseAndAllowsDownstreamFailures( } func TestBuildComposesDirectRequirementProofBindingAndInventory(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.080188824835127142028423449313693131405321184428152180020843046646894549524418") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.035702766579458185352787503340754906774314209827550424808834264957824734855602") input := validComposeInput(t, baseInventoryEntries()).(map[string]any) normalized := input["normalizedTestEvidenceInventory"].(map[string]any) delete(input, "compactProofContract") @@ -236,7 +236,7 @@ func TestBuildRejectsDirectInventoryWithoutNormalizedEnvelope(t *testing.T) { } func TestBuildRejectsFabricatedDirectEnvelopeWithSourceMetadata(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.080770507620948292540117235609331664327832420958915681711979800001742844264540") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.082003289099884097239481684858563895288262439491982038468563724937356701005316") input := validComposeInput(t, baseInventoryEntries()).(map[string]any) normalized := input["normalizedTestEvidenceInventory"].(map[string]any) normalized["sourceCount"] = json.Number("1") diff --git a/internal/command/requirementcoverageview/requirementcoverageview_test.go b/internal/command/requirementcoverageview/requirementcoverageview_test.go index a334893..4df0be5 100644 --- a/internal/command/requirementcoverageview/requirementcoverageview_test.go +++ b/internal/command/requirementcoverageview/requirementcoverageview_test.go @@ -763,7 +763,7 @@ func TestBuildJSONAgentEnvelopeUsesSharedEnvelopeKernel(t *testing.T) { } func TestBuildJSONRejectsRouteOnlyCoverageForBlockingRequirement(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.031691819956938118822991045596460919977101593213780644803440154333791119868791") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.031718276374641887712984669039991110874842587862832715070881908134515964679634") input := validCoverageInput(t) entry := inventoryEntry(input) entry["evidenceClass"] = "routing_smoke_nonclaim" diff --git a/internal/command/requirementdiff/requirementdiff_test.go b/internal/command/requirementdiff/requirementdiff_test.go index 8d64dc5..af33b1c 100644 --- a/internal/command/requirementdiff/requirementdiff_test.go +++ b/internal/command/requirementdiff/requirementdiff_test.go @@ -188,7 +188,7 @@ func TestOwnerFilterPreservesStableIdentityAcrossOwnershipChange(t *testing.T) { } func TestBuildCoversCompleteRequirementChangeAlgebra(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.091504423064274626695623049569025044424830049352504253663226902245461570004740") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.078549664710316599082962403097259031550446698569117313237162339371160705954076") baseline := contextFixture(t, "The shared requirement remains stable.") current := contextFixture(t, "The shared requirement changes its invariant.") baseRequirements := baseline["projections"].(map[string]any)["requirementSources"].([]any)[0].(map[string]any)["requirements"].([]any) diff --git a/internal/command/requirementgraph/requirementgraph_test.go b/internal/command/requirementgraph/requirementgraph_test.go index b8e4e2a..5ed4098 100644 --- a/internal/command/requirementgraph/requirementgraph_test.go +++ b/internal/command/requirementgraph/requirementgraph_test.go @@ -18,7 +18,7 @@ import ( ) func TestBuildKeepsTraceabilityEvidencePlanesDistinct(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.040236281331857613367866543934119806645341297834533138126303789519826727502569") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.022500508667327863554430021675314177435444085857004785904226266316397359221432") contextValue := graphContextFixture(t) code := "package handler\n\nfunc Handle() { return }\n" codeDigest := digest.SHA256TextRef(code) @@ -70,7 +70,7 @@ func TestBuildKeepsTraceabilityEvidencePlanesDistinct(t *testing.T) { } func TestAdmitOutputRejectsDanglingAndIncoherentCodeParents(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.093487220897786293104017571940100080664662437591187487146507089441802769113895") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.000951313542647162551115118684533813603727933293140621118035893199512478060879") output, err := Build(graphPermutationInput(t)) if err != nil { t.Fatal(err) diff --git a/internal/command/requirementimpactinput/requirementimpactinput_test.go b/internal/command/requirementimpactinput/requirementimpactinput_test.go index 279bc90..a391aad 100644 --- a/internal/command/requirementimpactinput/requirementimpactinput_test.go +++ b/internal/command/requirementimpactinput/requirementimpactinput_test.go @@ -14,7 +14,7 @@ import ( ) func TestBuildComposesInputAndRoutesChangedBlockingRequirement(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.012669526642153953357021471326490174173967306322522257377004543076904982321781") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.023234960845452171243291914185559159842773576460263008556765730794053386182086") input := validComposeInput(t) currentSource := input["currentRequirementSources"].([]any)[0].(map[string]any) currentSource["requirements"].([]any)[0].(map[string]any)["invariant"] = "Requirement impact input composition must route changed blocking requirement records to caller-owned proof obligations." diff --git a/internal/command/requirementproofsourceset/requirementproofsourceset_test.go b/internal/command/requirementproofsourceset/requirementproofsourceset_test.go index 445666e..3562f92 100644 --- a/internal/command/requirementproofsourceset/requirementproofsourceset_test.go +++ b/internal/command/requirementproofsourceset/requirementproofsourceset_test.go @@ -54,7 +54,7 @@ func TestBuildCombinesCanonicalSourceAndRejectsSHADrift(t *testing.T) { } func TestBuildSelectsSourceSetRowsAndEmitsResolverInput(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.103606270231233266473620303120670087653439368563763197580781857201403855835422") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.106477495378157513392694067858829301266454618332683944940209642307095597996567") input := validFragmentSourceSetInput(t) input["projection"] = map[string]any{ "kind": "resolver_input", diff --git a/internal/command/requirementproofview/requirementproofview_test.go b/internal/command/requirementproofview/requirementproofview_test.go index 00c54f4..69b5962 100644 --- a/internal/command/requirementproofview/requirementproofview_test.go +++ b/internal/command/requirementproofview/requirementproofview_test.go @@ -121,7 +121,7 @@ func TestBuildHTMLEscapesCallerControlledCompactFields(t *testing.T) { } func TestBuildMarkdownEscapesCallerControlledCompactFields(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.002958980837736388688570945636786727657668783716021703161976564587172015017594") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.004013119203828558628733921363223531050236398733138199684571194055855497152335") output, exitCode, err := BuildMarkdown(maliciousCompactContract(), Options{LocalEnvironmentClasses: []string{"local-go"}}) if err != nil { t.Fatalf("BuildMarkdown() error = %v", err) diff --git a/internal/command/requirementsourceadmission/requirementsourceadmission_test.go b/internal/command/requirementsourceadmission/requirementsourceadmission_test.go index d8ef5da..17e45f8 100644 --- a/internal/command/requirementsourceadmission/requirementsourceadmission_test.go +++ b/internal/command/requirementsourceadmission/requirementsourceadmission_test.go @@ -50,7 +50,7 @@ func TestEvaluateAcceptsActiveBlockingRequirementWithProofRoute(t *testing.T) { } func TestEvaluateRejectsBlockingRequirementWithoutProofRoute(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.100459756653360126553817815808542023226844129025849093125858776155160578989149") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.049105822013487377720513422847314427679030355003527858634951221583838655591366") input := validSource() requirement := input["requirements"].([]any)[0].(map[string]any) requirement["proofBindingRefs"] = []any{} diff --git a/internal/command/requirementsourcetransition/requirementsourcetransition_test.go b/internal/command/requirementsourcetransition/requirementsourcetransition_test.go index 1c96d1c..a33885c 100644 --- a/internal/command/requirementsourcetransition/requirementsourcetransition_test.go +++ b/internal/command/requirementsourcetransition/requirementsourcetransition_test.go @@ -29,7 +29,7 @@ func TestBuildRejectsLifecycleTransitionWithoutNewEvidence(t *testing.T) { } func TestBuildRejectsRequirementSourceTransitionContractViolations(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.000338353607129616419889377401978351258590626306307683363645508835961369207418") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.064796430251047638129056627618050421615250184739595429149390171993569993200591") cases := []struct { name string want string diff --git a/internal/command/requirementsourceview/requirementsourceview_test.go b/internal/command/requirementsourceview/requirementsourceview_test.go index fb80a3c..24d5df9 100644 --- a/internal/command/requirementsourceview/requirementsourceview_test.go +++ b/internal/command/requirementsourceview/requirementsourceview_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildMarkdownEscapesCallerControlledText(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.079155125739683862685982535478403487660808558239998414186036088598924884165357") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.005704906455553467725586561316636717684357758836129946697278325918285718879720") input := validRequirementSource() input["specPackagePath"] = "docs/specs/proofkit-``" input["overviewPath"] = "docs/specs/proofkit-``/overview.md" diff --git a/internal/command/requirementspectree/requirementspectree_test.go b/internal/command/requirementspectree/requirementspectree_test.go index d6882bc..b720f88 100644 --- a/internal/command/requirementspectree/requirementspectree_test.go +++ b/internal/command/requirementspectree/requirementspectree_test.go @@ -31,7 +31,7 @@ func TestBuildAdmitsSpecTree(t *testing.T) { } func TestBuildRejectsDAGAndStaleDigest(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.004274152517249135682493047858886787225089847311761896341362471207420838269909") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.101919815328177545466245213748768811374784723554857148933696569778978221061398") input := validTreeInput() edges := input["edges"].([]any) input["edges"] = append(edges, map[string]any{"parentNodeId": "meta", "childNodeId": "submodule"}) @@ -644,7 +644,7 @@ func TestBuildViewFailsClosedForInvalidTree(t *testing.T) { } func TestBuildViewMarkdownAndHTMLAreDeterministicAndEscaped(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.015614885815453293453950599593189230449223881412117163077427726161271792555871") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.003233567092711501460950786753414939758288157341158912542081410563017083291305") input := validTreeInput() nodeMap(input, "module")["label"] = "Module " overlayMap(input, "overlay.rendered.module")["label"] = "Rendered " diff --git a/internal/command/scaffoldprofileplan/scaffoldprofileplan_test.go b/internal/command/scaffoldprofileplan/scaffoldprofileplan_test.go index 1b31a61..d1d7726 100644 --- a/internal/command/scaffoldprofileplan/scaffoldprofileplan_test.go +++ b/internal/command/scaffoldprofileplan/scaffoldprofileplan_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAcceptsCommandMatcherHints(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.013321323303874251949085998799870857902727560387311658432878441406042996018615") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.066388111489441355762289763185156364551197665370738732325425583942920333391935") result, err := BuildResult(validScaffoldInput()) if err != nil { t.Fatalf("BuildResult() error = %v", err) diff --git a/internal/command/secretscan/secretscan_test.go b/internal/command/secretscan/secretscan_test.go index 5c2a0da..53c20b0 100644 --- a/internal/command/secretscan/secretscan_test.go +++ b/internal/command/secretscan/secretscan_test.go @@ -11,7 +11,7 @@ import ( ) func TestBuildFindsSecretLikeTextWithoutLeakingValue(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.044627298880588177227751226258246473779575152087534439636266250802154512519866") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.039817040606337266696292466844606855102291955096447906749917550657133356377901") const sentinel = "abc123456789" record, exitCode, err := Build(validInput(map[string][]byte{ "docs/ok.md": []byte("plain text\n"), diff --git a/internal/command/selectivegateevidence/selectivegateevidence_test.go b/internal/command/selectivegateevidence/selectivegateevidence_test.go index d14af0d..8a90258 100644 --- a/internal/command/selectivegateevidence/selectivegateevidence_test.go +++ b/internal/command/selectivegateevidence/selectivegateevidence_test.go @@ -33,7 +33,7 @@ func TestBuildRejectsNonBooleanPublicAPIPlanFlag(t *testing.T) { } func TestProjectObligationDecisionBuildsInputAndRejectsUnroutedCommand(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.070349223224769520259517966591709325720249553798073685946672869745658628533890") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.018388928679990711151132651696474760405977786383365901536442241248004003958482") projected, err := ProjectObligationDecision(validProjectionInput()) if err != nil { t.Fatalf("ProjectObligationDecision() error=%v", err) @@ -247,7 +247,7 @@ func TestBuildRejectsFailedProducerAdmissionReport(t *testing.T) { } func TestBuildRejectsMergeSatisfyingEvidenceWithoutProducerAdmission(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.084832385498800555802011138719665130606367582119329227221618245501473605705603") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.112550534058264454186632868263981916290936959138554398802013554630682769870797") input := validEvidenceInput() delete(input, "producerAdmission") @@ -473,7 +473,7 @@ func TestBuildAddsBoundaryNonClaims(t *testing.T) { } func TestBuildReportsMergeEvidenceWithoutApprovingMerge(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.062399083186293634119152170193059006692728517643142400331042963092772385607972") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.025252982453387603468032289952983834092931240062565910622005306249337590832380") result, err := Build(validEvidenceInput()) if err != nil { t.Fatalf("Build() error = %v", err) diff --git a/internal/command/selectivegateplan/selectivegateplan_test.go b/internal/command/selectivegateplan/selectivegateplan_test.go index 87ebbe9..d7c8fc3 100644 --- a/internal/command/selectivegateplan/selectivegateplan_test.go +++ b/internal/command/selectivegateplan/selectivegateplan_test.go @@ -182,7 +182,7 @@ func TestBuildRejectsDisplayOnlyCommandShellControlTokens(t *testing.T) { } func TestBuildFailsClosedForUncoveredUnknownEdge(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.114223091652506300873298948515061679816313892419980290837569919004743321148207") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.109203724066169697433019740792211532942871969238130040996597287475731754319862") input := validPlanInput() input["unknownEdges"] = []any{unknownEdgeInput("edge.dynamic", "dynamic_or_unknown")} diff --git a/internal/command/specoverviewclaims/specoverviewclaims_test.go b/internal/command/specoverviewclaims/specoverviewclaims_test.go index b4c3997..e8fbe2a 100644 --- a/internal/command/specoverviewclaims/specoverviewclaims_test.go +++ b/internal/command/specoverviewclaims/specoverviewclaims_test.go @@ -103,7 +103,7 @@ func TestBuildRejectsPathDrift(t *testing.T) { } func TestBuildRejectsInvalidOverviewClaimBoundaryFacts(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.025280307180968926567211146624399965026613890974304865664816068083127351219786") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.112682775494295917012327305050927168210732804488321621459110580984044195720333") cases := []struct { name string err string @@ -166,7 +166,7 @@ func TestBuildRejectsInvalidOverviewClaimBoundaryFacts(t *testing.T) { } func TestBuildRejectsNonDurableRequirementCitationsForEveryNonDurableKind(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.088402325428378402231209410729523974270570645927538147014098755159352586154144") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.009542672296287244834772220636187272276005726456609804775418682151340119727196") for _, claimKind := range []string{"example_or_rationale", "quoted_or_code", "section_heading"} { t.Run(claimKind, func(t *testing.T) { input := validBoundary() diff --git a/internal/command/specproofbundleadmission/specproofbundleadmission_test.go b/internal/command/specproofbundleadmission/specproofbundleadmission_test.go index a176662..0054e10 100644 --- a/internal/command/specproofbundleadmission/specproofbundleadmission_test.go +++ b/internal/command/specproofbundleadmission/specproofbundleadmission_test.go @@ -213,7 +213,7 @@ func childReportValueWithReceipts(state string, exitCode string, failures []any, } func TestBuildRejectsForgedReceiptAdmissionChild(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.032888675717298205828739761733737136832901453189109010451175224295171303909603") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.114880187029418565756618053091883445515213775524954029606797760062822359451460") input := validBundleInput(t) receiptAdmission := input["receiptAdmission"].(map[string]any) receiptAdmission["receipts"] = []any{ diff --git a/internal/command/stackpreset/preset_ids_generated.go b/internal/command/stackpreset/preset_ids_generated.go index 95ac412..950dccd 100644 --- a/internal/command/stackpreset/preset_ids_generated.go +++ b/internal/command/stackpreset/preset_ids_generated.go @@ -1,6 +1,6 @@ // Code generated by internal/tools/commandcontractgen; DO NOT EDIT. package stackpreset -const presetContractSourceSHA256 = "3d2fab7d4b8da24fb1c105ce8814b531926c50dee9a594eacad953f8af897132" +const presetContractSourceSHA256 = "f73bd8e28f0855963568866141d1e67283b2b8cd2e931658e73e704db2cf19c0" var presetIDs = []string{"agentic_runtime_repo", "generated_docs_contract_repo", "python_service", "python_typescript_service", "typescript_monorepo", "typescript_workspace"} diff --git a/internal/command/stackpreset/stackpreset_test.go b/internal/command/stackpreset/stackpreset_test.go index df6a3cb..a611fcc 100644 --- a/internal/command/stackpreset/stackpreset_test.go +++ b/internal/command/stackpreset/stackpreset_test.go @@ -9,7 +9,7 @@ import ( ) func TestPresetInventoryIsCompleteDeterministicAndDefensivelyCopied(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.021096613433872109936380474204032410537184334172715554175894648026752262369407") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.076944518366669573920389884498853536696905439692658896257172669916814288705961") if len(presetIDs) != len(presets) { t.Fatalf("presetIDs=%d presets=%d, want one id per preset", len(presetIDs), len(presets)) } @@ -84,7 +84,7 @@ func TestPresetInventoryIsCompleteDeterministicAndDefensivelyCopied(t *testing.T } func TestUnknownPresetIsRejected(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.086360966252379017579457694274778024752791571205403533089233505435328802293079") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.070141068460637498789620042072455227278667326750769337227893395903279066165874") if IsPresetID("unknown") { t.Fatal("IsPresetID accepted unknown preset") } diff --git a/internal/command/testevidenceinventory/testevidenceinventory_test.go b/internal/command/testevidenceinventory/testevidenceinventory_test.go index 536a2bb..342f0a8 100644 --- a/internal/command/testevidenceinventory/testevidenceinventory_test.go +++ b/internal/command/testevidenceinventory/testevidenceinventory_test.go @@ -57,7 +57,7 @@ func TestBuildRejectsUnanchoredProofRouteCandidate(t *testing.T) { } func TestBuildDiscoveryDraftEmitsCandidateOnlyInventory(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.091914923857893189235410404218309297051671488437122754346751725931508712189132") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.096318819151227230864150155118957615254328016145569586137207173056023856414640") record, exitCode, err := BuildDiscoveryDraft(validDiscoveryDraft()) if err != nil { t.Fatalf("BuildDiscoveryDraft() error = %v", err) @@ -283,7 +283,7 @@ func TestBuildDiscoveryDraftRejectsUnsafeAndContradictoryFacts(t *testing.T) { } func TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.107074977993094420943019248218359465183764901197584019504412197592065422528956") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.040537407725353522942442370438759713872175874575309347907987603239091331652992") input := validInventory(t) entries := input.(map[string]any)["entries"].([]any) first := cloneMap(entries[0].(map[string]any)) diff --git a/internal/command/textpolicy/textpolicy_test.go b/internal/command/textpolicy/textpolicy_test.go index 09a4a5b..c3bb399 100644 --- a/internal/command/textpolicy/textpolicy_test.go +++ b/internal/command/textpolicy/textpolicy_test.go @@ -10,7 +10,7 @@ import ( ) func TestEvaluatePreservesUTF8ASCIIWhitespaceAndBinaryFalsifiers(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.086855865554772597777106065642397508247799514617063694794143213323987594296092") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.031094312573272652863042278542901905945454475685240528067456704904488178168569") result, err := Evaluate(validInput(map[string][]byte{ "binary.ZIP": []byte{0xff, 0xfe}, "docs/empty.md": {}, diff --git a/internal/command/witnessplan/witnessplan_test.go b/internal/command/witnessplan/witnessplan_test.go index 0831c4b..82e7717 100644 --- a/internal/command/witnessplan/witnessplan_test.go +++ b/internal/command/witnessplan/witnessplan_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAdmitsSafeCommandAndRejectsShellCommand(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.061163857848999249192334582247265083240613726562619453864656308921156645063184") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.088457406797103795279543653822179063516575466007372745444447470717382611396483") plan, err := Build(validWitnessPlanInput()) if err != nil { t.Fatalf("Build() error = %v", err) @@ -44,7 +44,7 @@ func TestBuildAdmitsOptionalDirectInputSchemaVersionOne(t *testing.T) { } func TestBuildProjectsRequirementBindingsToWitnessPlan(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.033949758224172503973560419980040060865660836625689337975156681518110461106337") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.097064780769502204057926611195131645878255373766343559374932016116089291062973") input := map[string]any{ "schemaVersion": json.Number("1"), "projection": "requirement-bindings", diff --git a/internal/command/witnessschedulerplan/witnessschedulerplan_test.go b/internal/command/witnessschedulerplan/witnessschedulerplan_test.go index 090325c..cc7d9ae 100644 --- a/internal/command/witnessschedulerplan/witnessschedulerplan_test.go +++ b/internal/command/witnessschedulerplan/witnessschedulerplan_test.go @@ -22,7 +22,7 @@ func TestBuildRejectsNetworkMetadataContradictions(t *testing.T) { } func TestBuildRejectsUnsafeParallelWriteCollision(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.075339797262506844817194666452374470457527789495842132120572392330493764265328") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.104676796118456324923039124511231127526363393383801390782562378512319721449269") input := validSchedulerPlanInput() command(input)["id"] = "proofkit.left" schedulerPolicy(input)["commandId"] = "proofkit.left" diff --git a/internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go b/internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go index 6310603..4bb75a7 100644 --- a/internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go +++ b/internal/command/workspacemanifestfacts/workspace_manifest_facts_test.go @@ -13,7 +13,7 @@ import ( ) func TestBuildProjectsManifestFactsAndPlanningInputs(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.045131377523903059328620085892776244655592414046718681948892749746792823542716") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.072879230585900406731220760823785361194395031187183819675511483181285804453184") output, exitCode, err := Build(validInput(t)) if err != nil { t.Fatalf("Build() error = %v", err) @@ -79,7 +79,7 @@ func TestBuildOutputsAreAdmittedByWorkspacePlanningCommands(t *testing.T) { } func TestBuildRejectsUnsafeManifestPathAndDuplicatePackageIdentity(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.042789860620987407196887112869252006583828805316041850684999615267856780669343") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.076788067606744061308755388073116836267887021333035147502494382368969043524666") t.Run("unsafe manifest path", func(t *testing.T) { input := validInput(t).(map[string]any) input["packages"].([]any)[0].(map[string]any)["manifestPath"] = "../package.json" diff --git a/internal/command/workspaceplanning/workspaceplanning_test.go b/internal/command/workspaceplanning/workspaceplanning_test.go index 7ca9127..4194a74 100644 --- a/internal/command/workspaceplanning/workspaceplanning_test.go +++ b/internal/command/workspaceplanning/workspaceplanning_test.go @@ -8,7 +8,7 @@ import ( ) func TestChangedPackagePlanAdmitsPackagesRootAndSchema(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.012483302559028244967785281995194599578244586042338061566707694834914436961975") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.000915784104466894868430403781008280151510309816212488025513280699400857148130") input := validChangedPackagePlanInput() plan, err := BuildChangedPackagePlan(input) @@ -83,7 +83,7 @@ func TestShardPartitionRejectsUnknownNestedFields(t *testing.T) { } func TestShardPartitionAdmitsCoveredRootsAndRejectsMissingDependency(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.067233922180114604656007812711586055552725961620481057364664589070016617853243") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.011525493414109086910947302412685558638860644500578844199382862426651697571033") partition, exitCode, err := BuildShardPartition(validShardPartitionInput()) if err != nil { t.Fatalf("BuildShardPartition() error=%v", err) diff --git a/internal/command/workspaceregistry/workspaceregistry_test.go b/internal/command/workspaceregistry/workspaceregistry_test.go index 1bc5a68..92eb449 100644 --- a/internal/command/workspaceregistry/workspaceregistry_test.go +++ b/internal/command/workspaceregistry/workspaceregistry_test.go @@ -8,7 +8,7 @@ import ( ) func TestBuildAdmitsWorkspaceRegistryAndRejectsMissingScriptTarget(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.038162518284450631435313163265663645080169886890779400608856160417454162064127") + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.050787429179113333482394128925978139548470597502501014010332804049210083823789") record, exitCode, err := Build(validWorkspaceRegistryInput()) if err != nil { t.Fatalf("Build() error=%v", err) diff --git a/internal/testsupport/commandcoverage/semantic_route.go b/internal/testsupport/commandcoverage/semantic_route.go index 08cf3ef..01d047d 100644 --- a/internal/testsupport/commandcoverage/semantic_route.go +++ b/internal/testsupport/commandcoverage/semantic_route.go @@ -2,21 +2,44 @@ package commandcoverage import ( "strings" - "testing" ) const semanticRoutePrefix = "proofkit.command_coverage.source_oracle.v1." +const ExecutionAttributeKey = "proofkit.command-oracle" -// SemanticRoute validates a legacy source marker for a proof-route candidate. -// It does not execute a falsification event or produce semantic evidence. -func SemanticRoute(t testing.TB, marker string) { +type testContext interface { + Attr(string, string) + Cleanup(func()) + Failed() bool + Fatalf(string, ...any) + Helper() + Skipped() bool +} + +// SemanticRoute binds one source-owned route marker to a cooperative runtime event. +// The event proves successful completion of the selected test, not execution of +// any particular assertion branch. +func SemanticRoute(t testContext, marker string) { t.Helper() - if !strings.HasPrefix(marker, semanticRoutePrefix) || len(marker) != len(semanticRoutePrefix)+78 { + if !ValidSourceMarker(marker) { t.Fatalf("invalid command coverage semantic route marker %q", marker) + return + } + t.Cleanup(func() { + if !t.Failed() && !t.Skipped() { + t.Attr(ExecutionAttributeKey, marker) + } + }) +} + +func ValidSourceMarker(marker string) bool { + if !strings.HasPrefix(marker, semanticRoutePrefix) || len(marker) != len(semanticRoutePrefix)+78 { + return false } for _, character := range strings.TrimPrefix(marker, semanticRoutePrefix) { if character < '0' || character > '9' { - t.Fatalf("invalid command coverage semantic route marker %q", marker) + return false } } + return true } diff --git a/internal/testsupport/commandcoverage/semantic_route_test.go b/internal/testsupport/commandcoverage/semantic_route_test.go new file mode 100644 index 0000000..43db0ce --- /dev/null +++ b/internal/testsupport/commandcoverage/semantic_route_test.go @@ -0,0 +1,69 @@ +package commandcoverage + +import "testing" + +const validMarker = "proofkit.command_coverage.source_oracle.v1.000000000000000000000000000000000000000000000000000000000000000000000000000001" + +func TestSemanticRouteEmitsAttributeOnlyAfterSuccessfulCleanup(t *testing.T) { + context := &fakeTestContext{} + SemanticRoute(context, validMarker) + if len(context.attributes) != 0 || len(context.cleanups) != 1 { + t.Fatalf("registration state = %#v", context) + } + context.cleanups[0]() + if len(context.attributes) != 1 || context.attributes[0] != [2]string{ExecutionAttributeKey, validMarker} { + t.Fatalf("attributes = %#v", context.attributes) + } +} + +func TestSemanticRouteSuppressesFailedAndSkippedEvidence(t *testing.T) { + for _, item := range []struct { + name string + failed bool + skipped bool + }{ + {name: "failed", failed: true}, + {name: "skipped", skipped: true}, + } { + t.Run(item.name, func(t *testing.T) { + context := &fakeTestContext{failed: item.failed, skipped: item.skipped} + SemanticRoute(context, validMarker) + context.cleanups[0]() + if len(context.attributes) != 0 { + t.Fatalf("attributes = %#v", context.attributes) + } + }) + } +} + +func TestSemanticRouteRejectsMalformedMarker(t *testing.T) { + context := &fakeTestContext{} + SemanticRoute(context, "invalid") + if context.fatalCount != 1 || len(context.cleanups) != 0 { + t.Fatalf("malformed marker state = %#v", context) + } +} + +type fakeTestContext struct { + attributes [][2]string + cleanups []func() + failed bool + fatalCount int + skipped bool +} + +func (context *fakeTestContext) Attr(key, value string) { + context.attributes = append(context.attributes, [2]string{key, value}) +} + +func (context *fakeTestContext) Cleanup(cleanup func()) { + context.cleanups = append(context.cleanups, cleanup) +} + +func (context *fakeTestContext) Failed() bool { return context.failed } + +func (context *fakeTestContext) Fatalf(string, ...any) { context.fatalCount++ } + +func (context *fakeTestContext) Helper() {} + +func (context *fakeTestContext) Skipped() bool { return context.skipped } diff --git a/internal/tools/artifactfile/file.go b/internal/tools/artifactfile/file.go new file mode 100644 index 0000000..a85887c --- /dev/null +++ b/internal/tools/artifactfile/file.go @@ -0,0 +1,215 @@ +package artifactfile + +import ( + "crypto/rand" + "encoding/hex" + "errors" + "fmt" + "io" + "io/fs" + "math" + "os" + "path/filepath" + "strings" +) + +const temporaryCollisionLimit = 16 + +func WriteAtomic(rootPath, relativePath string, content []byte, mode fs.FileMode) error { + path, err := localPath(relativePath) + if err != nil { + return err + } + if mode&^0o777 != 0 { + return fmt.Errorf("artifact file mode is invalid") + } + root, err := os.OpenRoot(rootPath) + if err != nil { + return fmt.Errorf("open artifact root failed") + } + defer root.Close() + if _, err := admitDirectories(root, filepath.Dir(path), true); err != nil { + return err + } + if err := admitDestination(root, path); err != nil { + return err + } + for attempt := 0; attempt < temporaryCollisionLimit; attempt++ { + var nonce [16]byte + if _, err := rand.Read(nonce[:]); err != nil { + return fmt.Errorf("allocate artifact temporary identity failed") + } + temporaryPath := path + "." + hex.EncodeToString(nonce[:]) + ".tmp" + file, err := root.OpenFile(temporaryPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) + if errors.Is(err, fs.ErrExist) { + continue + } + if err != nil { + return fmt.Errorf("create artifact temporary file failed") + } + removeTemporary := true + defer func() { + if removeTemporary { + _ = root.Remove(temporaryPath) + } + }() + if _, err := file.Write(content); err != nil { + file.Close() + return fmt.Errorf("write artifact temporary file failed") + } + if err := file.Sync(); err != nil { + file.Close() + return fmt.Errorf("sync artifact temporary file failed") + } + if err := file.Chmod(mode); err != nil { + file.Close() + return fmt.Errorf("set artifact file mode failed") + } + if err := file.Close(); err != nil { + return fmt.Errorf("close artifact temporary file failed") + } + if _, err := admitDirectories(root, filepath.Dir(path), false); err != nil { + return err + } + if err := admitDestination(root, path); err != nil { + return err + } + if err := root.Rename(temporaryPath, path); err != nil { + return fmt.Errorf("publish artifact file failed") + } + removeTemporary = false + return nil + } + return fmt.Errorf("artifact temporary collision budget exhausted") +} + +func ReadBounded(rootPath, relativePath string, maxBytes int64) ([]byte, error) { + if maxBytes <= 0 || maxBytes == math.MaxInt64 { + return nil, fmt.Errorf("artifact read limit is invalid") + } + path, err := localPath(relativePath) + if err != nil { + return nil, err + } + root, err := os.OpenRoot(rootPath) + if err != nil { + return nil, fmt.Errorf("open artifact root failed") + } + defer root.Close() + if _, err := admitDirectories(root, filepath.Dir(path), false); err != nil { + return nil, err + } + before, err := root.Lstat(path) + if err != nil { + return nil, fmt.Errorf("inspect artifact file failed") + } + if before.Mode()&os.ModeSymlink != 0 || !before.Mode().IsRegular() { + return nil, fmt.Errorf("artifact source must be a regular non-symlink file") + } + if before.Size() > maxBytes { + return nil, fmt.Errorf("artifact file exceeds resource limit") + } + file, err := root.Open(path) + if err != nil { + return nil, fmt.Errorf("open artifact file failed") + } + defer file.Close() + opened, err := file.Stat() + if err != nil || !os.SameFile(before, opened) || !opened.Mode().IsRegular() { + return nil, fmt.Errorf("artifact source changed during admission") + } + content, err := io.ReadAll(io.LimitReader(file, maxBytes+1)) + if err != nil { + return nil, fmt.Errorf("read artifact file failed") + } + if int64(len(content)) > maxBytes { + return nil, fmt.Errorf("artifact file exceeds resource limit") + } + after, err := file.Stat() + if err != nil || !os.SameFile(opened, after) || opened.Size() != after.Size() || after.Size() != int64(len(content)) { + return nil, fmt.Errorf("artifact source changed during read") + } + return content, nil +} + +func Remove(rootPath, relativePath string) error { + path, err := localPath(relativePath) + if err != nil { + return err + } + root, err := os.OpenRoot(rootPath) + if err != nil { + return fmt.Errorf("open artifact root failed") + } + defer root.Close() + exists, err := admitDirectories(root, filepath.Dir(path), false) + if err != nil { + return err + } + if !exists { + return nil + } + info, err := root.Lstat(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return fmt.Errorf("inspect artifact destination failed") + } + if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() { + return fmt.Errorf("artifact destination must be a regular non-symlink file") + } + if err := root.Remove(path); err != nil { + return fmt.Errorf("remove artifact file failed") + } + return nil +} + +func localPath(relativePath string) (string, error) { + path := filepath.Clean(filepath.FromSlash(relativePath)) + if path == "." || !filepath.IsLocal(path) || filepath.ToSlash(path) != relativePath { + return "", fmt.Errorf("artifact path must be normalized and repository-relative") + } + return path, nil +} + +func admitDirectories(root *os.Root, directory string, create bool) (bool, error) { + if directory == "." { + return true, nil + } + current := "" + for _, component := range strings.Split(directory, string(filepath.Separator)) { + current = filepath.Join(current, component) + info, err := root.Lstat(current) + if errors.Is(err, fs.ErrNotExist) { + if !create { + return false, nil + } + if err := root.Mkdir(current, 0o755); err != nil { + return false, fmt.Errorf("create artifact directory failed") + } + info, err = root.Lstat(current) + } + if err != nil { + return false, fmt.Errorf("inspect artifact directory failed") + } + if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() { + return false, fmt.Errorf("artifact path traverses a symlink or non-directory") + } + } + return true, nil +} + +func admitDestination(root *os.Root, path string) error { + info, err := root.Lstat(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return fmt.Errorf("inspect artifact destination failed") + } + if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() { + return fmt.Errorf("artifact destination must be a regular non-symlink file") + } + return nil +} diff --git a/internal/tools/artifactfile/file_test.go b/internal/tools/artifactfile/file_test.go new file mode 100644 index 0000000..81b45d8 --- /dev/null +++ b/internal/tools/artifactfile/file_test.go @@ -0,0 +1,135 @@ +package artifactfile + +import ( + "errors" + "math" + "os" + "path/filepath" + "testing" +) + +func TestWriteReadAndRemoveRoundTrip(t *testing.T) { + root := t.TempDir() + const path = "artifacts/proofkit/report.json" + if err := WriteAtomic(root, path, []byte("report\n"), 0o640); err != nil { + t.Fatalf("WriteAtomic() error = %v", err) + } + content, err := ReadBounded(root, path, 64) + if err != nil { + t.Fatalf("ReadBounded() error = %v", err) + } + if string(content) != "report\n" { + t.Fatalf("ReadBounded() = %q", content) + } + info, err := os.Stat(filepath.Join(root, filepath.FromSlash(path))) + if err != nil { + t.Fatal(err) + } + if info.Mode().Perm() != 0o640 { + t.Fatalf("artifact mode = %04o, want 0640", info.Mode().Perm()) + } + if err := Remove(root, path); err != nil { + t.Fatalf("Remove() error = %v", err) + } + if _, err := os.Stat(filepath.Join(root, filepath.FromSlash(path))); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("artifact remains after Remove(): %v", err) + } +} + +func TestReadBoundedRejectsUnrepresentableLimit(t *testing.T) { + root := t.TempDir() + if _, err := ReadBounded(root, "artifact.json", math.MaxInt64); err == nil { + t.Fatal("ReadBounded() admitted a limit whose sentinel byte overflows") + } +} + +func TestOperationsRejectSymlinkComponentsWithoutOutsideMutation(t *testing.T) { + for _, testCase := range []struct { + name string + setup func(t *testing.T, root, outside string) + }{ + { + name: "parent outside root", + setup: func(t *testing.T, root, outside string) { + t.Helper() + if err := os.Mkdir(filepath.Join(root, "artifacts"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink(outside, filepath.Join(root, "artifacts", "proofkit")); err != nil { + t.Fatal(err) + } + }, + }, + { + name: "parent inside root", + setup: func(t *testing.T, root, _ string) { + t.Helper() + if err := os.MkdirAll(filepath.Join(root, "artifacts", "alias"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink("alias", filepath.Join(root, "artifacts", "proofkit")); err != nil { + t.Fatal(err) + } + }, + }, + } { + t.Run(testCase.name, func(t *testing.T) { + root := t.TempDir() + outside := t.TempDir() + testCase.setup(t, root, outside) + const path = "artifacts/proofkit/report.json" + if err := WriteAtomic(root, path, []byte("counterfeit"), 0o644); err == nil { + t.Fatal("WriteAtomic() admitted a symlink component") + } + if _, err := ReadBounded(root, path, 64); err == nil { + t.Fatal("ReadBounded() admitted a symlink component") + } + if err := Remove(root, path); err == nil { + t.Fatal("Remove() admitted a symlink component") + } + entries, err := os.ReadDir(outside) + if err != nil { + t.Fatal(err) + } + if len(entries) != 0 { + t.Fatalf("outside directory was mutated: %v", entries) + } + }) + } +} + +func TestOperationsRejectFinalSymlinkWithoutTargetMutation(t *testing.T) { + root := t.TempDir() + outside := filepath.Join(t.TempDir(), "target.json") + if err := os.WriteFile(outside, []byte("outside"), 0o600); err != nil { + t.Fatal(err) + } + directory := filepath.Join(root, "artifacts", "proofkit") + if err := os.MkdirAll(directory, 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink(outside, filepath.Join(directory, "report.json")); err != nil { + t.Fatal(err) + } + for _, operation := range []struct { + name string + run func() error + }{ + {name: "write", run: func() error { return WriteAtomic(root, "artifacts/proofkit/report.json", []byte("counterfeit"), 0o644) }}, + {name: "read", run: func() error { _, err := ReadBounded(root, "artifacts/proofkit/report.json", 64); return err }}, + {name: "remove", run: func() error { return Remove(root, "artifacts/proofkit/report.json") }}, + } { + t.Run(operation.name, func(t *testing.T) { + if err := operation.run(); err == nil { + t.Fatal("operation admitted a final symlink") + } + }) + } + content, err := os.ReadFile(outside) + if err != nil { + t.Fatal(err) + } + if string(content) != "outside" { + t.Fatalf("outside target was mutated: %q", content) + } +} diff --git a/internal/tools/commandoracle/corpus.go b/internal/tools/commandoracle/corpus.go new file mode 100644 index 0000000..1f021d3 --- /dev/null +++ b/internal/tools/commandoracle/corpus.go @@ -0,0 +1,195 @@ +package commandoracle + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "reflect" + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/tools/artifactfile" +) + +const ( + CounterfeitCorpusPath = "internal/tools/commandoracle/testdata/counterfeit-corpus.v1.json" + maxCorpusBytes = 1 << 20 +) + +type counterfeitCorpus struct { + CorpusID string `json:"corpusId"` + SchemaVersion int `json:"schemaVersion"` + Cases []counterfeitCase `json:"cases"` +} + +type counterfeitCase struct { + CaseID string `json:"caseId"` + Coordinate string `json:"coordinate"` + EvidenceClass string `json:"evidenceClass"` + ExpectedDecision string `json:"expectedDecision"` + MutationID string `json:"mutationId"` + PolicyID string `json:"policyId"` +} + +type counterfeitAxis struct { + MutationID string + EvidenceClass string +} + +var requiredCounterfeitAxes = []counterfeitAxis{ + {MutationID: "event-attribute-cross-test", EvidenceClass: "execution"}, + {MutationID: "event-attribute-duplicate", EvidenceClass: "execution"}, + {MutationID: "event-attribute-missing", EvidenceClass: "execution"}, + {MutationID: "event-descendant-skip", EvidenceClass: "execution"}, + {MutationID: "event-output-spoof", EvidenceClass: "execution"}, + {MutationID: "event-package-pass-before-tests", EvidenceClass: "execution"}, + {MutationID: "event-package-pass-missing", EvidenceClass: "execution"}, + {MutationID: "event-pass-before-run", EvidenceClass: "execution"}, + {MutationID: "event-pass-duplicate", EvidenceClass: "execution"}, + {MutationID: "event-pause-before-run", EvidenceClass: "execution"}, + {MutationID: "event-pause-duplicate", EvidenceClass: "execution"}, + {MutationID: "event-run-duplicate", EvidenceClass: "execution"}, + {MutationID: "event-selected-fail", EvidenceClass: "execution"}, + {MutationID: "event-selected-skip", EvidenceClass: "execution"}, + {MutationID: "event-unknown-action", EvidenceClass: "execution"}, + {MutationID: "join-correlated-command-identity", EvidenceClass: "joined"}, + {MutationID: "join-correlated-outcome-marker", EvidenceClass: "joined"}, + {MutationID: "join-correlated-selector-test", EvidenceClass: "joined"}, + {MutationID: "positive-candidate", EvidenceClass: "candidate"}, + {MutationID: "positive-execution-shared-test", EvidenceClass: "execution"}, + {MutationID: "positive-joined", EvidenceClass: "joined"}, + {MutationID: "record-execution-command-drift", EvidenceClass: "joined"}, + {MutationID: "source-correlated-identity", EvidenceClass: "joined"}, +} + +func ValidateCounterfeitCorpus(root string) (string, error) { + content, err := artifactfile.ReadBounded(root, CounterfeitCorpusPath, maxCorpusBytes) + if err != nil { + return "", decision("corpus.file_missing") + } + if len(content) == 0 { + return "", decision("corpus.resource_limit") + } + corpus, err := admitCorpus(content) + if err != nil { + return "", err + } + if err := validateCounterfeitCorpusClosure(corpus); err != nil { + return "", err + } + positiveClasses := map[string]bool{"candidate": false, "execution": false, "joined": false} + for _, item := range corpus.Cases { + decisionID := evaluateCounterfeit(item) + if decisionID != item.ExpectedDecision { + return "", decision("corpus.expected_decision_mismatch") + } + if item.ExpectedDecision == "admit" { + positiveClasses[item.EvidenceClass] = true + } + } + for _, evidenceClass := range []string{"candidate", "execution", "joined"} { + if !positiveClasses[evidenceClass] { + return "", decision("corpus.positive_control_missing") + } + } + digest := sha256.Sum256(content) + return hex.EncodeToString(digest[:]), nil +} + +func admitCorpus(content []byte) (counterfeitCorpus, error) { + raw, err := admission.DecodeJSON(strings.NewReader(string(content)), maxCorpusBytes) + if err != nil { + return counterfeitCorpus{}, decision("corpus.json_invalid") + } + record, ok := raw.(map[string]any) + if !ok { + return counterfeitCorpus{}, decision("corpus.object_required") + } + if err := admit.KnownKeys(record, []string{"cases", "corpusId", "schemaVersion"}, "command oracle counterfeit corpus"); err != nil { + return counterfeitCorpus{}, decision("corpus.unknown_field") + } + caseValues, ok := record["cases"].([]any) + if !ok || len(caseValues) == 0 { + return counterfeitCorpus{}, decision("corpus.cases_invalid") + } + for _, value := range caseValues { + item, ok := value.(map[string]any) + if !ok { + return counterfeitCorpus{}, decision("corpus.case_object_required") + } + if err := admit.KnownKeys(item, []string{"caseId", "coordinate", "evidenceClass", "expectedDecision", "mutationId", "policyId"}, "command oracle counterfeit case"); err != nil { + return counterfeitCorpus{}, decision("corpus.case_unknown_field") + } + } + encoded, err := json.Marshal(raw) + if err != nil { + return counterfeitCorpus{}, err + } + var corpus counterfeitCorpus + if err := json.Unmarshal(encoded, &corpus); err != nil { + return counterfeitCorpus{}, decision("corpus.case_type_invalid") + } + if corpus.SchemaVersion != 1 || corpus.CorpusID != "proofkit.command-oracle.counterfeit-corpus.v1" { + return counterfeitCorpus{}, decision("corpus.identity_invalid") + } + for index, item := range corpus.Cases { + if strings.TrimSpace(item.CaseID) == "" || strings.TrimSpace(item.EvidenceClass) == "" || strings.TrimSpace(item.ExpectedDecision) == "" || strings.TrimSpace(item.MutationID) == "" || strings.TrimSpace(item.PolicyID) == "" { + return counterfeitCorpus{}, decision("corpus.case_field_empty") + } + if index > 0 && corpus.Cases[index-1].CaseID >= item.CaseID { + return counterfeitCorpus{}, decision("corpus.case_order_invalid") + } + } + return corpus, nil +} + +func validateCounterfeitCorpusClosure(corpus counterfeitCorpus) error { + allowedClasses := map[string]struct{}{"candidate": {}, "execution": {}, "joined": {}} + requiredAxes := make(map[string]string, len(requiredCounterfeitAxes)) + for _, axis := range requiredCounterfeitAxes { + requiredAxes[axis.MutationID] = axis.EvidenceClass + } + requiredCoordinates := map[string]struct{}{} + for _, coordinate := range schemaCoordinates(reflect.TypeOf(Record{}), "record") { + requiredCoordinates[coordinate] = struct{}{} + } + seenMutations := map[string]struct{}{} + seenPolicies := map[string]struct{}{} + for _, item := range corpus.Cases { + if _, ok := allowedClasses[item.EvidenceClass]; !ok { + return decision("corpus.evidence_class_invalid") + } + if _, duplicate := seenMutations[item.MutationID]; duplicate { + return decision("corpus.mutation_duplicate") + } + if _, duplicate := seenPolicies[item.PolicyID]; duplicate { + return decision("corpus.policy_duplicate") + } + seenMutations[item.MutationID] = struct{}{} + seenPolicies[item.PolicyID] = struct{}{} + if strings.HasPrefix(item.MutationID, "record-coordinate:") { + coordinate := strings.TrimPrefix(item.MutationID, "record-coordinate:") + if item.Coordinate != coordinate || item.EvidenceClass != "joined" { + return decision("corpus.coordinate_identity_invalid") + } + if _, required := requiredCoordinates[coordinate]; !required { + return decision("corpus.coordinate_unknown") + } + delete(requiredCoordinates, coordinate) + continue + } + evidenceClass, required := requiredAxes[item.MutationID] + if !required || item.Coordinate != "" || item.EvidenceClass != evidenceClass { + return decision("corpus.policy_axis_invalid") + } + delete(requiredAxes, item.MutationID) + } + if len(requiredAxes) != 0 { + return decision("corpus.policy_axis_missing") + } + if len(requiredCoordinates) != 0 { + return decision("corpus.coordinate_missing") + } + return nil +} diff --git a/internal/tools/commandoracle/corpus_coordinates.go b/internal/tools/commandoracle/corpus_coordinates.go new file mode 100644 index 0000000..ddcf0d7 --- /dev/null +++ b/internal/tools/commandoracle/corpus_coordinates.go @@ -0,0 +1,139 @@ +package commandoracle + +import ( + "encoding/json" + "reflect" + "sort" + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" +) + +func schemaCoordinates(target reflect.Type, prefix string) []string { + for target.Kind() == reflect.Pointer { + target = target.Elem() + } + coordinates := []string{} + switch target.Kind() { + case reflect.Struct: + for index := 0; index < target.NumField(); index++ { + field := target.Field(index) + name := strings.Split(field.Tag.Get("json"), ",")[0] + if name == "" || name == "-" { + continue + } + coordinates = append(coordinates, schemaCoordinates(field.Type, prefix+"."+name)...) + } + case reflect.Slice, reflect.Array: + element := target.Elem() + for element.Kind() == reflect.Pointer { + element = element.Elem() + } + if element.Kind() == reflect.Struct { + coordinates = append(coordinates, schemaCoordinates(element, prefix+"[]")...) + } else { + coordinates = append(coordinates, prefix+"[]") + } + default: + coordinates = append(coordinates, prefix) + } + sort.Strings(coordinates) + return coordinates +} + +func syntheticRecordValue() (map[string]any, error) { + candidates := syntheticCandidates() + candidateDigest, err := CandidateSetDigest(candidates) + if err != nil { + return nil, err + } + imports := map[string]string{"./internal/sample": "example.test/proofkit/internal/sample"} + entries := make([]JoinedEntry, 0, len(candidates)) + for _, candidate := range candidates { + entries = append(entries, JoinedEntry{ + Candidate: candidate, + ExecutionState: "passed", + PackageImportPath: imports[candidate.PackagePath], + }) + } + record := Record{ + ArtifactKind: ArtifactKind, + CandidateSetDigest: candidateDigest, + CommandID: CommandID, + CounterfeitCorpusDigest: strings.Repeat("2", 64), + Entries: entries, + ExecutionCommands: executionCommands(candidates), + GoVersion: "go1.26.5", + NonClaims: RecordNonClaims(), + Platform: "darwin/arm64", + SchemaVersion: SchemaVersion, + SourceRevision: strings.Repeat("a", 40), + SourceSnapshotDigest: strings.Repeat("3", 64), + State: "passed", + } + if err := validateRecordShape(record); err != nil { + return nil, err + } + return recordValue(record), nil +} + +func mutateCoordinate(root map[string]any, coordinate string) bool { + parts := strings.Split(coordinate, ".") + var current any = root + for index, part := range parts { + isArray := strings.HasSuffix(part, "[]") + key := strings.TrimSuffix(part, "[]") + record, ok := current.(map[string]any) + if !ok { + return false + } + value, ok := record[key] + if !ok { + return false + } + if isArray { + values, ok := value.([]any) + if !ok || len(values) == 0 { + return false + } + if index == len(parts)-1 { + values[0] = counterfeitScalar(values[0]) + return true + } + current = values[0] + continue + } + if index == len(parts)-1 { + record[key] = counterfeitScalar(value) + return true + } + current = value + } + return false +} + +func counterfeitScalar(value any) any { + switch typed := value.(type) { + case string: + return "" + case json.Number: + return json.Number("0") + case bool: + return !typed + default: + return nil + } +} + +func admitMutatedRecord(value map[string]any) string { + content, err := stableRecordBytes(value) + if err != nil { + return "internal_error" + } + _, err = admitRecordBytes(content) + return decisionOrAdmit(err) +} + +func stableRecordBytes(value map[string]any) ([]byte, error) { + return stablejson.Marshal(value) +} diff --git a/internal/tools/commandoracle/corpus_mutations.go b/internal/tools/commandoracle/corpus_mutations.go new file mode 100644 index 0000000..c34fc49 --- /dev/null +++ b/internal/tools/commandoracle/corpus_mutations.go @@ -0,0 +1,226 @@ +package commandoracle + +import ( + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" + "github.com/research-engineering/agentic-proofkit/internal/tools/repositorysnapshot" +) + +func evaluateCounterfeit(item counterfeitCase) string { + if strings.HasPrefix(item.MutationID, "record-coordinate:") { + coordinate := strings.TrimPrefix(item.MutationID, "record-coordinate:") + mutated, err := syntheticRecordValue() + if err != nil { + return "internal_error" + } + if !mutateCoordinate(mutated, strings.TrimPrefix(coordinate, "record.")) { + return "internal_error" + } + return admitMutatedRecord(mutated) + } + switch item.MutationID { + case "positive-candidate": + return decisionOrAdmit(validateCandidates(syntheticCandidates())) + case "positive-joined": + return evaluateJoinMutation("") + case "positive-execution-shared-test": + return evaluateEventMutation("positive") + case "record-execution-command-drift": + mutated, err := syntheticRecordValue() + if err != nil { + return "internal_error" + } + commands := mutated["executionCommands"].([]any) + command := commands[0].(map[string]any) + argv := command["argv"].([]any) + argv[len(argv)-2] = "^TestCounterfeit$" + return admitMutatedRecord(mutated) + case "event-attribute-cross-test": + return evaluateEventMutation("attribute-cross-test") + case "event-attribute-duplicate": + return evaluateEventMutation("attribute-duplicate") + case "event-attribute-missing": + return evaluateEventMutation("attribute-missing") + case "event-descendant-skip": + return evaluateEventMutation("descendant-skip") + case "event-output-spoof": + return evaluateEventMutation("output-spoof") + case "event-package-pass-before-tests": + return evaluateEventMutation("package-pass-before-tests") + case "event-package-pass-missing": + return evaluateEventMutation("package-pass-missing") + case "event-pass-before-run": + return evaluateEventMutation("pass-before-run") + case "event-pass-duplicate": + return evaluateEventMutation("pass-duplicate") + case "event-pause-before-run": + return evaluateEventMutation("pause-before-run") + case "event-pause-duplicate": + return evaluateEventMutation("pause-duplicate") + case "event-run-duplicate": + return evaluateEventMutation("run-duplicate") + case "event-selected-fail": + return evaluateEventMutation("selected-fail") + case "event-selected-skip": + return evaluateEventMutation("selected-skip") + case "event-unknown-action": + return evaluateEventMutation("unknown-action") + case "join-correlated-command-identity": + return evaluateJoinMutation("commandRef") + case "join-correlated-outcome-marker": + return evaluateJoinMutation("outcomeMarker") + case "join-correlated-selector-test": + return evaluateJoinMutation("selector") + case "source-correlated-identity": + left := repositorysnapshot.Snapshot{Digest: strings.Repeat("1", 64), Paths: []string{"go.mod"}, Revision: strings.Repeat("a", 40)} + right := left + right.Digest = strings.Repeat("2", 64) + if repositorysnapshot.EqualIdentity(left, right) { + return "admit" + } + return "source.current_snapshot_mismatch" + default: + return "internal_error" + } +} + +func evaluateJoinMutation(field string) string { + candidates := syntheticCandidates() + imports := map[string]string{"./internal/sample": "example.test/proofkit/internal/sample"} + entries := make([]JoinedEntry, 0, len(candidates)) + for _, candidate := range candidates { + entries = append(entries, JoinedEntry{Candidate: candidate, ExecutionState: "passed", PackageImportPath: imports[candidate.PackagePath]}) + } + if field != "" { + switch field { + case "commandRef": + entries[0].Candidate.CommandRef += ".counterfeit" + case "outcomeMarker": + entries[0].Candidate.ExpectedPublicOutcome += " Counterfeit." + entries[0].Candidate.SourceMarker = strings.Repeat("9", len(entries[0].Candidate.SourceMarker)) + case "selector": + entries[0].Candidate.Selector += ".counterfeit" + } + } + return decisionOrAdmit(validateJoinedEntries(candidates, entries, imports)) +} + +func evaluateEventMutation(mutation string) string { + candidates := syntheticCandidates() + imports := map[string]string{"./internal/sample": "example.test/proofkit/internal/sample"} + ledger, err := newEventLedger(candidates, imports) + if err != nil { + return DecisionID(err) + } + key := selectedTestKey{Package: imports["./internal/sample"], Test: "TestShared"} + observe := func(event testEvent) string { + if err := ledger.observe(event); err != nil { + return DecisionID(err) + } + return "" + } + if got := observe(testEvent{Action: "start", Package: key.Package}); got != "" { + return got + } + if mutation == "unknown-action" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "counterfeit", Package: key.Package})) + } + if mutation == "package-pass-before-tests" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "pass", Package: key.Package})) + } + if mutation == "pass-before-run" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "pass", Package: key.Package, Test: key.Test})) + } + if mutation == "pause-before-run" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "pause", Package: key.Package, Test: key.Test})) + } + if got := observe(testEvent{Action: "run", Package: key.Package, Test: key.Test}); got != "" { + return got + } + if mutation == "pause-duplicate" { + if got := observe(testEvent{Action: "pause", Package: key.Package, Test: key.Test}); got != "" { + return got + } + return decisionOrAdmit(ledger.observe(testEvent{Action: "pause", Package: key.Package, Test: key.Test})) + } + if mutation == "run-duplicate" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "run", Package: key.Package, Test: key.Test})) + } + if mutation == "attribute-cross-test" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "attr", Package: key.Package, Test: "TestOther", Key: commandcoverage.ExecutionAttributeKey, Value: candidates[0].SourceMarker})) + } + if mutation == "descendant-skip" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "skip", Package: key.Package, Test: key.Test + "/child"})) + } + if mutation == "selected-fail" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "fail", Package: key.Package, Test: key.Test})) + } + if mutation == "selected-skip" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "skip", Package: key.Package, Test: key.Test})) + } + if mutation == "output-spoof" { + if got := observe(testEvent{Action: "output", Package: key.Package, Test: key.Test, Value: candidates[0].SourceMarker}); got != "" { + return got + } + return decisionOrAdmit(ledger.observe(testEvent{Action: "pass", Package: key.Package, Test: key.Test})) + } + if mutation != "attribute-missing" { + for _, candidate := range candidates { + if got := observe(testEvent{Action: "attr", Package: key.Package, Test: key.Test, Key: commandcoverage.ExecutionAttributeKey, Value: candidate.SourceMarker}); got != "" { + return got + } + } + } + if mutation == "attribute-duplicate" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "attr", Package: key.Package, Test: key.Test, Key: commandcoverage.ExecutionAttributeKey, Value: candidates[0].SourceMarker})) + } + if got := observe(testEvent{Action: "pass", Package: key.Package, Test: key.Test}); got != "" { + return got + } + if mutation == "pass-duplicate" { + return decisionOrAdmit(ledger.observe(testEvent{Action: "pass", Package: key.Package, Test: key.Test})) + } + if mutation != "package-pass-missing" { + if got := observe(testEvent{Action: "pass", Package: key.Package}); got != "" { + return got + } + } + return decisionOrAdmit(ledger.finalize()) +} + +func decisionOrAdmit(err error) string { + if err == nil { + return "admit" + } + return DecisionID(err) +} + +func syntheticCandidates() []app.CommandCoverageOracleCandidate { + base := app.CommandCoverageOracleCandidate{ + AssertionOracleID: "proofkit.oracle.one", + CommandRef: "proofkit.cli.sample", + ExpectedPublicOutcome: "Sample command rejects the counterfeit input.", + FalsificationEventID: "proofkit.falsifier.one", + NegativeCaseID: "proofkit.negative.one", + OracleKind: "semantic_route_falsifier", + OwnerInvariantID: "proofkit.invariant.one", + PackagePath: "./internal/sample", + Selector: "internal/sample/sample_test.go::TestShared", + SourceMarker: "proofkit.command_coverage.source_oracle.v1.000000000000000000000000000000000000000000000000000000000000000000000000000001", + SourcePath: "internal/sample/sample_test.go", + TestID: "proofkit.test.one", + TestName: "TestShared", + WrongImplementationClass: "proofkit.wrong.one", + } + second := base + second.AssertionOracleID = "proofkit.oracle.two" + second.FalsificationEventID = "proofkit.falsifier.two" + second.NegativeCaseID = "proofkit.negative.two" + second.OwnerInvariantID = "proofkit.invariant.two" + second.SourceMarker = "proofkit.command_coverage.source_oracle.v1.000000000000000000000000000000000000000000000000000000000000000000000000000002" + second.TestID = "proofkit.test.two" + second.WrongImplementationClass = "proofkit.wrong.two" + return []app.CommandCoverageOracleCandidate{base, second} +} diff --git a/internal/tools/commandoracle/corpus_test.go b/internal/tools/commandoracle/corpus_test.go new file mode 100644 index 0000000..abc8aa0 --- /dev/null +++ b/internal/tools/commandoracle/corpus_test.go @@ -0,0 +1,73 @@ +package commandoracle + +import ( + "os" + "path/filepath" + "testing" +) + +func TestCounterfeitCorpusClosesRequiredAxes(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatal(err) + } + digest, err := ValidateCounterfeitCorpus(root) + if err != nil { + t.Fatalf("ValidateCounterfeitCorpus() error = %v", err) + } + if !isSHA256(digest) { + t.Fatalf("ValidateCounterfeitCorpus() digest = %q, want SHA-256", digest) + } +} + +func TestEachCounterfeitCaseProducesItsCheckedInDecision(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatal(err) + } + content, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(CounterfeitCorpusPath))) + if err != nil { + t.Fatal(err) + } + corpus, err := admitCorpus(content) + if err != nil { + t.Fatal(err) + } + for _, item := range corpus.Cases { + t.Run(item.CaseID, func(t *testing.T) { + if got := evaluateCounterfeit(item); got != item.ExpectedDecision { + t.Fatalf("evaluateCounterfeit() = %q, want checked-in %q", got, item.ExpectedDecision) + } + }) + } +} + +func TestCounterfeitCorpusClosureRejectsMissingRequiredAxes(t *testing.T) { + corpus := readCounterfeitCorpusFixture(t) + for index, item := range corpus.Cases { + if item.MutationID == "record-execution-command-drift" { + corpus.Cases = append(corpus.Cases[:index], corpus.Cases[index+1:]...) + break + } + } + if err := validateCounterfeitCorpusClosure(corpus); DecisionID(err) != "corpus.policy_axis_missing" { + t.Fatalf("validateCounterfeitCorpusClosure() error = %v", err) + } +} + +func readCounterfeitCorpusFixture(t *testing.T) counterfeitCorpus { + t.Helper() + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatal(err) + } + content, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(CounterfeitCorpusPath))) + if err != nil { + t.Fatal(err) + } + corpus, err := admitCorpus(content) + if err != nil { + t.Fatal(err) + } + return corpus +} diff --git a/internal/tools/commandoracle/current.go b/internal/tools/commandoracle/current.go new file mode 100644 index 0000000..6336370 --- /dev/null +++ b/internal/tools/commandoracle/current.go @@ -0,0 +1,57 @@ +package commandoracle + +import ( + "bytes" + "context" + "reflect" + "runtime" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/tools/repositorysnapshot" +) + +func ValidateCurrent(ctx context.Context, root string, evidence Evidence) error { + admitted, err := EvidenceForRecord(evidence.Record) + if err != nil { + return err + } + if admitted.RecordDigest != evidence.RecordDigest || !bytes.Equal(admitted.RecordBytes, evidence.RecordBytes) { + return decision("current.record_bytes_mismatch") + } + candidates, err := app.CommandCoverageOracleCandidatesAtRoot(root) + if err != nil { + return err + } + if !reflect.DeepEqual(candidates, evidence.Candidates) { + return decision("current.candidate_projection_mismatch") + } + candidateDigest, err := CandidateSetDigest(candidates) + if err != nil || candidateDigest != evidence.Record.CandidateSetDigest { + return decision("current.candidate_set_digest_mismatch") + } + corpusDigest, err := ValidateCounterfeitCorpus(root) + if err != nil { + return err + } + if corpusDigest != evidence.Record.CounterfeitCorpusDigest { + return decision("current.counterfeit_corpus_digest_mismatch") + } + modulePath, err := readModulePath(root) + if err != nil { + return err + } + if err := validateJoinedEntries(candidates, evidence.Record.Entries, packageImportPaths(modulePath, candidates)); err != nil { + return err + } + if evidence.Record.GoVersion != runtime.Version() || evidence.Record.Platform != runtime.GOOS+"/"+runtime.GOARCH { + return decision("current.runtime_identity_mismatch") + } + snapshot, err := repositorysnapshot.CaptureContext(ctx, root) + if err != nil { + return err + } + if snapshot.Revision != evidence.Record.SourceRevision || snapshot.Digest != evidence.Record.SourceSnapshotDigest { + return decision("current.source_snapshot_mismatch") + } + return nil +} diff --git a/internal/tools/commandoracle/events.go b/internal/tools/commandoracle/events.go new file mode 100644 index 0000000..92b2889 --- /dev/null +++ b/internal/tools/commandoracle/events.go @@ -0,0 +1,400 @@ +package commandoracle + +import ( + "bufio" + "encoding/json" + "io" + "sort" + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" +) + +const ( + maxEventLineBytes = 1 << 20 + maxEventBytes = 64 << 20 + maxEventCount = 2_000_000 +) + +type testEvent struct { + Action string + FailedBuild string + Key string + Package string + Test string + Value string +} + +type selectedTestKey struct { + Package string + Test string +} + +type selectedTestState struct { + attributes map[string]struct{} + passed bool + paused bool + run bool +} + +type descendantTestState struct { + passed bool + paused bool + run bool +} + +type packageEventState struct { + passed bool + started bool +} + +type eventLedger struct { + expectedAttributes map[selectedTestKey]map[string]struct{} + descendants map[selectedTestKey]*descendantTestState + packages map[string]*packageEventState + tests map[selectedTestKey]*selectedTestState +} + +func newEventLedger(candidates []app.CommandCoverageOracleCandidate, packageImports map[string]string) (*eventLedger, error) { + ledger := &eventLedger{ + expectedAttributes: map[selectedTestKey]map[string]struct{}{}, + descendants: map[selectedTestKey]*descendantTestState{}, + packages: map[string]*packageEventState{}, + tests: map[selectedTestKey]*selectedTestState{}, + } + for _, candidate := range candidates { + packageImport, ok := packageImports[candidate.PackagePath] + if !ok { + return nil, decision("join.package_import_missing") + } + key := selectedTestKey{Package: packageImport, Test: candidate.TestName} + if _, ok := ledger.tests[key]; !ok { + ledger.tests[key] = &selectedTestState{attributes: map[string]struct{}{}} + ledger.expectedAttributes[key] = map[string]struct{}{} + } + if _, exists := ledger.expectedAttributes[key][candidate.SourceMarker]; exists { + return nil, decision("join.attribute_identity_duplicate") + } + ledger.expectedAttributes[key][candidate.SourceMarker] = struct{}{} + if _, exists := ledger.packages[packageImport]; !exists { + ledger.packages[packageImport] = &packageEventState{} + } + } + return ledger, nil +} + +func parseEvents(reader io.Reader, ledger *eventLedger) error { + scanner := bufio.NewScanner(reader) + scanner.Buffer(make([]byte, 64<<10), maxEventLineBytes) + totalBytes := 0 + eventCount := 0 + for scanner.Scan() { + line := append([]byte(nil), scanner.Bytes()...) + totalBytes += len(line) + 1 + eventCount++ + if totalBytes > maxEventBytes { + return decision("event.total_bytes_exceeded") + } + if eventCount > maxEventCount { + return decision("event.count_exceeded") + } + event, err := admitEvent(line) + if err != nil { + return err + } + if err := ledger.observe(event); err != nil { + return err + } + } + if err := scanner.Err(); err != nil { + return decision("event.line_invalid_or_oversized") + } + return nil +} + +func admitEvent(line []byte) (testEvent, error) { + raw, err := admission.DecodeJSON(strings.NewReader(string(line)), maxEventLineBytes) + if err != nil { + return testEvent{}, decision("event.json_invalid") + } + record, ok := raw.(map[string]any) + if !ok { + return testEvent{}, decision("event.object_required") + } + if err := admit.KnownKeys(record, []string{"Action", "Elapsed", "FailedBuild", "Key", "Output", "Package", "Test", "Time", "Value"}, "go test event"); err != nil { + return testEvent{}, decision("event.unknown_field") + } + action, ok := record["Action"].(string) + if !ok || strings.TrimSpace(action) == "" { + return testEvent{}, decision("event.action_invalid") + } + if !validEventAction(action) { + return testEvent{}, decision("event.action_unknown") + } + for _, key := range []string{"FailedBuild", "Key", "Package", "Test", "Value"} { + if value, exists := record[key]; exists { + if _, ok := value.(string); !ok { + return testEvent{}, decision("event.field_type_invalid") + } + } + } + for _, key := range []string{"Output", "Time"} { + if value, exists := record[key]; exists { + if _, ok := value.(string); !ok { + return testEvent{}, decision("event.field_type_invalid") + } + } + } + if value, exists := record["Elapsed"]; exists { + if _, ok := value.(json.Number); !ok { + return testEvent{}, decision("event.field_type_invalid") + } + } + return testEvent{ + Action: action, + FailedBuild: stringField(record, "FailedBuild"), + Key: stringField(record, "Key"), + Package: stringField(record, "Package"), + Test: stringField(record, "Test"), + Value: stringField(record, "Value"), + }, nil +} + +func stringField(record map[string]any, key string) string { + value, _ := record[key].(string) + return value +} + +func validEventAction(action string) bool { + switch action { + case "attr", "bench", "cont", "fail", "output", "pass", "pause", "run", "skip", "start": + return true + default: + return false + } +} + +func (ledger *eventLedger) observe(event testEvent) error { + if !validEventAction(event.Action) { + return decision("event.action_unknown") + } + key := selectedTestKey{Package: event.Package, Test: event.Test} + state, selected := ledger.tests[key] + packageState, expectedPackage := ledger.packages[event.Package] + if !expectedPackage { + return decision("event.package_unknown") + } + if event.Test == "" && event.Action == "start" { + if packageState.started || packageState.passed { + return decision("event.package_start_duplicate") + } + packageState.started = true + return nil + } + if !packageState.started { + return decision("event.package_not_started") + } + if packageState.passed { + return decision("event.package_already_passed") + } + descendant := ledger.selectedDescendant(key) + if event.Test == "" { + switch event.Action { + case "fail", "output", "pass", "skip": + default: + return decision("event.package_action_invalid") + } + } + if event.Action == "attr" && event.Key == commandcoverage.ExecutionAttributeKey { + if !selected { + return decision("event.reserved_attribute_unknown_test") + } + if !state.run || state.passed { + return decision("event.reserved_attribute_wrong_order") + } + if _, expected := ledger.expectedAttributes[key][event.Value]; !expected { + return decision("event.reserved_attribute_unknown_value") + } + if _, duplicate := state.attributes[event.Value]; duplicate { + return decision("event.reserved_attribute_duplicate") + } + state.attributes[event.Value] = struct{}{} + return nil + } + if event.Test != "" && !selected && !descendant { + return decision("event.unselected_test_observed") + } + if ledger.selectedDescendantFailedOrSkipped(event) { + return decision("event.selected_descendant_failed_or_skipped") + } + if descendant { + parent := ledger.selectedParent(key) + if parent == nil || !parent.run || parent.passed { + return decision("event.selected_descendant_wrong_order") + } + descendantState := ledger.descendants[key] + switch event.Action { + case "run": + if descendantState != nil { + return decision("event.selected_descendant_run_duplicate") + } + ledger.descendants[key] = &descendantTestState{run: true} + case "pause": + if descendantState == nil || !descendantState.run || descendantState.paused || descendantState.passed { + return decision("event.selected_descendant_pause_wrong_order") + } + descendantState.paused = true + case "cont": + if descendantState == nil || !descendantState.run || !descendantState.paused || descendantState.passed { + return decision("event.selected_descendant_cont_wrong_order") + } + descendantState.paused = false + case "pass": + if descendantState == nil || !descendantState.run || descendantState.paused || descendantState.passed { + return decision("event.selected_descendant_pass_wrong_order") + } + descendantState.passed = true + case "attr", "output": + if descendantState == nil || !descendantState.run || descendantState.passed { + return decision("event.selected_descendant_auxiliary_wrong_order") + } + case "bench", "start": + return decision("event.selected_descendant_action_invalid") + } + return nil + } + if selected { + switch event.Action { + case "run": + if state.run || state.passed { + return decision("event.test_run_duplicate") + } + state.run = true + case "pass": + if !state.run || state.paused || state.passed || !ledger.descendantsPassed(key) { + return decision("event.test_pass_wrong_order") + } + if len(state.attributes) != len(ledger.expectedAttributes[key]) { + return decision("event.test_pass_missing_attributes") + } + state.passed = true + case "fail", "skip": + return decision("event.selected_test_failed_or_skipped") + case "pause": + if !state.run || state.paused || state.passed { + return decision("event.test_pause_wrong_order") + } + state.paused = true + case "cont": + if !state.run || !state.paused || state.passed { + return decision("event.test_cont_wrong_order") + } + state.paused = false + case "attr", "output": + if !state.run || state.passed { + return decision("event.selected_test_auxiliary_wrong_order") + } + case "bench", "start": + return decision("event.selected_test_action_invalid") + } + } + if event.Test == "" { + switch event.Action { + case "pass": + if !ledger.packageTestsPassed(event.Package) { + return decision("event.package_pass_before_tests") + } + packageState.passed = true + case "fail", "skip": + return decision("event.selected_package_failed_or_skipped") + } + } + return nil +} + +func (ledger *eventLedger) descendantsPassed(parent selectedTestKey) bool { + for key, state := range ledger.descendants { + if key.Package == parent.Package && strings.HasPrefix(key.Test, parent.Test+"/") && !state.passed { + return false + } + } + return true +} + +func (ledger *eventLedger) selectedDescendant(key selectedTestKey) bool { + return ledger.selectedParent(key) != nil +} + +func (ledger *eventLedger) selectedParent(key selectedTestKey) *selectedTestState { + for selectedKey, state := range ledger.tests { + if key.Package == selectedKey.Package && strings.HasPrefix(key.Test, selectedKey.Test+"/") { + return state + } + } + return nil +} + +func (ledger *eventLedger) packageTestsPassed(packageImport string) bool { + for key, state := range ledger.tests { + if key.Package == packageImport && !state.passed { + return false + } + } + return true +} + +func (ledger *eventLedger) selectedDescendantFailedOrSkipped(event testEvent) bool { + if event.Action != "fail" && event.Action != "skip" { + return false + } + for key := range ledger.tests { + if event.Package == key.Package && strings.HasPrefix(event.Test, key.Test+"/") { + return true + } + } + return false +} + +func (ledger *eventLedger) finalize() error { + keys := make([]selectedTestKey, 0, len(ledger.tests)) + for key := range ledger.tests { + keys = append(keys, key) + } + sort.Slice(keys, func(left, right int) bool { + return keys[left].Package+"\x00"+keys[left].Test < keys[right].Package+"\x00"+keys[right].Test + }) + for _, key := range keys { + state := ledger.tests[key] + if !state.run { + return decision("event.test_run_missing") + } + if !state.passed { + return decision("event.test_pass_missing") + } + if len(state.attributes) != len(ledger.expectedAttributes[key]) { + return decision("event.attribute_closure_missing") + } + } + for _, state := range ledger.descendants { + if !state.run || state.paused || !state.passed { + return decision("event.selected_descendant_incomplete") + } + } + packages := make([]string, 0, len(ledger.packages)) + for packageImport := range ledger.packages { + packages = append(packages, packageImport) + } + sort.Strings(packages) + for _, packageImport := range packages { + if !ledger.packages[packageImport].started { + return decision("event.package_start_missing") + } + if !ledger.packages[packageImport].passed { + return decision("event.package_pass_missing") + } + } + return nil +} diff --git a/internal/tools/commandoracle/events_test.go b/internal/tools/commandoracle/events_test.go new file mode 100644 index 0000000..24fe044 --- /dev/null +++ b/internal/tools/commandoracle/events_test.go @@ -0,0 +1,48 @@ +package commandoracle + +import "testing" + +func TestAdmitEventRejectsUnknownActionAndInvalidAuxiliaryTypes(t *testing.T) { + for _, testCase := range []struct { + name string + line string + want string + }{ + {name: "unknown action", line: `{"Action":"counterfeit","Package":"example.test/p"}`, want: "event.action_unknown"}, + {name: "elapsed string", line: `{"Action":"start","Elapsed":"0","Package":"example.test/p"}`, want: "event.field_type_invalid"}, + {name: "output object", line: `{"Action":"output","Output":{},"Package":"example.test/p"}`, want: "event.field_type_invalid"}, + {name: "time number", line: `{"Action":"start","Package":"example.test/p","Time":1}`, want: "event.field_type_invalid"}, + } { + t.Run(testCase.name, func(t *testing.T) { + if _, err := admitEvent([]byte(testCase.line)); DecisionID(err) != testCase.want { + t.Fatalf("admitEvent() error = %v, want %s", err, testCase.want) + } + }) + } +} + +func TestEventLedgerRejectsPackageAndTestActionContextDrift(t *testing.T) { + candidates := syntheticCandidates() + imports := map[string]string{"./internal/sample": "example.test/proofkit/internal/sample"} + for _, testCase := range []struct { + name string + event testEvent + want string + }{ + {name: "package run", event: testEvent{Action: "run", Package: imports["./internal/sample"]}, want: "event.package_action_invalid"}, + {name: "test start", event: testEvent{Action: "start", Package: imports["./internal/sample"], Test: "TestShared"}, want: "event.selected_test_action_invalid"}, + } { + t.Run(testCase.name, func(t *testing.T) { + ledger, err := newEventLedger(candidates, imports) + if err != nil { + t.Fatal(err) + } + if err := ledger.observe(testEvent{Action: "start", Package: imports["./internal/sample"]}); err != nil { + t.Fatal(err) + } + if err := ledger.observe(testCase.event); DecisionID(err) != testCase.want { + t.Fatalf("observe() error = %v, want %s", err, testCase.want) + } + }) + } +} diff --git a/internal/tools/commandoracle/execute.go b/internal/tools/commandoracle/execute.go new file mode 100644 index 0000000..2da5c55 --- /dev/null +++ b/internal/tools/commandoracle/execute.go @@ -0,0 +1,332 @@ +package commandoracle + +import ( + "context" + "fmt" + "go/ast" + "go/parser" + "go/token" + "io" + "os" + "os/exec" + "path/filepath" + "regexp" + "sort" + "strconv" + "strings" + "sync" + "time" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" + "github.com/research-engineering/agentic-proofkit/internal/tools/artifactfile" + "github.com/research-engineering/agentic-proofkit/internal/tools/repositorysnapshot" + "golang.org/x/mod/modfile" +) + +const ( + innerTestTimeout = 4*time.Minute + 30*time.Second + outerRunTimeout = 5 * time.Minute + processWaitDelay = 10 * time.Second + maxStderrBytes = 1 << 20 + maxGoModBytes = 1 << 20 +) + +type commandRunner func(context.Context, string, []ExecutionCommand, *eventLedger) error + +var runSelectedTests commandRunner = runGoTests + +func Execute(ctx context.Context, liveRoot string) (Evidence, error) { + operationContext, cancel := context.WithTimeout(ctx, outerRunTimeout) + defer cancel() + if err := InvalidateDiagnostic(liveRoot); err != nil { + return Evidence{}, err + } + materializedRoot, err := os.MkdirTemp("", "proofkit-command-oracle-") + if err != nil { + return Evidence{}, fmt.Errorf("create command oracle snapshot root: %w", err) + } + defer os.RemoveAll(materializedRoot) + + snapshot, err := repositorysnapshot.MaterializeContext(operationContext, liveRoot, materializedRoot) + if err != nil { + return Evidence{}, err + } + if err := repositorysnapshot.ValidateMaterializedContext(operationContext, materializedRoot, snapshot); err != nil { + return Evidence{}, err + } + currentAfterCopy, err := repositorysnapshot.CaptureContext(operationContext, liveRoot) + if err != nil { + return Evidence{}, err + } + if !repositorysnapshot.EqualIdentity(snapshot, currentAfterCopy) { + return Evidence{}, decision("source.changed_during_materialization") + } + + candidates, err := app.CommandCoverageOracleCandidatesAtRoot(materializedRoot) + if err != nil { + return Evidence{}, err + } + if len(candidates) == 0 { + return Evidence{}, decision("candidate.inventory_empty") + } + if err := rejectReservedAttributeForgery(materializedRoot, snapshot.Paths); err != nil { + return Evidence{}, err + } + modulePath, err := readModulePath(materializedRoot) + if err != nil { + return Evidence{}, err + } + packageImports := packageImportPaths(modulePath, candidates) + ledger, err := newEventLedger(candidates, packageImports) + if err != nil { + return Evidence{}, err + } + commands := executionCommands(candidates) + if err := runSelectedTests(operationContext, materializedRoot, commands, ledger); err != nil { + return Evidence{}, err + } + if err := ledger.finalize(); err != nil { + return Evidence{}, err + } + if err := repositorysnapshot.ValidateMaterializedContext(operationContext, materializedRoot, snapshot); err != nil { + return Evidence{}, decision("source.materialized_snapshot_mutated") + } + for attempt := 0; attempt < 2; attempt++ { + current, err := repositorysnapshot.CaptureContext(operationContext, liveRoot) + if err != nil { + return Evidence{}, err + } + if !repositorysnapshot.EqualIdentity(snapshot, current) { + return Evidence{}, decision("source.current_snapshot_mismatch") + } + } + corpusDigest, err := ValidateCounterfeitCorpus(materializedRoot) + if err != nil { + return Evidence{}, err + } + return buildEvidence(snapshot, candidates, packageImports, commands, corpusDigest) +} + +func executionCommands(candidates []app.CommandCoverageOracleCandidate) []ExecutionCommand { + testsByPackage := map[string]map[string]struct{}{} + for _, candidate := range candidates { + if testsByPackage[candidate.PackagePath] == nil { + testsByPackage[candidate.PackagePath] = map[string]struct{}{} + } + testsByPackage[candidate.PackagePath][candidate.TestName] = struct{}{} + } + packagePaths := make([]string, 0, len(testsByPackage)) + for packagePath := range testsByPackage { + packagePaths = append(packagePaths, packagePath) + } + sort.Strings(packagePaths) + commands := make([]ExecutionCommand, 0, len(packagePaths)) + for _, packagePath := range packagePaths { + testNames := make([]string, 0, len(testsByPackage[packagePath])) + for testName := range testsByPackage[packagePath] { + testNames = append(testNames, testName) + } + sort.Strings(testNames) + quoted := make([]string, 0, len(testNames)) + for _, testName := range testNames { + quoted = append(quoted, regexp.QuoteMeta(testName)) + } + commands = append(commands, ExecutionCommand{ + Argv: []string{"go", "test", "-json", "-count=1", "-timeout=" + innerTestTimeout.String(), "-run", "^(" + strings.Join(quoted, "|") + ")$", packagePath}, + PackagePath: packagePath, + }) + } + return commands +} + +func ExecutionCommandsForCandidates(candidates []app.CommandCoverageOracleCandidate) []ExecutionCommand { + return cloneExecutionCommands(executionCommands(candidates)) +} + +func runGoTests(ctx context.Context, root string, commands []ExecutionCommand, ledger *eventLedger) error { + for _, command := range commands { + if err := runGoTestCommand(ctx, root, command.Argv, ledger); err != nil { + return fmt.Errorf("command oracle package %s: %w", command.PackagePath, err) + } + } + return nil +} + +func runGoTestCommand(ctx context.Context, root string, argv []string, ledger *eventLedger) error { + goExecutable, err := exec.LookPath(argv[0]) + if err != nil { + return decision("process.go_executable_missing") + } + command := exec.CommandContext(ctx, goExecutable, argv[1:]...) + command.Dir = root + command.WaitDelay = processWaitDelay + configureProcessGroup(command) + stdout, err := command.StdoutPipe() + if err != nil { + return decision("process.stdout_pipe_failed") + } + stderr, err := command.StderrPipe() + if err != nil { + return decision("process.stderr_pipe_failed") + } + if err := command.Start(); err != nil { + return decision("process.start_failed") + } + stderrState := newBoundedDrain() + stderrDone := make(chan struct{}) + go func() { + defer close(stderrDone) + _, _ = io.Copy(stderrState, stderr) + }() + parseDone := make(chan error, 1) + go func() { parseDone <- parseEvents(stdout, ledger) }() + + var parseErr error + overflowChannel := stderrState.Exceeded() + contextChannel := ctx.Done() + overflowed := false + parseComplete := false + for !parseComplete { + select { + case parseErr = <-parseDone: + parseComplete = true + if parseErr != nil { + _ = terminateProcessGroup(command) + } + case <-overflowChannel: + overflowed = true + overflowChannel = nil + _ = terminateProcessGroup(command) + case <-contextChannel: + contextChannel = nil + _ = terminateProcessGroup(command) + } + } + waitDone := make(chan error, 1) + go func() { waitDone <- command.Wait() }() + var waitErr error + waitComplete := false + for !waitComplete { + select { + case waitErr = <-waitDone: + waitComplete = true + case <-overflowChannel: + overflowed = true + overflowChannel = nil + _ = terminateProcessGroup(command) + case <-contextChannel: + contextChannel = nil + _ = terminateProcessGroup(command) + } + } + <-stderrDone + if overflowed || stderrState.Overflowed() { + return decision("process.stderr_exceeded") + } + if ctx.Err() != nil { + return decision("process.timeout") + } + if parseErr != nil { + return parseErr + } + if waitErr != nil { + return decision("process.suite_failed") + } + return nil +} + +type boundedDrain struct { + count int + exceeded chan struct{} + once sync.Once +} + +func newBoundedDrain() *boundedDrain { + return &boundedDrain{exceeded: make(chan struct{})} +} + +func (drain *boundedDrain) Write(value []byte) (int, error) { + drain.count += len(value) + if drain.count > maxStderrBytes { + drain.once.Do(func() { close(drain.exceeded) }) + } + return len(value), nil +} + +func (drain *boundedDrain) Exceeded() <-chan struct{} { return drain.exceeded } + +func (drain *boundedDrain) Overflowed() bool { + select { + case <-drain.exceeded: + return true + default: + return false + } +} + +func readModulePath(root string) (string, error) { + content, err := artifactfile.ReadBounded(root, "go.mod", maxGoModBytes) + if err != nil { + return "", decision("module.file_missing") + } + modulePath := modfile.ModulePath(content) + if strings.TrimSpace(modulePath) == "" { + return "", decision("module.path_missing") + } + return modulePath, nil +} + +func packageImportPaths(modulePath string, candidates []app.CommandCoverageOracleCandidate) map[string]string { + out := map[string]string{} + for _, candidate := range candidates { + relative := strings.TrimPrefix(candidate.PackagePath, "./") + out[candidate.PackagePath] = modulePath + "/" + relative + } + return out +} + +func rejectReservedAttributeForgery(root string, paths []string) error { + const helperPath = "internal/testsupport/commandcoverage/semantic_route.go" + for _, path := range paths { + if !strings.HasSuffix(path, ".go") || path == helperPath { + continue + } + parsed, err := parser.ParseFile(token.NewFileSet(), filepath.Join(root, filepath.FromSlash(path)), nil, 0) + if err != nil { + return decision("source.go_parse_failed") + } + forged := false + ast.Inspect(parsed, func(node ast.Node) bool { + call, ok := node.(*ast.CallExpr) + if !ok || len(call.Args) < 2 { + return true + } + selector, ok := call.Fun.(*ast.SelectorExpr) + if !ok || selector.Sel.Name != "Attr" { + return true + } + if reservedAttributeArgument(call.Args[0]) { + forged = true + return false + } + return true + }) + if forged { + return decision("source.reserved_attribute_direct_use") + } + } + return nil +} + +func reservedAttributeArgument(expression ast.Expr) bool { + switch typed := expression.(type) { + case *ast.BasicLit: + value, err := strconv.Unquote(typed.Value) + return err == nil && value == commandcoverage.ExecutionAttributeKey + case *ast.SelectorExpr: + return typed.Sel.Name == "ExecutionAttributeKey" + default: + return false + } +} diff --git a/internal/tools/commandoracle/execute_test.go b/internal/tools/commandoracle/execute_test.go new file mode 100644 index 0000000..c9618f5 --- /dev/null +++ b/internal/tools/commandoracle/execute_test.go @@ -0,0 +1,267 @@ +package commandoracle + +import ( + "context" + "os" + "path/filepath" + "slices" + "sort" + "testing" + "time" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" +) + +func TestExecuteBindsMaterializedSourceCandidatesAndRuntimeEvents(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatal(err) + } + previousRunner := runSelectedTests + runSelectedTests = emitPassingSelectedTestEvents + t.Cleanup(func() { runSelectedTests = previousRunner }) + + evidence, err := Execute(context.Background(), root) + if err != nil { + t.Fatalf("Execute() error = %v", err) + } + if len(evidence.Candidates) == 0 || len(evidence.Record.Entries) != len(evidence.Candidates) { + t.Fatalf("Execute() evidence is not candidate-closed: %#v", evidence.Record) + } + if evidence.Record.State != "passed" || !isSHA256(evidence.RecordDigest) || !isSHA256(evidence.Record.SourceSnapshotDigest) { + t.Fatalf("Execute() identity is incomplete: %#v", evidence.Record) + } + if len(ExecutionCommandRefs(evidence)) == 0 || len(evidence.Record.ExecutionCommands) == 0 { + t.Fatalf("Execute() command projection is incomplete: %#v", evidence.Record.ExecutionCommands) + } + if err := ValidateCurrent(context.Background(), root, evidence); err != nil { + t.Fatalf("ValidateCurrent() rejected current evidence: %v", err) + } +} + +func TestValidateCurrentRejectsProducerUnreachableCandidateProjection(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatal(err) + } + previousRunner := runSelectedTests + runSelectedTests = emitPassingSelectedTestEvents + t.Cleanup(func() { runSelectedTests = previousRunner }) + evidence, err := Execute(context.Background(), root) + if err != nil { + t.Fatal(err) + } + record := evidence.Record + record.Entries = append([]JoinedEntry(nil), record.Entries...) + record.Entries[0].Candidate.TestID += ".counterfeit" + candidates := make([]app.CommandCoverageOracleCandidate, 0, len(record.Entries)) + for _, entry := range record.Entries { + candidates = append(candidates, entry.Candidate) + } + record.CandidateSetDigest, err = CandidateSetDigest(candidates) + if err != nil { + t.Fatal(err) + } + mutated, err := EvidenceForRecord(record) + if err != nil { + t.Fatalf("counterfeit record must remain internally valid: %v", err) + } + if err := ValidateCurrent(context.Background(), root, mutated); DecisionID(err) != "current.candidate_projection_mismatch" { + t.Fatalf("ValidateCurrent() error = %v, want current.candidate_projection_mismatch", err) + } +} + +func emitPassingSelectedTestEvents(_ context.Context, _ string, _ []ExecutionCommand, ledger *eventLedger) error { + packages := make([]string, 0, len(ledger.packages)) + for packagePath := range ledger.packages { + packages = append(packages, packagePath) + } + sort.Strings(packages) + for _, packagePath := range packages { + if err := ledger.observe(testEvent{Action: "start", Package: packagePath}); err != nil { + return err + } + } + tests := make([]selectedTestKey, 0, len(ledger.tests)) + for key := range ledger.tests { + tests = append(tests, key) + } + sort.Slice(tests, func(left, right int) bool { + return tests[left].Package+"\x00"+tests[left].Test < tests[right].Package+"\x00"+tests[right].Test + }) + for _, key := range tests { + if err := ledger.observe(testEvent{Action: "run", Package: key.Package, Test: key.Test}); err != nil { + return err + } + attributes := make([]string, 0, len(ledger.expectedAttributes[key])) + for attribute := range ledger.expectedAttributes[key] { + attributes = append(attributes, attribute) + } + sort.Strings(attributes) + for _, attribute := range attributes { + if err := ledger.observe(testEvent{Action: "attr", Package: key.Package, Test: key.Test, Key: commandcoverage.ExecutionAttributeKey, Value: attribute}); err != nil { + return err + } + } + if err := ledger.observe(testEvent{Action: "pass", Package: key.Package, Test: key.Test}); err != nil { + return err + } + } + for _, packagePath := range packages { + if err := ledger.observe(testEvent{Action: "pass", Package: packagePath}); err != nil { + return err + } + } + return nil +} + +func TestRejectReservedAttributeForgeryRejectsDirectOwnerKeyUse(t *testing.T) { + root := t.TempDir() + path := "sample_test.go" + source := `package sample + +import "testing" + +func TestForged(t *testing.T) { + t.Attr("proofkit.command-oracle", "counterfeit") +} +` + if err := os.WriteFile(filepath.Join(root, path), []byte(source), 0o600); err != nil { + t.Fatal(err) + } + if err := rejectReservedAttributeForgery(root, []string{path}); DecisionID(err) != "source.reserved_attribute_direct_use" { + t.Fatalf("rejectReservedAttributeForgery() error = %v", err) + } +} + +func TestRunGoTestsTerminatesOnContextDeadline(t *testing.T) { + root := t.TempDir() + if err := os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.test/timeout\n\ngo 1.26\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "timeout_test.go"), []byte(`package timeout + +import ( + "testing" + "time" +) + +func TestHang(t *testing.T) { time.Sleep(time.Minute) } +`), 0o600); err != nil { + t.Fatal(err) + } + candidate := syntheticCandidates()[0] + candidate.PackagePath = "." + candidate.Selector = "timeout_test.go::TestHang" + candidate.SourcePath = "timeout_test.go" + candidate.TestName = "TestHang" + ledger, err := newEventLedger([]app.CommandCoverageOracleCandidate{candidate}, map[string]string{".": "example.test/timeout"}) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(context.Background(), 250*time.Millisecond) + defer cancel() + started := time.Now() + err = runGoTests(ctx, root, []ExecutionCommand{{ + Argv: []string{"go", "test", "-json", "-count=1", "-run", "^TestHang$", "."}, + PackagePath: ".", + }}, ledger) + if DecisionID(err) != "process.timeout" { + t.Fatalf("runGoTests() error = %v, want process.timeout", err) + } + if elapsed := time.Since(started); elapsed > 10*time.Second { + t.Fatalf("runGoTests() termination took %s", elapsed) + } +} + +func TestRunGoTestCommandTerminatesImmediatelyWhenStderrExceedsBound(t *testing.T) { + bin := t.TempDir() + script := "#!/bin/sh\n/bin/dd if=/dev/zero bs=1048577 count=1 1>&2\n/bin/sleep 5\n" + if err := os.WriteFile(filepath.Join(bin, "go"), []byte(script), 0o700); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) + + started := time.Now() + err := runGoTestCommand(context.Background(), t.TempDir(), []string{"go", "test"}, nil) + if DecisionID(err) != "process.stderr_exceeded" { + t.Fatalf("runGoTestCommand() error = %v, want process.stderr_exceeded", err) + } + if elapsed := time.Since(started); elapsed > 2*time.Second { + t.Fatalf("stderr overflow termination took %s", elapsed) + } +} + +func TestExecutionCommandsKeepPackageTestSetsDisjoint(t *testing.T) { + candidates := syntheticCandidates() + candidates[0].PackagePath = "./internal/one" + candidates[0].TestName = "TestOne" + candidates[1].PackagePath = "./internal/two" + candidates[1].TestName = "TestTwo" + + commands := executionCommands(candidates) + if len(commands) != 2 { + t.Fatalf("executionCommands() count = %d, want 2", len(commands)) + } + if got := commands[0].Argv[len(commands[0].Argv)-2:]; !slices.Equal(got, []string{"^(TestOne)$", "./internal/one"}) { + t.Fatalf("first package command = %#v", commands[0].Argv) + } + if got := commands[1].Argv[len(commands[1].Argv)-2:]; !slices.Equal(got, []string{"^(TestTwo)$", "./internal/two"}) { + t.Fatalf("second package command = %#v", commands[1].Argv) + } +} + +func TestRunGoTestsDoesNotExecuteCrossPackageNameMatches(t *testing.T) { + root := t.TempDir() + if err := os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.test/exact\n\ngo 1.26\n"), 0o600); err != nil { + t.Fatal(err) + } + writePackage := func(directory string, selectedName string, selectedMarker string, forbiddenName string) { + t.Helper() + path := filepath.Join(root, directory) + if err := os.MkdirAll(path, 0o755); err != nil { + t.Fatal(err) + } + source := "package " + directory + ` + +import "testing" + +func ` + selectedName + `(t *testing.T) { + t.Attr("proofkit.command-oracle", "` + selectedMarker + `") +} + +func ` + forbiddenName + `(t *testing.T) { + t.Fatal("cross-package selector executed a non-candidate test") +} +` + if err := os.WriteFile(filepath.Join(path, directory+"_test.go"), []byte(source), 0o600); err != nil { + t.Fatal(err) + } + } + candidates := syntheticCandidates() + candidates[0].PackagePath = "./one" + candidates[0].Selector = "one/one_test.go::TestOne" + candidates[0].SourcePath = "one/one_test.go" + candidates[0].TestName = "TestOne" + candidates[1].PackagePath = "./two" + candidates[1].Selector = "two/two_test.go::TestTwo" + candidates[1].SourcePath = "two/two_test.go" + candidates[1].TestName = "TestTwo" + writePackage("one", "TestOne", candidates[0].SourceMarker, "TestTwo") + writePackage("two", "TestTwo", candidates[1].SourceMarker, "TestOne") + imports := map[string]string{ + "./one": "example.test/exact/one", + "./two": "example.test/exact/two", + } + ledger, err := newEventLedger(candidates, imports) + if err != nil { + t.Fatal(err) + } + if err := runGoTests(context.Background(), root, executionCommands(candidates), ledger); err != nil { + t.Fatalf("runGoTests() error = %v", err) + } + if err := ledger.finalize(); err != nil { + t.Fatalf("event ledger did not close: %v", err) + } +} diff --git a/internal/tools/commandoracle/model.go b/internal/tools/commandoracle/model.go new file mode 100644 index 0000000..a1f6191 --- /dev/null +++ b/internal/tools/commandoracle/model.go @@ -0,0 +1,270 @@ +package commandoracle + +import ( + "crypto/sha256" + "encoding/hex" + "path/filepath" + "reflect" + "runtime" + "sort" + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" + "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" + "github.com/research-engineering/agentic-proofkit/internal/tools/repositorysnapshot" +) + +const ( + ArtifactKind = "proofkit.command-oracle-execution.v1" + CommandID = "proofkit.command-oracle-execution" + RecordPath = "artifacts/proofkit/command-oracle-execution.json" + SchemaVersion = 1 +) + +var recordNonClaims = [...]string{ + "Command oracle execution proves only that current owner-selected Go tests reached cooperative route attributes and passed from the recorded materialized source snapshot.", + "Command oracle execution does not prove assertion-branch execution, mutation adequacy, exhaustive semantic correctness, malicious-test resistance, producer authentication, merge satisfaction, or production readiness.", +} + +func RecordNonClaims() []string { + return append([]string(nil), recordNonClaims[:]...) +} + +type JoinedEntry struct { + Candidate app.CommandCoverageOracleCandidate `json:"candidate"` + ExecutionState string `json:"executionState"` + PackageImportPath string `json:"packageImportPath"` +} + +type ExecutionCommand struct { + Argv []string `json:"argv"` + PackagePath string `json:"packagePath"` +} + +type Record struct { + ArtifactKind string `json:"artifactKind"` + CandidateSetDigest string `json:"candidateSetDigest"` + CommandID string `json:"commandId"` + CounterfeitCorpusDigest string `json:"counterfeitCorpusDigest"` + Entries []JoinedEntry `json:"entries"` + ExecutionCommands []ExecutionCommand `json:"executionCommands"` + GoVersion string `json:"goVersion"` + NonClaims []string `json:"nonClaims"` + Platform string `json:"platform"` + SchemaVersion int `json:"schemaVersion"` + SourceRevision string `json:"sourceRevision"` + SourceSnapshotDigest string `json:"sourceSnapshotDigest"` + State string `json:"state"` +} + +type Evidence struct { + Candidates []app.CommandCoverageOracleCandidate + Record Record + RecordBytes []byte + RecordDigest string +} + +func buildEvidence(snapshot repositorysnapshot.Snapshot, candidates []app.CommandCoverageOracleCandidate, packageImports map[string]string, commands []ExecutionCommand, corpusDigest string) (Evidence, error) { + orderedCandidates := append([]app.CommandCoverageOracleCandidate(nil), candidates...) + if err := validateCandidates(orderedCandidates); err != nil { + return Evidence{}, err + } + entries := make([]JoinedEntry, 0, len(orderedCandidates)) + for _, candidate := range orderedCandidates { + packageImport, ok := packageImports[candidate.PackagePath] + if !ok { + return Evidence{}, decision("join.package_import_missing") + } + entries = append(entries, JoinedEntry{ + Candidate: candidate, + ExecutionState: "passed", + PackageImportPath: packageImport, + }) + } + sort.Slice(entries, func(left, right int) bool { + return joinedIdentity(entries[left]) < joinedIdentity(entries[right]) + }) + if err := validateJoinedEntries(orderedCandidates, entries, packageImports); err != nil { + return Evidence{}, err + } + candidateDigest, err := CandidateSetDigest(orderedCandidates) + if err != nil { + return Evidence{}, err + } + record := Record{ + ArtifactKind: ArtifactKind, + CandidateSetDigest: candidateDigest, + CommandID: CommandID, + CounterfeitCorpusDigest: corpusDigest, + Entries: entries, + ExecutionCommands: cloneExecutionCommands(commands), + GoVersion: runtime.Version(), + NonClaims: RecordNonClaims(), + Platform: runtime.GOOS + "/" + runtime.GOARCH, + SchemaVersion: SchemaVersion, + SourceRevision: snapshot.Revision, + SourceSnapshotDigest: snapshot.Digest, + State: "passed", + } + if err := validateRecordShape(record); err != nil { + return Evidence{}, err + } + content, recordDigest, err := encodeRecord(record) + if err != nil { + return Evidence{}, err + } + return Evidence{ + Candidates: orderedCandidates, + Record: record, + RecordBytes: content, + RecordDigest: recordDigest, + }, nil +} + +func validateCandidates(candidates []app.CommandCoverageOracleCandidate) error { + if len(candidates) == 0 { + return decision("candidate.inventory_empty") + } + seenTestIDs := map[string]struct{}{} + seenMarkers := map[string]struct{}{} + for index, candidate := range candidates { + fields := []string{ + candidate.AssertionOracleID, + candidate.CommandRef, + candidate.ExpectedPublicOutcome, + candidate.FalsificationEventID, + candidate.NegativeCaseID, + candidate.OracleKind, + candidate.OwnerInvariantID, + candidate.PackagePath, + candidate.Selector, + candidate.SourceMarker, + candidate.SourcePath, + candidate.TestID, + candidate.TestName, + candidate.WrongImplementationClass, + } + for _, field := range fields { + if strings.TrimSpace(field) == "" { + return decision("candidate.field_empty") + } + } + if candidate.OracleKind != "semantic_route_falsifier" || + !strings.HasPrefix(candidate.PackagePath, "./") || + candidate.Selector != candidate.SourcePath+"::"+candidate.TestName || + !strings.HasSuffix(candidate.SourcePath, "_test.go") || + "./"+filepath.ToSlash(filepath.Dir(candidate.SourcePath)) != candidate.PackagePath || + !commandcoverage.ValidSourceMarker(candidate.SourceMarker) { + return decision("candidate.identity_invalid") + } + if _, duplicate := seenTestIDs[candidate.TestID]; duplicate { + return decision("candidate.test_id_duplicate") + } + if _, duplicate := seenMarkers[candidate.SourceMarker]; duplicate { + return decision("candidate.source_marker_duplicate") + } + seenTestIDs[candidate.TestID] = struct{}{} + seenMarkers[candidate.SourceMarker] = struct{}{} + if index > 0 && candidateIdentity(candidates[index-1]) >= candidateIdentity(candidate) { + return decision("candidate.order_invalid") + } + } + return nil +} + +func validateJoinedEntries(candidates []app.CommandCoverageOracleCandidate, entries []JoinedEntry, packageImports map[string]string) error { + if len(entries) != len(candidates) { + return decision("join.cardinality_mismatch") + } + for index, candidate := range candidates { + entry := entries[index] + if !reflect.DeepEqual(entry.Candidate, candidate) { + return candidateMismatchDecision(candidate, entry.Candidate) + } + if entry.ExecutionState != "passed" { + return decision("join.execution_state_invalid") + } + if entry.PackageImportPath != packageImports[candidate.PackagePath] || entry.PackageImportPath == "" { + return decision("join.package_import_mismatch") + } + if index > 0 && joinedIdentity(entries[index-1]) >= joinedIdentity(entry) { + return decision("join.order_invalid") + } + } + return nil +} + +func candidateMismatchDecision(expected, actual app.CommandCoverageOracleCandidate) error { + checks := []struct { + name string + expectedValue string + actualValue string + }{ + {"assertionOracleId", expected.AssertionOracleID, actual.AssertionOracleID}, + {"commandRef", expected.CommandRef, actual.CommandRef}, + {"expectedPublicOutcome", expected.ExpectedPublicOutcome, actual.ExpectedPublicOutcome}, + {"falsificationEventId", expected.FalsificationEventID, actual.FalsificationEventID}, + {"negativeCaseId", expected.NegativeCaseID, actual.NegativeCaseID}, + {"oracleKind", expected.OracleKind, actual.OracleKind}, + {"ownerInvariantId", expected.OwnerInvariantID, actual.OwnerInvariantID}, + {"packagePath", expected.PackagePath, actual.PackagePath}, + {"selector", expected.Selector, actual.Selector}, + {"sourceMarker", expected.SourceMarker, actual.SourceMarker}, + {"sourcePath", expected.SourcePath, actual.SourcePath}, + {"testId", expected.TestID, actual.TestID}, + {"testName", expected.TestName, actual.TestName}, + {"wrongImplementationClassId", expected.WrongImplementationClass, actual.WrongImplementationClass}, + } + for _, check := range checks { + if check.expectedValue != check.actualValue { + return decision("join.candidate_mismatch." + check.name) + } + } + return decision("join.candidate_mismatch.unknown") +} + +func candidateIdentity(candidate app.CommandCoverageOracleCandidate) string { + return strings.Join([]string{ + candidate.CommandRef, + candidate.Selector, + candidate.TestID, + candidate.OwnerInvariantID, + candidate.FalsificationEventID, + candidate.NegativeCaseID, + candidate.WrongImplementationClass, + candidate.AssertionOracleID, + candidate.OracleKind, + candidate.ExpectedPublicOutcome, + candidate.SourceMarker, + candidate.SourcePath, + candidate.PackagePath, + candidate.TestName, + }, "\x00") +} + +func CandidateSetDigest(candidates []app.CommandCoverageOracleCandidate) (string, error) { + values := make([]any, 0, len(candidates)) + for _, candidate := range candidates { + values = append(values, candidateValue(candidate)) + } + content, err := stablejson.MarshalLayout(values, stablejson.LayoutCompact) + if err != nil { + return "", err + } + digest := sha256.Sum256(content) + return hex.EncodeToString(digest[:]), nil +} + +func ExecutionCommandRefs(evidence Evidence) []string { + seen := map[string]struct{}{} + for _, candidate := range evidence.Candidates { + seen[candidate.CommandRef] = struct{}{} + } + refs := make([]string, 0, len(seen)) + for ref := range seen { + refs = append(refs, ref) + } + sort.Strings(refs) + return refs +} diff --git a/internal/tools/commandoracle/process_unix.go b/internal/tools/commandoracle/process_unix.go new file mode 100644 index 0000000..a8e1c36 --- /dev/null +++ b/internal/tools/commandoracle/process_unix.go @@ -0,0 +1,28 @@ +//go:build darwin || linux + +package commandoracle + +import ( + "errors" + "os" + "os/exec" + "syscall" +) + +func configureProcessGroup(command *exec.Cmd) { + command.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + command.Cancel = func() error { + return terminateProcessGroup(command) + } +} + +func terminateProcessGroup(command *exec.Cmd) error { + if command.Process == nil { + return nil + } + err := syscall.Kill(-command.Process.Pid, syscall.SIGKILL) + if errors.Is(err, os.ErrProcessDone) || errors.Is(err, syscall.ESRCH) { + return nil + } + return err +} diff --git a/internal/tools/commandoracle/record.go b/internal/tools/commandoracle/record.go new file mode 100644 index 0000000..027aeb7 --- /dev/null +++ b/internal/tools/commandoracle/record.go @@ -0,0 +1,95 @@ +package commandoracle + +import ( + "bytes" + "crypto/sha256" + "encoding/hex" + "encoding/json" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" + "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" + "github.com/research-engineering/agentic-proofkit/internal/tools/artifactfile" +) + +const maxRecordBytes = 32 << 20 + +func InvalidateDiagnostic(root string) error { + return artifactfile.Remove(root, RecordPath) +} + +func WriteDiagnostic(root string, evidence Evidence) error { + if err := validateRecordShape(evidence.Record); err != nil { + return err + } + canonical, digest, err := encodeRecord(evidence.Record) + if err != nil { + return err + } + if !bytes.Equal(evidence.RecordBytes, canonical) || evidence.RecordDigest != digest { + return decision("record.bytes_invalid") + } + return artifactfile.WriteAtomic(root, RecordPath, evidence.RecordBytes, 0o644) +} + +func EvidenceForRecord(record Record) (Evidence, error) { + if err := validateRecordShape(record); err != nil { + return Evidence{}, err + } + content, digest, err := encodeRecord(record) + if err != nil { + return Evidence{}, err + } + candidates := make([]app.CommandCoverageOracleCandidate, 0, len(record.Entries)) + for _, entry := range record.Entries { + candidates = append(candidates, entry.Candidate) + } + return Evidence{ + Candidates: candidates, + Record: record, + RecordBytes: content, + RecordDigest: digest, + }, nil +} + +func ReadDiagnostic(root string) (Evidence, error) { + content, err := artifactfile.ReadBounded(root, RecordPath, maxRecordBytes) + if err != nil { + return Evidence{}, err + } + if len(content) == 0 { + return Evidence{}, decision("record.resource_limit") + } + return admitRecordBytes(content) +} + +func admitRecordBytes(content []byte) (Evidence, error) { + if len(content) == 0 || len(content) > maxRecordBytes { + return Evidence{}, decision("record.resource_limit") + } + if _, err := admission.DecodeJSON(bytes.NewReader(content), maxRecordBytes); err != nil { + return Evidence{}, decision("record.json_invalid") + } + var record Record + if err := json.Unmarshal(content, &record); err != nil { + return Evidence{}, decision("record.type_invalid") + } + evidence, err := EvidenceForRecord(record) + if err != nil { + return Evidence{}, err + } + if !bytes.Equal(content, evidence.RecordBytes) { + return Evidence{}, decision("record.canonical_bytes_mismatch") + } + evidence.RecordBytes = content + return evidence, nil +} + +func encodeRecord(record Record) ([]byte, string, error) { + content, err := stablejson.Marshal(recordValue(record)) + if err != nil { + return nil, "", err + } + digest := sha256.Sum256(content) + return content, hex.EncodeToString(digest[:]), nil +} diff --git a/internal/tools/commandoracle/record_projection.go b/internal/tools/commandoracle/record_projection.go new file mode 100644 index 0000000..f0a4f85 --- /dev/null +++ b/internal/tools/commandoracle/record_projection.go @@ -0,0 +1,104 @@ +package commandoracle + +import ( + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/app" +) + +func recordValue(record Record) map[string]any { + entries := make([]any, 0, len(record.Entries)) + for _, entry := range record.Entries { + entries = append(entries, map[string]any{ + "candidate": candidateValue(entry.Candidate), + "executionState": entry.ExecutionState, + "packageImportPath": entry.PackageImportPath, + }) + } + return map[string]any{ + "artifactKind": record.ArtifactKind, + "candidateSetDigest": record.CandidateSetDigest, + "commandId": record.CommandID, + "counterfeitCorpusDigest": record.CounterfeitCorpusDigest, + "entries": entries, + "executionCommands": executionCommandsValue(record.ExecutionCommands), + "goVersion": record.GoVersion, + "nonClaims": stringsToAny(record.NonClaims), + "platform": record.Platform, + "schemaVersion": record.SchemaVersion, + "sourceRevision": record.SourceRevision, + "sourceSnapshotDigest": record.SourceSnapshotDigest, + "state": record.State, + } +} + +func candidateValue(candidate app.CommandCoverageOracleCandidate) map[string]any { + return map[string]any{ + "assertionOracleId": candidate.AssertionOracleID, + "commandRef": candidate.CommandRef, + "expectedPublicOutcome": candidate.ExpectedPublicOutcome, + "falsificationEventId": candidate.FalsificationEventID, + "negativeCaseId": candidate.NegativeCaseID, + "oracleKind": candidate.OracleKind, + "ownerInvariantId": candidate.OwnerInvariantID, + "packagePath": candidate.PackagePath, + "selector": candidate.Selector, + "sourceMarker": candidate.SourceMarker, + "sourcePath": candidate.SourcePath, + "testId": candidate.TestID, + "testName": candidate.TestName, + "wrongImplementationClassId": candidate.WrongImplementationClass, + } +} + +func joinedIdentity(entry JoinedEntry) string { + parts := []string{ + entry.Candidate.CommandRef, + entry.Candidate.Selector, + entry.Candidate.TestID, + entry.Candidate.OwnerInvariantID, + entry.Candidate.FalsificationEventID, + entry.Candidate.NegativeCaseID, + entry.Candidate.WrongImplementationClass, + entry.Candidate.AssertionOracleID, + entry.Candidate.OracleKind, + entry.Candidate.ExpectedPublicOutcome, + entry.Candidate.SourceMarker, + entry.Candidate.SourcePath, + entry.Candidate.PackagePath, + entry.Candidate.TestName, + entry.PackageImportPath, + entry.ExecutionState, + } + return strings.Join(parts, "\x00") +} + +func stringsToAny(values []string) []any { + out := make([]any, 0, len(values)) + for _, value := range values { + out = append(out, value) + } + return out +} + +func executionCommandsValue(commands []ExecutionCommand) []any { + values := make([]any, 0, len(commands)) + for _, command := range commands { + values = append(values, map[string]any{ + "argv": stringsToAny(command.Argv), + "packagePath": command.PackagePath, + }) + } + return values +} + +func cloneExecutionCommands(commands []ExecutionCommand) []ExecutionCommand { + cloned := make([]ExecutionCommand, 0, len(commands)) + for _, command := range commands { + cloned = append(cloned, ExecutionCommand{ + Argv: append([]string(nil), command.Argv...), + PackagePath: command.PackagePath, + }) + } + return cloned +} diff --git a/internal/tools/commandoracle/record_test.go b/internal/tools/commandoracle/record_test.go new file mode 100644 index 0000000..7a7685c --- /dev/null +++ b/internal/tools/commandoracle/record_test.go @@ -0,0 +1,121 @@ +package commandoracle + +import ( + "bytes" + "os" + "path/filepath" + "reflect" + "runtime" + "strings" + "testing" + + "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" + "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" +) + +func TestDiagnosticRoundTripUsesOneStrictOwner(t *testing.T) { + root := t.TempDir() + evidence := validSyntheticEvidence(t) + if err := WriteDiagnostic(root, evidence); err != nil { + t.Fatalf("WriteDiagnostic() error = %v", err) + } + read, err := ReadDiagnostic(root) + if err != nil { + t.Fatalf("ReadDiagnostic() error = %v", err) + } + if read.RecordDigest != evidence.RecordDigest || !bytes.Equal(read.RecordBytes, evidence.RecordBytes) || !reflect.DeepEqual(read.Record, evidence.Record) { + t.Fatalf("diagnostic round trip drifted: got %#v want %#v", read, evidence) + } +} + +func TestReadDiagnosticRejectsUnknownFieldAndOwnerInvalidJoin(t *testing.T) { + for _, testCase := range []struct { + name string + mutate func(map[string]any) + }{ + { + name: "unknown field", + mutate: func(record map[string]any) { + record["unexpected"] = true + }, + }, + { + name: "candidate join drift", + mutate: func(record map[string]any) { + entries := record["entries"].([]any) + entries[0].(map[string]any)["candidate"].(map[string]any)["expectedPublicOutcome"] = "Counterfeit outcome." + }, + }, + } { + t.Run(testCase.name, func(t *testing.T) { + root := t.TempDir() + evidence := validSyntheticEvidence(t) + raw, err := admission.DecodeJSON(bytes.NewReader(evidence.RecordBytes), int64(len(evidence.RecordBytes))) + if err != nil { + t.Fatal(err) + } + record := raw.(map[string]any) + testCase.mutate(record) + content, err := stablejson.Marshal(record) + if err != nil { + t.Fatal(err) + } + path := filepath.Join(root, filepath.FromSlash(RecordPath)) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, content, 0o644); err != nil { + t.Fatal(err) + } + if _, err := ReadDiagnostic(root); err == nil { + t.Fatal("ReadDiagnostic() admitted owner-invalid record") + } + }) + } +} + +func TestWriteDiagnosticRejectsBytesFromAnotherRecord(t *testing.T) { + evidence := validSyntheticEvidence(t) + evidence.RecordBytes = append([]byte(nil), evidence.RecordBytes...) + evidence.RecordBytes[0] = '[' + if err := WriteDiagnostic(t.TempDir(), evidence); DecisionID(err) != "record.bytes_invalid" { + t.Fatalf("WriteDiagnostic() error = %v, want record.bytes_invalid", err) + } +} + +func validSyntheticEvidence(t *testing.T) Evidence { + t.Helper() + candidates := syntheticCandidates() + candidateDigest, err := CandidateSetDigest(candidates) + if err != nil { + t.Fatal(err) + } + entries := make([]JoinedEntry, 0, len(candidates)) + for _, candidate := range candidates { + entries = append(entries, JoinedEntry{ + Candidate: candidate, + ExecutionState: "passed", + PackageImportPath: "example.test/proofkit/internal/sample", + }) + } + record := Record{ + ArtifactKind: ArtifactKind, + CandidateSetDigest: candidateDigest, + CommandID: CommandID, + CounterfeitCorpusDigest: strings.Repeat("2", 64), + Entries: entries, + ExecutionCommands: ExecutionCommandsForCandidates(candidates), + GoVersion: runtime.Version(), + NonClaims: RecordNonClaims(), + Platform: runtime.GOOS + "/" + runtime.GOARCH, + SchemaVersion: SchemaVersion, + SourceRevision: strings.Repeat("a", 40), + SourceSnapshotDigest: strings.Repeat("3", 64), + State: "passed", + } + evidence, err := EvidenceForRecord(record) + if err != nil { + t.Fatal(err) + } + return evidence +} diff --git a/internal/tools/commandoracle/record_validation.go b/internal/tools/commandoracle/record_validation.go new file mode 100644 index 0000000..7ac1d1d --- /dev/null +++ b/internal/tools/commandoracle/record_validation.go @@ -0,0 +1,79 @@ +package commandoracle + +import ( + "errors" + "reflect" + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/tools/repositorysnapshot" +) + +type DecisionError struct { + ID string +} + +func (err *DecisionError) Error() string { return err.ID } + +func decision(id string) error { return &DecisionError{ID: id} } + +func DecisionID(err error) string { + var typed *DecisionError + if errors.As(err, &typed) { + return typed.ID + } + return "internal_error" +} + +func validateRecordShape(record Record) error { + if record.ArtifactKind != ArtifactKind || record.CommandID != CommandID || record.SchemaVersion != SchemaVersion || record.State != "passed" { + return decision("record.identity_invalid") + } + if !isSHA256(record.CandidateSetDigest) || !isSHA256(record.CounterfeitCorpusDigest) || !isSHA256(record.SourceSnapshotDigest) { + return decision("record.digest_invalid") + } + if len(record.Entries) == 0 || len(record.ExecutionCommands) == 0 || len(record.NonClaims) != len(recordNonClaims) || + strings.TrimSpace(record.GoVersion) == "" || strings.TrimSpace(record.Platform) == "" || !repositorysnapshot.ValidRevision(record.SourceRevision) { + return decision("record.closure_invalid") + } + for index := range record.NonClaims { + if record.NonClaims[index] != recordNonClaims[index] { + return decision("record.non_claims_invalid") + } + } + candidates := make([]app.CommandCoverageOracleCandidate, 0, len(record.Entries)) + packageImports := map[string]string{} + for _, entry := range record.Entries { + candidates = append(candidates, entry.Candidate) + if existing, exists := packageImports[entry.Candidate.PackagePath]; exists && existing != entry.PackageImportPath { + return decision("record.package_import_conflict") + } + packageImports[entry.Candidate.PackagePath] = entry.PackageImportPath + } + if err := validateCandidates(candidates); err != nil { + return err + } + if err := validateJoinedEntries(candidates, record.Entries, packageImports); err != nil { + return err + } + candidateDigest, err := CandidateSetDigest(candidates) + if err != nil || candidateDigest != record.CandidateSetDigest { + return decision("record.candidate_set_digest_mismatch") + } + if !reflect.DeepEqual(record.ExecutionCommands, executionCommands(candidates)) { + return decision("record.execution_commands_mismatch") + } + return nil +} + +func isSHA256(value string) bool { + if len(value) != 64 { + return false + } + for _, character := range value { + if (character < '0' || character > '9') && (character < 'a' || character > 'f') { + return false + } + } + return true +} diff --git a/internal/tools/commandoracle/testdata/counterfeit-corpus.v1.json b/internal/tools/commandoracle/testdata/counterfeit-corpus.v1.json new file mode 100644 index 0000000..94f331a --- /dev/null +++ b/internal/tools/commandoracle/testdata/counterfeit-corpus.v1.json @@ -0,0 +1,58 @@ +{ + "corpusId": "proofkit.command-oracle.counterfeit-corpus.v1", + "schemaVersion": 1, + "cases": [ + {"caseId":"positive.candidate","coordinate":"","evidenceClass":"candidate","expectedDecision":"admit","mutationId":"positive-candidate","policyId":"policy.positive.candidate"}, + {"caseId":"positive.execution-shared-test","coordinate":"","evidenceClass":"execution","expectedDecision":"admit","mutationId":"positive-execution-shared-test","policyId":"policy.positive.execution.shared.test"}, + {"caseId":"positive.joined","coordinate":"","evidenceClass":"joined","expectedDecision":"admit","mutationId":"positive-joined","policyId":"policy.positive.joined"}, + {"caseId":"reject.event.attribute-cross-test","coordinate":"","evidenceClass":"execution","expectedDecision":"event.reserved_attribute_unknown_test","mutationId":"event-attribute-cross-test","policyId":"policy.reject.event.attribute.cross.test"}, + {"caseId":"reject.event.attribute-duplicate","coordinate":"","evidenceClass":"execution","expectedDecision":"event.reserved_attribute_duplicate","mutationId":"event-attribute-duplicate","policyId":"policy.reject.event.attribute.duplicate"}, + {"caseId":"reject.event.attribute-missing","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_pass_missing_attributes","mutationId":"event-attribute-missing","policyId":"policy.reject.event.attribute.missing"}, + {"caseId":"reject.event.descendant-skip","coordinate":"","evidenceClass":"execution","expectedDecision":"event.selected_descendant_failed_or_skipped","mutationId":"event-descendant-skip","policyId":"policy.reject.event.descendant.skip"}, + {"caseId":"reject.event.output-spoof","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_pass_missing_attributes","mutationId":"event-output-spoof","policyId":"policy.reject.event.output.spoof"}, + {"caseId":"reject.event.package-pass-before-tests","coordinate":"","evidenceClass":"execution","expectedDecision":"event.package_pass_before_tests","mutationId":"event-package-pass-before-tests","policyId":"policy.reject.event.package.pass.before.tests"}, + {"caseId":"reject.event.package-pass-missing","coordinate":"","evidenceClass":"execution","expectedDecision":"event.package_pass_missing","mutationId":"event-package-pass-missing","policyId":"policy.reject.event.package.pass.missing"}, + {"caseId":"reject.event.pass-before-run","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_pass_wrong_order","mutationId":"event-pass-before-run","policyId":"policy.reject.event.pass.before.run"}, + {"caseId":"reject.event.pass-duplicate","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_pass_wrong_order","mutationId":"event-pass-duplicate","policyId":"policy.reject.event.pass.duplicate"}, + {"caseId":"reject.event.pause-before-run","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_pause_wrong_order","mutationId":"event-pause-before-run","policyId":"policy.reject.event.pause.before.run"}, + {"caseId":"reject.event.pause-duplicate","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_pause_wrong_order","mutationId":"event-pause-duplicate","policyId":"policy.reject.event.pause.duplicate"}, + {"caseId":"reject.event.run-duplicate","coordinate":"","evidenceClass":"execution","expectedDecision":"event.test_run_duplicate","mutationId":"event-run-duplicate","policyId":"policy.reject.event.run.duplicate"}, + {"caseId":"reject.event.selected-fail","coordinate":"","evidenceClass":"execution","expectedDecision":"event.selected_test_failed_or_skipped","mutationId":"event-selected-fail","policyId":"policy.reject.event.selected.fail"}, + {"caseId":"reject.event.selected-skip","coordinate":"","evidenceClass":"execution","expectedDecision":"event.selected_test_failed_or_skipped","mutationId":"event-selected-skip","policyId":"policy.reject.event.selected.skip"}, + {"caseId":"reject.event.unknown-action","coordinate":"","evidenceClass":"execution","expectedDecision":"event.action_unknown","mutationId":"event-unknown-action","policyId":"policy.reject.event.unknown.action"}, + {"caseId":"reject.join.correlated-command-identity","coordinate":"","evidenceClass":"joined","expectedDecision":"join.candidate_mismatch.commandRef","mutationId":"join-correlated-command-identity","policyId":"policy.reject.join.correlated.command.identity"}, + {"caseId":"reject.join.correlated-outcome-marker","coordinate":"","evidenceClass":"joined","expectedDecision":"join.candidate_mismatch.expectedPublicOutcome","mutationId":"join-correlated-outcome-marker","policyId":"policy.reject.join.correlated.outcome.marker"}, + {"caseId":"reject.join.correlated-selector-test","coordinate":"","evidenceClass":"joined","expectedDecision":"join.candidate_mismatch.selector","mutationId":"join-correlated-selector-test","policyId":"policy.reject.join.correlated.selector.test"}, + {"caseId":"reject.record.execution-command-drift","coordinate":"","evidenceClass":"joined","expectedDecision":"record.execution_commands_mismatch","mutationId":"record-execution-command-drift","policyId":"policy.reject.record.execution.command.drift"}, + {"caseId":"reject.schema.artifactKind","coordinate":"record.artifactKind","evidenceClass":"joined","expectedDecision":"record.identity_invalid","mutationId":"record-coordinate:record.artifactKind","policyId":"schema.coordinate.artifactKind"}, + {"caseId":"reject.schema.candidateSetDigest","coordinate":"record.candidateSetDigest","evidenceClass":"joined","expectedDecision":"record.digest_invalid","mutationId":"record-coordinate:record.candidateSetDigest","policyId":"schema.coordinate.candidateSetDigest"}, + {"caseId":"reject.schema.commandId","coordinate":"record.commandId","evidenceClass":"joined","expectedDecision":"record.identity_invalid","mutationId":"record-coordinate:record.commandId","policyId":"schema.coordinate.commandId"}, + {"caseId":"reject.schema.counterfeitCorpusDigest","coordinate":"record.counterfeitCorpusDigest","evidenceClass":"joined","expectedDecision":"record.digest_invalid","mutationId":"record-coordinate:record.counterfeitCorpusDigest","policyId":"schema.coordinate.counterfeitCorpusDigest"}, + {"caseId":"reject.schema.entries-candidate-assertionOracleId","coordinate":"record.entries[].candidate.assertionOracleId","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.assertionOracleId","policyId":"schema.coordinate.entries-candidate-assertionOracleId"}, + {"caseId":"reject.schema.entries-candidate-commandRef","coordinate":"record.entries[].candidate.commandRef","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.commandRef","policyId":"schema.coordinate.entries-candidate-commandRef"}, + {"caseId":"reject.schema.entries-candidate-expectedPublicOutcome","coordinate":"record.entries[].candidate.expectedPublicOutcome","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.expectedPublicOutcome","policyId":"schema.coordinate.entries-candidate-expectedPublicOutcome"}, + {"caseId":"reject.schema.entries-candidate-falsificationEventId","coordinate":"record.entries[].candidate.falsificationEventId","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.falsificationEventId","policyId":"schema.coordinate.entries-candidate-falsificationEventId"}, + {"caseId":"reject.schema.entries-candidate-negativeCaseId","coordinate":"record.entries[].candidate.negativeCaseId","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.negativeCaseId","policyId":"schema.coordinate.entries-candidate-negativeCaseId"}, + {"caseId":"reject.schema.entries-candidate-oracleKind","coordinate":"record.entries[].candidate.oracleKind","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.oracleKind","policyId":"schema.coordinate.entries-candidate-oracleKind"}, + {"caseId":"reject.schema.entries-candidate-ownerInvariantId","coordinate":"record.entries[].candidate.ownerInvariantId","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.ownerInvariantId","policyId":"schema.coordinate.entries-candidate-ownerInvariantId"}, + {"caseId":"reject.schema.entries-candidate-packagePath","coordinate":"record.entries[].candidate.packagePath","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.packagePath","policyId":"schema.coordinate.entries-candidate-packagePath"}, + {"caseId":"reject.schema.entries-candidate-selector","coordinate":"record.entries[].candidate.selector","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.selector","policyId":"schema.coordinate.entries-candidate-selector"}, + {"caseId":"reject.schema.entries-candidate-sourceMarker","coordinate":"record.entries[].candidate.sourceMarker","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.sourceMarker","policyId":"schema.coordinate.entries-candidate-sourceMarker"}, + {"caseId":"reject.schema.entries-candidate-sourcePath","coordinate":"record.entries[].candidate.sourcePath","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.sourcePath","policyId":"schema.coordinate.entries-candidate-sourcePath"}, + {"caseId":"reject.schema.entries-candidate-testId","coordinate":"record.entries[].candidate.testId","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.testId","policyId":"schema.coordinate.entries-candidate-testId"}, + {"caseId":"reject.schema.entries-candidate-testName","coordinate":"record.entries[].candidate.testName","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.testName","policyId":"schema.coordinate.entries-candidate-testName"}, + {"caseId":"reject.schema.entries-candidate-wrongImplementationClassId","coordinate":"record.entries[].candidate.wrongImplementationClassId","evidenceClass":"joined","expectedDecision":"candidate.field_empty","mutationId":"record-coordinate:record.entries[].candidate.wrongImplementationClassId","policyId":"schema.coordinate.entries-candidate-wrongImplementationClassId"}, + {"caseId":"reject.schema.entries-executionState","coordinate":"record.entries[].executionState","evidenceClass":"joined","expectedDecision":"join.execution_state_invalid","mutationId":"record-coordinate:record.entries[].executionState","policyId":"schema.coordinate.entries-executionState"}, + {"caseId":"reject.schema.entries-packageImportPath","coordinate":"record.entries[].packageImportPath","evidenceClass":"joined","expectedDecision":"record.package_import_conflict","mutationId":"record-coordinate:record.entries[].packageImportPath","policyId":"schema.coordinate.entries-packageImportPath"}, + {"caseId":"reject.schema.executionCommands-item-argv-item","coordinate":"record.executionCommands[].argv[]","evidenceClass":"joined","expectedDecision":"record.execution_commands_mismatch","mutationId":"record-coordinate:record.executionCommands[].argv[]","policyId":"schema.coordinate.executionCommands-item-argv-item"}, + {"caseId":"reject.schema.executionCommands-item-packagePath","coordinate":"record.executionCommands[].packagePath","evidenceClass":"joined","expectedDecision":"record.execution_commands_mismatch","mutationId":"record-coordinate:record.executionCommands[].packagePath","policyId":"schema.coordinate.executionCommands-item-packagePath"}, + {"caseId":"reject.schema.goVersion","coordinate":"record.goVersion","evidenceClass":"joined","expectedDecision":"record.closure_invalid","mutationId":"record-coordinate:record.goVersion","policyId":"schema.coordinate.goVersion"}, + {"caseId":"reject.schema.nonClaims-item","coordinate":"record.nonClaims[]","evidenceClass":"joined","expectedDecision":"record.non_claims_invalid","mutationId":"record-coordinate:record.nonClaims[]","policyId":"schema.coordinate.nonClaims-item"}, + {"caseId":"reject.schema.platform","coordinate":"record.platform","evidenceClass":"joined","expectedDecision":"record.closure_invalid","mutationId":"record-coordinate:record.platform","policyId":"schema.coordinate.platform"}, + {"caseId":"reject.schema.schemaVersion","coordinate":"record.schemaVersion","evidenceClass":"joined","expectedDecision":"record.identity_invalid","mutationId":"record-coordinate:record.schemaVersion","policyId":"schema.coordinate.schemaVersion"}, + {"caseId":"reject.schema.sourceRevision","coordinate":"record.sourceRevision","evidenceClass":"joined","expectedDecision":"record.closure_invalid","mutationId":"record-coordinate:record.sourceRevision","policyId":"schema.coordinate.sourceRevision"}, + {"caseId":"reject.schema.sourceSnapshotDigest","coordinate":"record.sourceSnapshotDigest","evidenceClass":"joined","expectedDecision":"record.digest_invalid","mutationId":"record-coordinate:record.sourceSnapshotDigest","policyId":"schema.coordinate.sourceSnapshotDigest"}, + {"caseId":"reject.schema.state","coordinate":"record.state","evidenceClass":"joined","expectedDecision":"record.identity_invalid","mutationId":"record-coordinate:record.state","policyId":"schema.coordinate.state"}, + {"caseId":"reject.source.correlated-identity","coordinate":"","evidenceClass":"joined","expectedDecision":"source.current_snapshot_mismatch","mutationId":"source-correlated-identity","policyId":"policy.reject.source.correlated.identity"} + ] +} diff --git a/internal/tools/coveragemetrics/command_routes.go b/internal/tools/coveragemetrics/command_routes.go new file mode 100644 index 0000000..55430a0 --- /dev/null +++ b/internal/tools/coveragemetrics/command_routes.go @@ -0,0 +1,169 @@ +package main + +import ( + "fmt" + "sort" + + "github.com/research-engineering/agentic-proofkit/internal/app" + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" +) + +func buildCommandRouteMetrics(contract cliContract, summaries []app.CommandCoverageSummary, inventory testevidenceinventory.Inventory) commandRouteMetrics { + return buildCommandRouteMetricsWithExecution(contract, summaries, inventory, commandExecutionSummary{}) +} + +func buildCommandRouteMetricsWithExecution(contract cliContract, summaries []app.CommandCoverageSummary, inventory testevidenceinventory.Inventory, execution commandExecutionSummary) commandRouteMetrics { + missingCandidates := []string{} + missingDeclaredSemanticRoutes := []string{} + missingExecutionBackedSemanticRoutes := []string{} + contractRefs := map[string]string{} + knownRefs := map[string]struct{}{} + candidateRefs := map[string]struct{}{} + declaredSemanticRouteRefs := map[string]struct{}{} + executionBackedSemanticRouteRefs := map[string]struct{}{} + routeOnlyCount := 0 + candidateEntryCount := 0 + declaredSemanticRouteEntryCount := 0 + for _, command := range contract.Commands { + contractRefs[app.CommandCoverageCommandRef(command.Command)] = command.Command + } + for _, summary := range summaries { + knownRefs[summary.CommandRef] = struct{}{} + } + for _, commandRef := range execution.CommandRefs { + executionBackedSemanticRouteRefs[commandRef] = struct{}{} + } + for _, entry := range inventory.Entries { + switch entry.EvidenceClass { + case testevidenceinventory.EvidenceClassDeclaredSemanticFalsifierRoute: + declaredSemanticRouteEntryCount++ + for _, commandRef := range entry.CommandRefs { + declaredSemanticRouteRefs[commandRef] = struct{}{} + } + case testevidenceinventory.EvidenceClassProofRouteCandidate: + candidateEntryCount++ + for _, commandRef := range entry.CommandRefs { + candidateRefs[commandRef] = struct{}{} + } + case "routing_smoke_nonclaim": + routeOnlyCount++ + } + } + unknownDeclaredSemanticRouteRefs := unknownCommandRefs(declaredSemanticRouteRefs, knownRefs) + unknownCandidateRefs := unknownCommandRefs(candidateRefs, knownRefs) + unknownExecutionBackedSemanticRouteRefs := unknownCommandRefs(executionBackedSemanticRouteRefs, knownRefs) + contractOnly := []string{} + for ref, command := range contractRefs { + if _, ok := knownRefs[ref]; !ok { + contractOnly = append(contractOnly, command) + } + } + routeOnly := []string{} + for _, summary := range summaries { + if _, ok := contractRefs[summary.CommandRef]; !ok { + routeOnly = append(routeOnly, summary.Command) + } + } + sort.Strings(contractOnly) + sort.Strings(routeOnly) + out := commandRouteMetrics{ + AdmittedInventoryEntryCount: len(inventory.Entries), + CommandCount: len(summaries), + ContractOnlyCommands: contractOnly, + ContractOnlyCommandCount: len(contractOnly), + RouteOnlyCommands: routeOnly, + RouteOnlyCommandCount: len(routeOnly), + RouteSmokeCount: routeOnlyCount, + ProofRouteCandidateInventoryEntryCount: candidateEntryCount, + DeclaredSemanticFalsifierRouteEntryCount: declaredSemanticRouteEntryCount, + UnknownProofRouteCandidateRefs: unknownCandidateRefs, + UnknownProofRouteCandidateRefCount: len(unknownCandidateRefs), + UnknownDeclaredSemanticRouteCommandRefs: unknownDeclaredSemanticRouteRefs, + UnknownDeclaredSemanticRouteCommandRefCount: len(unknownDeclaredSemanticRouteRefs), + CommandOracleCandidateSetDigest: execution.CandidateSetDigest, + CommandOracleCounterfeitCorpusDigest: execution.CounterfeitCorpusDigest, + CommandOracleRecordDigest: execution.RecordDigest, + CommandOracleSourceSnapshotDigest: execution.SourceSnapshotDigest, + ExecutionBackedSemanticRouteEntryCount: execution.CandidateCount, + UnknownExecutionBackedSemanticRouteCommandRefs: unknownExecutionBackedSemanticRouteRefs, + UnknownExecutionBackedSemanticRouteCommandRefCount: len(unknownExecutionBackedSemanticRouteRefs), + } + for _, summary := range summaries { + out.Commands = append(out.Commands, summary.Command) + out.RouteCount += summary.RouteCount + out.ProofRouteCandidateRouteCount += summary.ProofRouteCandidateCount + if _, ok := candidateRefs[summary.CommandRef]; !ok { + missingCandidates = append(missingCandidates, summary.Command) + } + if _, ok := declaredSemanticRouteRefs[summary.CommandRef]; !ok { + missingDeclaredSemanticRoutes = append(missingDeclaredSemanticRoutes, summary.Command) + } + if _, ok := executionBackedSemanticRouteRefs[summary.CommandRef]; !ok { + missingExecutionBackedSemanticRoutes = append(missingExecutionBackedSemanticRoutes, summary.Command) + } + } + sort.Strings(out.Commands) + sort.Strings(missingCandidates) + sort.Strings(missingDeclaredSemanticRoutes) + sort.Strings(missingExecutionBackedSemanticRoutes) + out.CommandsWithoutProofRouteCandidate = missingCandidates + out.CommandWithoutProofRouteCandidateCount = len(missingCandidates) + out.CommandsWithoutDeclaredSemanticFalsifierRoute = missingDeclaredSemanticRoutes + out.CommandWithoutDeclaredSemanticFalsifierRouteCount = len(missingDeclaredSemanticRoutes) + out.CommandsWithoutExecutionBackedSemanticRoute = missingExecutionBackedSemanticRoutes + out.CommandWithoutExecutionBackedSemanticRouteCount = len(missingExecutionBackedSemanticRoutes) + return out +} + +func unknownCommandRefs(refs, known map[string]struct{}) []string { + unknown := []string{} + for ref := range refs { + if _, ok := known[ref]; !ok { + unknown = append(unknown, ref) + } + } + sort.Strings(unknown) + return unknown +} + +func requireCommandRouteInventoryClosure(metrics commandRouteMetrics) error { + digestsValid := isSHA256Text(metrics.CommandOracleCandidateSetDigest) && + isSHA256Text(metrics.CommandOracleCounterfeitCorpusDigest) && + isSHA256Text(metrics.CommandOracleRecordDigest) && + isSHA256Text(metrics.CommandOracleSourceSnapshotDigest) + if len(metrics.CommandsWithoutProofRouteCandidate) == 0 && + len(metrics.UnknownProofRouteCandidateRefs) == 0 && + len(metrics.UnknownDeclaredSemanticRouteCommandRefs) == 0 && + len(metrics.CommandsWithoutExecutionBackedSemanticRoute) == 0 && + len(metrics.UnknownExecutionBackedSemanticRouteCommandRefs) == 0 && + len(metrics.ContractOnlyCommands) == 0 && + len(metrics.RouteOnlyCommands) == 0 && + metrics.ExecutionBackedSemanticRouteEntryCount == metrics.ProofRouteCandidateInventoryEntryCount && + digestsValid { + return nil + } + return fmt.Errorf("command proof-route inventory defects: missingCandidates=%v unknownCandidateRefs=%v unknownDeclaredSemanticRouteRefs=%v missingExecutionBackedRoutes=%v unknownExecutionBackedRefs=%v executionEntries=%d candidateEntries=%d commandOracleDigestsValid=%t contractOnly=%v routeOnly=%v", + metrics.CommandsWithoutProofRouteCandidate, + metrics.UnknownProofRouteCandidateRefs, + metrics.UnknownDeclaredSemanticRouteCommandRefs, + metrics.CommandsWithoutExecutionBackedSemanticRoute, + metrics.UnknownExecutionBackedSemanticRouteCommandRefs, + metrics.ExecutionBackedSemanticRouteEntryCount, + metrics.ProofRouteCandidateInventoryEntryCount, + digestsValid, + metrics.ContractOnlyCommands, + metrics.RouteOnlyCommands, + ) +} + +func isSHA256Text(value string) bool { + if len(value) != 64 { + return false + } + for _, character := range value { + if (character < '0' || character > '9') && (character < 'a' || character > 'f') { + return false + } + } + return true +} diff --git a/internal/tools/coveragemetrics/main.go b/internal/tools/coveragemetrics/main.go index baa1dd4..708e935 100644 --- a/internal/tools/coveragemetrics/main.go +++ b/internal/tools/coveragemetrics/main.go @@ -1,6 +1,7 @@ package main import ( + "context" "encoding/json" "errors" "fmt" @@ -22,12 +23,18 @@ import ( "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" "github.com/research-engineering/agentic-proofkit/internal/kernel/gotestsource" + "github.com/research-engineering/agentic-proofkit/internal/tools/artifactfile" + "github.com/research-engineering/agentic-proofkit/internal/tools/commandoracle" "github.com/research-engineering/agentic-proofkit/internal/tools/packageartifactrecord" ) const outputPath = "artifacts/proofkit/coverage-metrics.json" var commandCoverageInventoryInput = app.CommandCoverageInventory +var commandOracleExecute = commandoracle.Execute +var commandOracleInvalidateDiagnostic = commandoracle.InvalidateDiagnostic +var commandOracleValidateCurrent = commandoracle.ValidateCurrent +var commandOracleWriteDiagnostic = commandoracle.WriteDiagnostic type requirementSource struct { Requirements []requirementRecord `json:"requirements"` @@ -125,26 +132,44 @@ type cliContractMetrics struct { } type commandRouteMetrics struct { - AdmittedInventoryEntryCount int `json:"admittedInventoryEntryCount"` - CommandCount int `json:"commandCount"` - Commands []string `json:"commands"` - CommandWithoutProofRouteCandidateCount int `json:"commandWithoutProofRouteCandidateCount"` - CommandsWithoutProofRouteCandidate []string `json:"commandsWithoutProofRouteCandidate"` - ContractOnlyCommandCount int `json:"contractOnlyCommandCount"` - ContractOnlyCommands []string `json:"contractOnlyCommands"` - CommandWithoutDeclaredSemanticFalsifierRouteCount int `json:"commandWithoutDeclaredSemanticFalsifierRouteCount"` - CommandsWithoutDeclaredSemanticFalsifierRoute []string `json:"commandsWithoutDeclaredSemanticFalsifierRoute"` - RouteCount int `json:"routeCount"` - RouteOnlyCommandCount int `json:"routeOnlyCommandCount"` - RouteOnlyCommands []string `json:"routeOnlyCommands"` - RouteSmokeCount int `json:"routeSmokeCount"` - ProofRouteCandidateInventoryEntryCount int `json:"proofRouteCandidateInventoryEntryCount"` - ProofRouteCandidateRouteCount int `json:"proofRouteCandidateRouteCount"` - DeclaredSemanticFalsifierRouteEntryCount int `json:"declaredSemanticFalsifierRouteEntryCount"` - UnknownProofRouteCandidateRefs []string `json:"unknownProofRouteCandidateRefs"` - UnknownProofRouteCandidateRefCount int `json:"unknownProofRouteCandidateRefCount"` - UnknownDeclaredSemanticRouteCommandRefs []string `json:"unknownDeclaredSemanticRouteCommandRefs"` - UnknownDeclaredSemanticRouteCommandRefCount int `json:"unknownDeclaredSemanticRouteCommandRefCount"` + AdmittedInventoryEntryCount int `json:"admittedInventoryEntryCount"` + CommandCount int `json:"commandCount"` + Commands []string `json:"commands"` + CommandWithoutProofRouteCandidateCount int `json:"commandWithoutProofRouteCandidateCount"` + CommandsWithoutProofRouteCandidate []string `json:"commandsWithoutProofRouteCandidate"` + ContractOnlyCommandCount int `json:"contractOnlyCommandCount"` + ContractOnlyCommands []string `json:"contractOnlyCommands"` + CommandWithoutDeclaredSemanticFalsifierRouteCount int `json:"commandWithoutDeclaredSemanticFalsifierRouteCount"` + CommandsWithoutDeclaredSemanticFalsifierRoute []string `json:"commandsWithoutDeclaredSemanticFalsifierRoute"` + RouteCount int `json:"routeCount"` + RouteOnlyCommandCount int `json:"routeOnlyCommandCount"` + RouteOnlyCommands []string `json:"routeOnlyCommands"` + RouteSmokeCount int `json:"routeSmokeCount"` + ProofRouteCandidateInventoryEntryCount int `json:"proofRouteCandidateInventoryEntryCount"` + ProofRouteCandidateRouteCount int `json:"proofRouteCandidateRouteCount"` + DeclaredSemanticFalsifierRouteEntryCount int `json:"declaredSemanticFalsifierRouteEntryCount"` + UnknownProofRouteCandidateRefs []string `json:"unknownProofRouteCandidateRefs"` + UnknownProofRouteCandidateRefCount int `json:"unknownProofRouteCandidateRefCount"` + UnknownDeclaredSemanticRouteCommandRefs []string `json:"unknownDeclaredSemanticRouteCommandRefs"` + UnknownDeclaredSemanticRouteCommandRefCount int `json:"unknownDeclaredSemanticRouteCommandRefCount"` + CommandOracleCandidateSetDigest string `json:"commandOracleCandidateSetDigest"` + CommandOracleCounterfeitCorpusDigest string `json:"commandOracleCounterfeitCorpusDigest"` + CommandOracleRecordDigest string `json:"commandOracleRecordDigest"` + CommandOracleSourceSnapshotDigest string `json:"commandOracleSourceSnapshotDigest"` + CommandWithoutExecutionBackedSemanticRouteCount int `json:"commandWithoutExecutionBackedSemanticRouteCount"` + CommandsWithoutExecutionBackedSemanticRoute []string `json:"commandsWithoutExecutionBackedSemanticRoute"` + ExecutionBackedSemanticRouteEntryCount int `json:"executionBackedSemanticRouteEntryCount"` + UnknownExecutionBackedSemanticRouteCommandRefCount int `json:"unknownExecutionBackedSemanticRouteCommandRefCount"` + UnknownExecutionBackedSemanticRouteCommandRefs []string `json:"unknownExecutionBackedSemanticRouteCommandRefs"` +} + +type commandExecutionSummary struct { + CandidateCount int + CandidateSetDigest string + CommandRefs []string + CounterfeitCorpusDigest string + RecordDigest string + SourceSnapshotDigest string } type deadZoneMetrics struct { @@ -162,6 +187,9 @@ func main() { } func run() error { + if err := invalidateExecutionMetrics(); err != nil { + return err + } requirements, err := readRequirements() if err != nil { return err @@ -183,16 +211,36 @@ func run() error { out := buildMetrics(requirements, bindings, witnesses, contract, testevidenceinventory.Inventory{}) return writeMetrics(out, err) } - out := buildMetrics(requirements, bindings, witnesses, contract, commandInventory) - if err := bindCurrentSourceProvenance(&out); err != nil { + executionEvidence, err := commandOracleExecute(context.Background(), ".") + if err != nil { + out := buildMetrics(requirements, bindings, witnesses, contract, commandInventory) + if provenanceErr := bindCurrentSourceProvenance(&out); provenanceErr != nil { + err = errors.Join(err, provenanceErr) + } return writeMetrics(out, err) } + out := buildMetricsWithExecution(requirements, bindings, witnesses, contract, commandInventory, commandExecutionSummaryFromEvidence(executionEvidence)) + bindCommandOracleProvenance(&out, executionEvidence) closeoutErr := errors.Join( requireCommandRouteInventoryClosure(out.CommandRoutes), requireNoLinkageDeadZones(out.DeadZones), validateBindingWitnessSelectorsAtRoot(".", bindings), ) - return writeMetrics(out, closeoutErr) + if closeoutErr != nil { + return writeMetrics(out, closeoutErr) + } + return writeCurrentExecutionMetrics(context.Background(), out, executionEvidence) +} + +func commandExecutionSummaryFromEvidence(evidence commandoracle.Evidence) commandExecutionSummary { + return commandExecutionSummary{ + CandidateCount: len(evidence.Candidates), + CandidateSetDigest: evidence.Record.CandidateSetDigest, + CommandRefs: commandoracle.ExecutionCommandRefs(evidence), + CounterfeitCorpusDigest: evidence.Record.CounterfeitCorpusDigest, + RecordDigest: evidence.RecordDigest, + SourceSnapshotDigest: evidence.Record.SourceSnapshotDigest, + } } func validateBindingWitnessSelectorsAtRoot(root string, bindings bindingFile) error { @@ -272,7 +320,40 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { "TestScorecardPublicPublishDeclaresRequiredOutputInputs", "TestSecurityScannerWorkflowsSeparateProviderPublicationPermissions", }, - {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.coverage-metrics"}: {"TestEachCommandRouteClosureConjunctHasIndependentFalsifier", "TestEachLinkageDeadZoneConjunctHasIndependentFalsifier"}, + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.artifact-file-boundary"}: { + "TestOperationsRejectFinalSymlinkWithoutTargetMutation", + "TestOperationsRejectSymlinkComponentsWithoutOutsideMutation", + "TestReadBoundedRejectsUnrepresentableLimit", + "TestWriteReadAndRemoveRoundTrip", + }, + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.coverage-metrics"}: { + "TestEachCommandRouteClosureConjunctHasIndependentFalsifier", + "TestEachLinkageDeadZoneConjunctHasIndependentFalsifier", + "TestInvalidateMetricsFileRejectsSymlinkParentWithoutDeletingOutsideFile", + "TestWriteMetricsFileRejectsSymlinkEscapeWithoutMutation", + }, + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.command-oracle-execution-ledger"}: { + "TestExecuteBindsMaterializedSourceCandidatesAndRuntimeEvents", + "TestRunGoTestCommandTerminatesImmediatelyWhenStderrExceedsBound", + "TestRunGoTestsDoesNotExecuteCrossPackageNameMatches", + "TestRunGoTestsTerminatesOnContextDeadline", + "TestValidateCurrentRejectsProducerUnreachableCandidateProjection", + }, + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.command-oracle-counterfeit-corpus"}: { + "TestCounterfeitCorpusClosesRequiredAxes", + "TestCounterfeitCorpusClosureRejectsMissingRequiredAxes", + "TestEachCounterfeitCaseProducesItsCheckedInDecision", + }, + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.command-oracle-source-snapshot"}: { + "TestCaptureContextTerminatesCanceledGitProcessGroup", + "TestCaptureContextTerminatesGitProcessGroupOnOutputOverflow", + "TestCaptureRejectsSuccessfulGitDiagnosticsWithoutEcho", + "TestMaterializeBindsCopiedBytesAndRejectsLiveMutation", + "TestMaterializeRejectsSymlinkAndNonEmptyDestination", + "TestMaterializeRejectsSymlinkedDestinationInsideSource", + "TestValidRevisionAdmitsOnlyGitObjectIdentityAndOptionalSnapshotDigest", + "TestValidateMaterializedRejectsSurplusFile", + }, {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.binding-selector-executability"}: { "TestBindingWitnessSelectorsAcceptUnnamedGoTestParameter", "TestBindingWitnessSelectorsRejectInvalidGoTestSignature", @@ -341,6 +422,8 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { }, {"REQ-PROOFKIT-QUALITY-015", "proofkit.supply-chain-quality.release-closeout-completion-criteria"}: { "TestBuildInputFailsClosedForEachBlockingEvidenceClass", + "TestSelfEvidenceInvokesCurrentCommandOracleOwner", + "TestSelfEvidenceRejectsProducerUnreachableCommandOracleRef", }, {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.release-change-record-projection"}: { "TestAdmitEnforcesVersionedChangeClass", @@ -427,8 +510,12 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { {"REQ-PROOFKIT-QUALITY-005", "proofkit.supply-chain-quality.codeql-permission-separation"}: "scripts/workflow_security_scanner_oracles_test.go", {"REQ-PROOFKIT-QUALITY-006", "proofkit.supply-chain-quality.osv-permission-separation"}: "scripts/workflow_security_scanner_oracles_test.go", {"REQ-PROOFKIT-QUALITY-007", "proofkit.supply-chain-quality.scorecard-permission-and-publication-inputs"}: "scripts/workflow_security_scanner_oracles_test.go", + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.artifact-file-boundary"}: "internal/tools/artifactfile/file_test.go", {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.binding-selector-executability"}: "internal/tools/coveragemetrics/main_test.go", {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.coverage-metrics"}: "internal/tools/coveragemetrics/main_test.go", + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.command-oracle-execution-ledger"}: "internal/tools/commandoracle/execute_test.go", + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.command-oracle-counterfeit-corpus"}: "internal/tools/commandoracle/corpus_test.go", + {"REQ-PROOFKIT-QUALITY-010", "proofkit.supply-chain-quality.command-oracle-source-snapshot"}: "internal/tools/repositorysnapshot/snapshot_test.go", {"REQ-PROOFKIT-QUALITY-011", "proofkit.supply-chain-quality.ci-required-aggregate-exactness"}: "scripts/workflow_package_gate_oracle_test.go", {"REQ-PROOFKIT-QUALITY-013", "proofkit.supply-chain-quality.workflow-package-gate-oracle"}: "scripts/workflow_package_gate_oracle_test.go", {"REQ-PROOFKIT-QUALITY-016", "proofkit.supply-chain-quality.release-platform-python-wheels"}: "internal/tools/pythonpackage/metadata_test.go", @@ -698,27 +785,70 @@ func bindCurrentSourceProvenance(out *metrics) error { return nil } +func bindCommandOracleProvenance(out *metrics, evidence commandoracle.Evidence) { + out.Provenance = coverageProvenance{ + GeneratedAt: time.Now().UTC().Format(time.RFC3339Nano), + ProducerCommandID: "proofkit.coverage-metrics", + SourceRevision: evidence.Record.SourceRevision, + SourceSnapshotDigest: evidence.Record.SourceSnapshotDigest, + } +} + func writeMetrics(out metrics, routeErr error) error { - if err := os.MkdirAll(filepath.Dir(outputPath), 0o755); err != nil { + if err := writeMetricsFile(out); err != nil { return err } + if routeErr != nil { + return routeErr + } + printMetricsSummary(out) + return nil +} + +func writeCurrentExecutionMetrics(ctx context.Context, out metrics, evidence commandoracle.Evidence) error { + if err := commandOracleValidateCurrent(ctx, ".", evidence); err != nil { + return errors.Join(err, invalidateExecutionMetrics()) + } + if err := commandOracleWriteDiagnostic(".", evidence); err != nil { + return errors.Join(err, invalidateExecutionMetrics()) + } + if err := writeMetricsFile(out); err != nil { + return errors.Join(err, invalidateExecutionMetrics()) + } + if err := commandOracleValidateCurrent(ctx, ".", evidence); err != nil { + return errors.Join(err, invalidateExecutionMetrics()) + } + printMetricsSummary(out) + return nil +} + +func invalidateExecutionMetrics() error { + err := commandOracleInvalidateDiagnostic(".") + if removeErr := invalidateMetricsFile(); removeErr != nil { + err = errors.Join(err, removeErr) + } + return err +} + +func invalidateMetricsFile() error { + return artifactfile.Remove(".", outputPath) +} + +func writeMetricsFile(out metrics) error { content, err := json.MarshalIndent(out, "", " ") if err != nil { return err } - if err := os.WriteFile(outputPath, append(content, '\n'), 0o644); err != nil { - return err - } - if routeErr != nil { - return routeErr - } + return artifactfile.WriteAtomic(".", outputPath, append(content, '\n'), 0o644) +} + +func printMetricsSummary(out metrics) { fmt.Printf("coverage metrics: requirements=%d bound=%d scenarios=%d commands=%d\n", out.Requirements.TotalRecords, out.ProofBindings.BoundRequirementCount, out.ProofBindings.ScenarioCount, out.CLIContract.CommandCount, ) - return nil } func readRequirements() ([]requirementRecord, error) { @@ -785,6 +915,10 @@ func readJSON[T any](path string) (T, error) { } func buildMetrics(requirements []requirementRecord, bindings bindingFile, witnesses witnessPlan, contract cliContract, commandInventory testevidenceinventory.Inventory) metrics { + return buildMetricsWithExecution(requirements, bindings, witnesses, contract, commandInventory, commandExecutionSummary{}) +} + +func buildMetricsWithExecution(requirements []requirementRecord, bindings bindingFile, witnesses witnessPlan, contract cliContract, commandInventory testevidenceinventory.Inventory, execution commandExecutionSummary) metrics { requirementIDs := map[string]struct{}{} active := 0 blocking := 0 @@ -837,10 +971,10 @@ func buildMetrics(requirements []requirementRecord, bindings bindingFile, witnes sort.Strings(scenarioWithoutCommand) sort.Strings(scenarioWithoutRequirement) contractCommands := cliContractCommandNames(contract) - commandRoutes := buildCommandRouteMetrics(contract, app.CommandCoverageSummaries(), commandInventory) + commandRoutes := buildCommandRouteMetricsWithExecution(contract, app.CommandCoverageSummaries(), commandInventory, execution) return metrics{ - ArtifactKind: "proofkit.coverage-metrics.v1", - SchemaVersion: 1, + ArtifactKind: "proofkit.coverage-metrics.v2", + SchemaVersion: 2, Requirements: requirementMetrics{ Active: active, Blocking: blocking, @@ -863,9 +997,9 @@ func buildMetrics(requirements []requirementRecord, bindings bindingFile, witnes }, NonClaims: []string{ "Coverage metrics report explicit requirement, binding, witness, and CLI inventory linkage only.", - "Coverage metrics classify static command route metadata as proof_route_candidate; route prose, source markers, test existence, and failure-capable AST nodes do not become execution-backed semantic evidence.", - "Coverage metrics do not execute command route candidates or observe a concrete falsification event.", - "Coverage metrics do not claim line coverage, semantic correctness, command execution, receipt freshness, or merge satisfaction.", + "Static command route metadata remains proof_route_candidate; route prose, source markers, test existence, and failure-capable AST nodes do not become execution-backed semantic evidence.", + "Execution-backed command route counts require a current materialized source snapshot, exact selected Go test lifecycle events, and owner-reserved cooperative attributes.", + "Successful selected tests do not prove assertion-branch execution, mutation adequacy, exhaustive semantic correctness, producer authentication, receipt freshness, merge satisfaction, or production readiness.", }, } } @@ -898,119 +1032,6 @@ func readCommandCoverageInventoryFrom(raw any) (testevidenceinventory.Inventory, return result.Inventory, nil } -func buildCommandRouteMetrics(contract cliContract, summaries []app.CommandCoverageSummary, inventory testevidenceinventory.Inventory) commandRouteMetrics { - missingCandidates := []string{} - missingDeclaredSemanticRoutes := []string{} - contractRefs := map[string]string{} - knownRefs := map[string]struct{}{} - candidateRefs := map[string]struct{}{} - declaredSemanticRouteRefs := map[string]struct{}{} - routeOnlyCount := 0 - candidateEntryCount := 0 - declaredSemanticRouteEntryCount := 0 - for _, command := range contract.Commands { - contractRefs[app.CommandCoverageCommandRef(command.Command)] = command.Command - } - for _, summary := range summaries { - knownRefs[summary.CommandRef] = struct{}{} - } - for _, entry := range inventory.Entries { - switch entry.EvidenceClass { - case testevidenceinventory.EvidenceClassDeclaredSemanticFalsifierRoute: - declaredSemanticRouteEntryCount++ - for _, commandRef := range entry.CommandRefs { - declaredSemanticRouteRefs[commandRef] = struct{}{} - } - case testevidenceinventory.EvidenceClassProofRouteCandidate: - candidateEntryCount++ - for _, commandRef := range entry.CommandRefs { - candidateRefs[commandRef] = struct{}{} - } - case "routing_smoke_nonclaim": - routeOnlyCount++ - } - } - unknownDeclaredSemanticRouteRefs := []string{} - for ref := range declaredSemanticRouteRefs { - if _, ok := knownRefs[ref]; !ok { - unknownDeclaredSemanticRouteRefs = append(unknownDeclaredSemanticRouteRefs, ref) - } - } - unknownCandidateRefs := []string{} - for ref := range candidateRefs { - if _, ok := knownRefs[ref]; !ok { - unknownCandidateRefs = append(unknownCandidateRefs, ref) - } - } - contractOnly := []string{} - for ref, command := range contractRefs { - if _, ok := knownRefs[ref]; !ok { - contractOnly = append(contractOnly, command) - } - } - routeOnly := []string{} - for _, summary := range summaries { - if _, ok := contractRefs[summary.CommandRef]; !ok { - routeOnly = append(routeOnly, summary.Command) - } - } - sort.Strings(contractOnly) - sort.Strings(routeOnly) - sort.Strings(unknownCandidateRefs) - sort.Strings(unknownDeclaredSemanticRouteRefs) - out := commandRouteMetrics{ - AdmittedInventoryEntryCount: len(inventory.Entries), - CommandCount: len(summaries), - ContractOnlyCommands: contractOnly, - ContractOnlyCommandCount: len(contractOnly), - RouteOnlyCommands: routeOnly, - RouteOnlyCommandCount: len(routeOnly), - RouteSmokeCount: routeOnlyCount, - ProofRouteCandidateInventoryEntryCount: candidateEntryCount, - DeclaredSemanticFalsifierRouteEntryCount: declaredSemanticRouteEntryCount, - UnknownProofRouteCandidateRefs: unknownCandidateRefs, - UnknownProofRouteCandidateRefCount: len(unknownCandidateRefs), - UnknownDeclaredSemanticRouteCommandRefs: unknownDeclaredSemanticRouteRefs, - UnknownDeclaredSemanticRouteCommandRefCount: len(unknownDeclaredSemanticRouteRefs), - } - for _, summary := range summaries { - out.Commands = append(out.Commands, summary.Command) - out.RouteCount += summary.RouteCount - out.ProofRouteCandidateRouteCount += summary.ProofRouteCandidateCount - if _, ok := candidateRefs[summary.CommandRef]; !ok { - missingCandidates = append(missingCandidates, summary.Command) - } - if _, ok := declaredSemanticRouteRefs[summary.CommandRef]; !ok { - missingDeclaredSemanticRoutes = append(missingDeclaredSemanticRoutes, summary.Command) - } - } - sort.Strings(out.Commands) - sort.Strings(missingCandidates) - sort.Strings(missingDeclaredSemanticRoutes) - out.CommandsWithoutProofRouteCandidate = missingCandidates - out.CommandWithoutProofRouteCandidateCount = len(missingCandidates) - out.CommandsWithoutDeclaredSemanticFalsifierRoute = missingDeclaredSemanticRoutes - out.CommandWithoutDeclaredSemanticFalsifierRouteCount = len(missingDeclaredSemanticRoutes) - return out -} - -func requireCommandRouteInventoryClosure(metrics commandRouteMetrics) error { - if len(metrics.CommandsWithoutProofRouteCandidate) == 0 && - len(metrics.UnknownProofRouteCandidateRefs) == 0 && - len(metrics.UnknownDeclaredSemanticRouteCommandRefs) == 0 && - len(metrics.ContractOnlyCommands) == 0 && - len(metrics.RouteOnlyCommands) == 0 { - return nil - } - return fmt.Errorf("command proof-route inventory defects: missingCandidates=%v unknownCandidateRefs=%v unknownDeclaredSemanticRouteRefs=%v contractOnly=%v routeOnly=%v", - metrics.CommandsWithoutProofRouteCandidate, - metrics.UnknownProofRouteCandidateRefs, - metrics.UnknownDeclaredSemanticRouteCommandRefs, - metrics.ContractOnlyCommands, - metrics.RouteOnlyCommands, - ) -} - func requireNoLinkageDeadZones(metrics deadZoneMetrics) error { if len(metrics.BindingWithoutRequirementIDs) == 0 && len(metrics.RequirementWithoutBindingIDs) == 0 && diff --git a/internal/tools/coveragemetrics/main_test.go b/internal/tools/coveragemetrics/main_test.go index ce468e1..ba5aa41 100644 --- a/internal/tools/coveragemetrics/main_test.go +++ b/internal/tools/coveragemetrics/main_test.go @@ -1,6 +1,7 @@ package main import ( + "context" "errors" "fmt" "os" @@ -11,6 +12,7 @@ import ( "github.com/research-engineering/agentic-proofkit/internal/app" "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + "github.com/research-engineering/agentic-proofkit/internal/tools/commandoracle" ) func TestReadJSONRejectsDuplicateKeys(t *testing.T) { @@ -161,6 +163,48 @@ func TestBuildCommandRouteMetricsReportsUnknownDeclaredSemanticRouteCommandRefs( } } +func TestBuildCommandRouteMetricsSeparatesStaticDeclarationsFromExecutionEvidence(t *testing.T) { + commandRef := app.CommandCoverageCommandRef("target") + metrics := buildCommandRouteMetricsWithExecution( + cliContractWithCommands("target"), + []app.CommandCoverageSummary{{Command: "target", CommandRef: commandRef, RouteCount: 1, ProofRouteCandidateCount: 1}}, + testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{{CommandRefs: []string{commandRef}, EvidenceClass: "proof_route_candidate"}}}, + commandExecutionSummary{ + CandidateCount: 1, + CandidateSetDigest: strings.Repeat("1", 64), + CommandRefs: []string{commandRef}, + CounterfeitCorpusDigest: strings.Repeat("2", 64), + RecordDigest: strings.Repeat("3", 64), + SourceSnapshotDigest: strings.Repeat("4", 64), + }, + ) + + if metrics.CommandWithoutExecutionBackedSemanticRouteCount != 0 || len(metrics.CommandsWithoutExecutionBackedSemanticRoute) != 0 { + t.Fatalf("execution-backed route was not closed: %#v", metrics) + } + if metrics.CommandWithoutDeclaredSemanticFalsifierRouteCount != 1 || strings.Join(metrics.CommandsWithoutDeclaredSemanticFalsifierRoute, ",") != "target" { + t.Fatalf("static declaration was incorrectly upgraded: %#v", metrics) + } + if metrics.ExecutionBackedSemanticRouteEntryCount != 1 || metrics.CommandOracleRecordDigest != strings.Repeat("3", 64) { + t.Fatalf("execution evidence identity was not projected: %#v", metrics) + } + if err := requireCommandRouteInventoryClosure(metrics); err != nil { + t.Fatalf("requireCommandRouteInventoryClosure() error = %v", err) + } +} + +func TestBuildCommandRouteMetricsRejectsExecutionEvidenceForUnknownCommand(t *testing.T) { + metrics := buildCommandRouteMetricsWithExecution( + cliContractWithCommands("target"), + []app.CommandCoverageSummary{{Command: "target", CommandRef: app.CommandCoverageCommandRef("target"), RouteCount: 1, ProofRouteCandidateCount: 1}}, + testevidenceinventory.Inventory{}, + commandExecutionSummary{CommandRefs: []string{"proofkit.cli.unknown"}}, + ) + if metrics.UnknownExecutionBackedSemanticRouteCommandRefCount != 1 || strings.Join(metrics.UnknownExecutionBackedSemanticRouteCommandRefs, ",") != "proofkit.cli.unknown" { + t.Fatalf("unknown execution command ref was not retained: %#v", metrics) + } +} + func TestBuildCommandRouteMetricsReportsContractRouteDrift(t *testing.T) { metrics := buildCommandRouteMetrics(cliContractWithCommands("contract-only", "shared"), []app.CommandCoverageSummary{ {Command: "route-only", CommandRef: app.CommandCoverageCommandRef("route-only"), RouteCount: 1}, @@ -292,17 +336,170 @@ func TestRunWritesCurrentMetricsWhenCommandRouteInventoryBuilderFails(t *testing } } +func TestWriteCurrentExecutionMetricsInvalidatesBothArtifactsOnTerminalSourceDrift(t *testing.T) { + root := t.TempDir() + oldwd, err := os.Getwd() + if err != nil { + t.Fatal(err) + } + if err := os.Chdir(root); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chdir(oldwd) }) + previousValidate := commandOracleValidateCurrent + previousWrite := commandOracleWriteDiagnostic + previousInvalidate := commandOracleInvalidateDiagnostic + t.Cleanup(func() { + commandOracleValidateCurrent = previousValidate + commandOracleWriteDiagnostic = previousWrite + commandOracleInvalidateDiagnostic = previousInvalidate + }) + validationCount := 0 + commandOracleValidateCurrent = func(context.Context, string, commandoracle.Evidence) error { + validationCount++ + if validationCount == 2 { + return errors.New("terminal source drift") + } + return nil + } + commandOracleWriteDiagnostic = func(root string, _ commandoracle.Evidence) error { + path := filepath.Join(root, filepath.FromSlash(commandoracle.RecordPath)) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + return os.WriteFile(path, []byte("diagnostic"), 0o644) + } + commandOracleInvalidateDiagnostic = func(root string) error { + err := os.Remove(filepath.Join(root, filepath.FromSlash(commandoracle.RecordPath))) + if errors.Is(err, os.ErrNotExist) { + return nil + } + return err + } + err = writeCurrentExecutionMetrics(context.Background(), metrics{}, commandoracle.Evidence{}) + if err == nil || !strings.Contains(err.Error(), "terminal source drift") { + t.Fatalf("writeCurrentExecutionMetrics() error = %v", err) + } + for _, path := range []string{outputPath, commandoracle.RecordPath} { + if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("stale artifact %s remains after source drift: %v", path, err) + } + } +} + +func TestWriteMetricsFileRejectsSymlinkEscapeWithoutMutation(t *testing.T) { + for _, testCase := range []struct { + name string + setup func(t *testing.T, root, outside string) + }{ + { + name: "destination symlink", + setup: func(t *testing.T, root, outside string) { + t.Helper() + if err := os.MkdirAll(filepath.Join(root, "artifacts", "proofkit"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink(filepath.Join(outside, "victim.json"), filepath.Join(root, filepath.FromSlash(outputPath))); err != nil { + t.Fatal(err) + } + }, + }, + { + name: "parent symlink", + setup: func(t *testing.T, root, outside string) { + t.Helper() + if err := os.Mkdir(filepath.Join(root, "artifacts"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink(outside, filepath.Join(root, "artifacts", "proofkit")); err != nil { + t.Fatal(err) + } + }, + }, + } { + t.Run(testCase.name, func(t *testing.T) { + root := t.TempDir() + outside := t.TempDir() + testCase.setup(t, root, outside) + oldwd, err := os.Getwd() + if err != nil { + t.Fatal(err) + } + if err := os.Chdir(root); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chdir(oldwd) }) + + if err := writeMetricsFile(metrics{}); err == nil { + t.Fatal("writeMetricsFile() admitted a symlink escape") + } + entries, err := os.ReadDir(outside) + if err != nil { + t.Fatal(err) + } + if len(entries) != 0 { + t.Fatalf("outside directory was mutated: %v", entries) + } + }) + } +} + +func TestInvalidateMetricsFileRejectsSymlinkParentWithoutDeletingOutsideFile(t *testing.T) { + root := t.TempDir() + outside := t.TempDir() + if err := os.Mkdir(filepath.Join(root, "artifacts"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink(outside, filepath.Join(root, "artifacts", "proofkit")); err != nil { + t.Fatal(err) + } + outsidePath := filepath.Join(outside, "coverage-metrics.json") + if err := os.WriteFile(outsidePath, []byte("outside sentinel"), 0o600); err != nil { + t.Fatal(err) + } + oldwd, err := os.Getwd() + if err != nil { + t.Fatal(err) + } + if err := os.Chdir(root); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chdir(oldwd) }) + + if err := invalidateMetricsFile(); err == nil { + t.Fatal("invalidateMetricsFile() admitted a symlink escape") + } + content, err := os.ReadFile(outsidePath) + if err != nil { + t.Fatalf("outside file was removed: %v", err) + } + if string(content) != "outside sentinel" { + t.Fatalf("outside file was mutated: %q", content) + } +} + func TestEachCommandRouteClosureConjunctHasIndependentFalsifier(t *testing.T) { + closed := closedCommandRouteMetricsFixture() cases := []struct { name string metrics commandRouteMetrics want string }{ - {name: "missing candidate", metrics: commandRouteMetrics{CommandsWithoutProofRouteCandidate: []string{"target"}}, want: "missingCandidates=[target]"}, - {name: "unknown candidate ref", metrics: commandRouteMetrics{UnknownProofRouteCandidateRefs: []string{"proofkit.unknown"}}, want: "unknownCandidateRefs=[proofkit.unknown]"}, - {name: "unknown declared route ref", metrics: commandRouteMetrics{UnknownDeclaredSemanticRouteCommandRefs: []string{"proofkit.unknown"}}, want: "unknownDeclaredSemanticRouteRefs=[proofkit.unknown]"}, - {name: "contract only", metrics: commandRouteMetrics{ContractOnlyCommands: []string{"contract-only"}}, want: "contractOnly=[contract-only]"}, - {name: "route only", metrics: commandRouteMetrics{RouteOnlyCommands: []string{"route-only"}}, want: "routeOnly=[route-only]"}, + {name: "missing candidate", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { value.CommandsWithoutProofRouteCandidate = []string{"target"} }), want: "missingCandidates=[target]"}, + {name: "unknown candidate ref", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { value.UnknownProofRouteCandidateRefs = []string{"proofkit.unknown"} }), want: "unknownCandidateRefs=[proofkit.unknown]"}, + {name: "unknown declared route ref", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { + value.UnknownDeclaredSemanticRouteCommandRefs = []string{"proofkit.unknown"} + }), want: "unknownDeclaredSemanticRouteRefs=[proofkit.unknown]"}, + {name: "missing execution-backed route", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { + value.CommandsWithoutExecutionBackedSemanticRoute = []string{"target"} + }), want: "missingExecutionBackedRoutes=[target]"}, + {name: "unknown execution-backed ref", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { + value.UnknownExecutionBackedSemanticRouteCommandRefs = []string{"proofkit.unknown"} + }), want: "unknownExecutionBackedRefs=[proofkit.unknown]"}, + {name: "execution candidate partition mismatch", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { value.ExecutionBackedSemanticRouteEntryCount = 2 }), want: "executionEntries=2 candidateEntries=1"}, + {name: "invalid command oracle digest", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { value.CommandOracleRecordDigest = "invalid" }), want: "commandOracleDigestsValid=false"}, + {name: "contract only", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { value.ContractOnlyCommands = []string{"contract-only"} }), want: "contractOnly=[contract-only]"}, + {name: "route only", metrics: mutateCommandRouteMetrics(closed, func(value *commandRouteMetrics) { value.RouteOnlyCommands = []string{"route-only"} }), want: "routeOnly=[route-only]"}, } for _, test := range cases { t.Run(test.name, func(t *testing.T) { @@ -312,11 +509,27 @@ func TestEachCommandRouteClosureConjunctHasIndependentFalsifier(t *testing.T) { } }) } - if err := requireCommandRouteInventoryClosure(commandRouteMetrics{}); err != nil { + if err := requireCommandRouteInventoryClosure(closed); err != nil { t.Fatalf("requireCommandRouteInventoryClosure() error = %v, want nil", err) } } +func closedCommandRouteMetricsFixture() commandRouteMetrics { + return commandRouteMetrics{ + ProofRouteCandidateInventoryEntryCount: 1, + ExecutionBackedSemanticRouteEntryCount: 1, + CommandOracleCandidateSetDigest: strings.Repeat("1", 64), + CommandOracleCounterfeitCorpusDigest: strings.Repeat("2", 64), + CommandOracleRecordDigest: strings.Repeat("3", 64), + CommandOracleSourceSnapshotDigest: strings.Repeat("4", 64), + } +} + +func mutateCommandRouteMetrics(value commandRouteMetrics, mutate func(*commandRouteMetrics)) commandRouteMetrics { + mutate(&value) + return value +} + func TestEachLinkageDeadZoneConjunctHasIndependentFalsifier(t *testing.T) { cases := []struct { name string @@ -849,8 +1062,11 @@ func TestBuildMetricsCarriesRealCommandRouteCandidatesAndNonClaim(t *testing.T) if metrics.CommandRoutes.CommandWithoutDeclaredSemanticFalsifierRouteCount != metrics.CommandRoutes.CommandCount { t.Fatalf("candidate-only commands did not remain missing semantic evidence: %#v", metrics.CommandRoutes) } - if !containsNonClaim(metrics.NonClaims, "do not execute command route candidates") { - t.Fatalf("metrics nonClaims=%#v, want candidate execution non-claim", metrics.NonClaims) + if metrics.CommandRoutes.CommandWithoutExecutionBackedSemanticRouteCount != metrics.CommandRoutes.CommandCount { + t.Fatalf("unexecuted candidates did not remain missing execution evidence: %#v", metrics.CommandRoutes) + } + if !containsNonClaim(metrics.NonClaims, "do not become execution-backed semantic evidence") { + t.Fatalf("metrics nonClaims=%#v, want static-evidence boundary", metrics.NonClaims) } } diff --git a/internal/tools/packageartifactrecord/record.go b/internal/tools/packageartifactrecord/record.go index df529ab..b5781e0 100644 --- a/internal/tools/packageartifactrecord/record.go +++ b/internal/tools/packageartifactrecord/record.go @@ -20,6 +20,7 @@ import ( "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" + "github.com/research-engineering/agentic-proofkit/internal/tools/repositorysnapshot" ) const ( @@ -450,27 +451,11 @@ func isSHA256(value string) bool { } func SourceSnapshot(root string) (string, string, error) { - paths, err := gitPaths(root) + snapshot, err := repositorysnapshot.Capture(root) if err != nil { return "", "", err } - digest, err := digestPaths(root, paths) - if err != nil { - return "", "", err - } - head, err := gitOutput(root, "rev-parse", "HEAD") - if err != nil { - return "", "", err - } - status, err := gitOutput(root, "status", "--porcelain=v1", "--untracked-files=all") - if err != nil { - return "", "", err - } - revision := strings.TrimSpace(head) - if strings.TrimSpace(status) != "" { - revision += "+worktree.sha256:" + digest - } - return revision, digest, nil + return snapshot.Revision, snapshot.Digest, nil } func ArtifactSnapshot(root string) (string, error) { @@ -559,53 +544,6 @@ func ToolchainDigest() (string, error) { return hex.EncodeToString(hash.Sum(nil)), nil } -func gitPaths(root string) ([]string, error) { - paths, err := gitNullPaths(root, "ls-files", "-z", "--cached", "--others", "--exclude-standard") - if err != nil { - return nil, err - } - deleted, err := gitNullPaths(root, "ls-files", "-z", "--deleted") - if err != nil { - return nil, err - } - deletedSet := make(map[string]struct{}, len(deleted)) - for _, path := range deleted { - deletedSet[path] = struct{}{} - } - current := make([]string, 0, len(paths)) - for _, path := range paths { - if _, removed := deletedSet[path]; !removed { - current = append(current, path) - } - } - sort.Strings(current) - return current, nil -} - -func gitNullPaths(root string, args ...string) ([]string, error) { - command := exec.Command("git", args...) - command.Dir = root - output, err := command.Output() - if err != nil { - return nil, err - } - parts := strings.Split(string(output), "\x00") - paths := make([]string, 0, len(parts)) - for _, path := range parts { - if path != "" { - paths = append(paths, filepath.ToSlash(path)) - } - } - return paths, nil -} - -func gitOutput(root string, args ...string) (string, error) { - command := exec.Command("git", args...) - command.Dir = root - output, err := command.Output() - return string(output), err -} - func digestPaths(root string, paths []string) (string, error) { rootFS, err := os.OpenRoot(root) if err != nil { diff --git a/internal/tools/releasecloseoutinput/main.go b/internal/tools/releasecloseoutinput/main.go index 4957503..2f9eb17 100644 --- a/internal/tools/releasecloseoutinput/main.go +++ b/internal/tools/releasecloseoutinput/main.go @@ -1,6 +1,7 @@ package main import ( + "context" "crypto/sha1" "crypto/sha256" "crypto/sha512" @@ -25,16 +26,20 @@ import ( "github.com/research-engineering/agentic-proofkit/internal/kernel/releasechannel" "github.com/research-engineering/agentic-proofkit/internal/kernel/releasepublisher" "github.com/research-engineering/agentic-proofkit/internal/kernel/trustedpublisher" + "github.com/research-engineering/agentic-proofkit/internal/tools/commandoracle" "github.com/research-engineering/agentic-proofkit/internal/tools/packageartifactrecord" "github.com/research-engineering/agentic-proofkit/internal/tools/releasechange" "github.com/research-engineering/agentic-proofkit/internal/tools/retainedevidence" ) +var commandOracleValidateCurrent = commandoracle.ValidateCurrent + const ( completionID = "proofkit.release_closeout.current_package_gate" ciProvenancePath = "artifacts/proofkit/ci-provenance.json" cliContractPath = "proofkit/cli-contract.v2.json" coverageMetricsPath = "artifacts/proofkit/coverage-metrics.json" + commandOracleRecordPath = commandoracle.RecordPath proofReceiptReportPath = "artifacts/proofkit/self-hosting-proof-receipt-admission-report.json" proofReceiptsPath = "artifacts/proofkit/self-hosting-proof-receipts.json" producerReportPath = "artifacts/proofkit/self-hosting-receipt-producer-admission-report.json" @@ -153,26 +158,35 @@ type coverageMetricsProvenance struct { } type coverageCommandRouteMetrics struct { - AdmittedInventoryEntryCount *int `json:"admittedInventoryEntryCount"` - CommandCount *int `json:"commandCount"` - Commands *[]string `json:"commands"` - CommandWithoutProofRouteCandidateCount *int `json:"commandWithoutProofRouteCandidateCount"` - CommandsWithoutProofRouteCandidate *[]string `json:"commandsWithoutProofRouteCandidate"` - ContractOnlyCommandCount *int `json:"contractOnlyCommandCount"` - ContractOnlyCommands *[]string `json:"contractOnlyCommands"` - CommandWithoutDeclaredSemanticFalsifierRouteCount *int `json:"commandWithoutDeclaredSemanticFalsifierRouteCount"` - CommandsWithoutDeclaredSemanticFalsifierRoute *[]string `json:"commandsWithoutDeclaredSemanticFalsifierRoute"` - RouteCount *int `json:"routeCount"` - RouteOnlyCommandCount *int `json:"routeOnlyCommandCount"` - RouteOnlyCommands *[]string `json:"routeOnlyCommands"` - RouteSmokeCount *int `json:"routeSmokeCount"` - ProofRouteCandidateInventoryEntryCount *int `json:"proofRouteCandidateInventoryEntryCount"` - ProofRouteCandidateRouteCount *int `json:"proofRouteCandidateRouteCount"` - DeclaredSemanticFalsifierRouteEntryCount *int `json:"declaredSemanticFalsifierRouteEntryCount"` - UnknownProofRouteCandidateRefCount *int `json:"unknownProofRouteCandidateRefCount"` - UnknownProofRouteCandidateRefs *[]string `json:"unknownProofRouteCandidateRefs"` - UnknownDeclaredSemanticRouteCommandRefCount *int `json:"unknownDeclaredSemanticRouteCommandRefCount"` - UnknownDeclaredSemanticRouteCommandRefs *[]string `json:"unknownDeclaredSemanticRouteCommandRefs"` + AdmittedInventoryEntryCount *int `json:"admittedInventoryEntryCount"` + CommandCount *int `json:"commandCount"` + Commands *[]string `json:"commands"` + CommandWithoutProofRouteCandidateCount *int `json:"commandWithoutProofRouteCandidateCount"` + CommandsWithoutProofRouteCandidate *[]string `json:"commandsWithoutProofRouteCandidate"` + ContractOnlyCommandCount *int `json:"contractOnlyCommandCount"` + ContractOnlyCommands *[]string `json:"contractOnlyCommands"` + CommandWithoutDeclaredSemanticFalsifierRouteCount *int `json:"commandWithoutDeclaredSemanticFalsifierRouteCount"` + CommandsWithoutDeclaredSemanticFalsifierRoute *[]string `json:"commandsWithoutDeclaredSemanticFalsifierRoute"` + RouteCount *int `json:"routeCount"` + RouteOnlyCommandCount *int `json:"routeOnlyCommandCount"` + RouteOnlyCommands *[]string `json:"routeOnlyCommands"` + RouteSmokeCount *int `json:"routeSmokeCount"` + ProofRouteCandidateInventoryEntryCount *int `json:"proofRouteCandidateInventoryEntryCount"` + ProofRouteCandidateRouteCount *int `json:"proofRouteCandidateRouteCount"` + DeclaredSemanticFalsifierRouteEntryCount *int `json:"declaredSemanticFalsifierRouteEntryCount"` + UnknownProofRouteCandidateRefCount *int `json:"unknownProofRouteCandidateRefCount"` + UnknownProofRouteCandidateRefs *[]string `json:"unknownProofRouteCandidateRefs"` + UnknownDeclaredSemanticRouteCommandRefCount *int `json:"unknownDeclaredSemanticRouteCommandRefCount"` + UnknownDeclaredSemanticRouteCommandRefs *[]string `json:"unknownDeclaredSemanticRouteCommandRefs"` + CommandOracleCandidateSetDigest string `json:"commandOracleCandidateSetDigest"` + CommandOracleCounterfeitCorpusDigest string `json:"commandOracleCounterfeitCorpusDigest"` + CommandOracleRecordDigest string `json:"commandOracleRecordDigest"` + CommandOracleSourceSnapshotDigest string `json:"commandOracleSourceSnapshotDigest"` + CommandWithoutExecutionBackedSemanticRouteCount *int `json:"commandWithoutExecutionBackedSemanticRouteCount"` + CommandsWithoutExecutionBackedSemanticRoute *[]string `json:"commandsWithoutExecutionBackedSemanticRoute"` + ExecutionBackedSemanticRouteEntryCount *int `json:"executionBackedSemanticRouteEntryCount"` + UnknownExecutionBackedSemanticRouteCommandRefCount *int `json:"unknownExecutionBackedSemanticRouteCommandRefCount"` + UnknownExecutionBackedSemanticRouteCommandRefs *[]string `json:"unknownExecutionBackedSemanticRouteCommandRefs"` } type coverageDeadZoneMetrics struct { @@ -245,6 +259,7 @@ type SelfEvidenceSnapshot struct { cliContractCommands []string ciProvenance selfEvidenceDocument[map[string]any] coverageMetrics selfEvidenceDocument[coverageMetricsEvidence] + commandOracle commandoracle.Evidence execution packageartifactrecord.Record proofReceiptReport selfEvidenceDocument[selfEvidenceReport] proofReceipts selfEvidenceDocument[proofReceiptSetEvidence] @@ -512,6 +527,7 @@ func releaseChannelClassificationCriterion(root string, packageManifest packageJ func selfEvidenceCriterion(root string) criterion { evidence := []string{ + commandOracleRecordPath, coverageMetricsPath, packageartifactrecord.RecordPath, proofReceiptReportPath, @@ -524,7 +540,7 @@ func selfEvidenceCriterion(root string) criterion { ok := selfEvidenceValid(root) return blockingCriterion( "proofkit.release_closeout.self_evidence", - "Current package-artifact execution, self-hosting receipt, producer admission, spec-proof bundle, and coverage metrics evidence must form one coherent local advisory closeout snapshot.", + "Current package-artifact execution, command-oracle diagnostic, self-hosting receipt, producer admission, spec-proof bundle, and coverage metrics evidence must form one coherent local advisory closeout snapshot.", ok, evidence, []string{"npm:self:receipt", "npm:self:coverage"}, @@ -558,6 +574,13 @@ func readSelfEvidenceSnapshot(root string) (SelfEvidenceSnapshot, error) { if err != nil { return SelfEvidenceSnapshot{}, err } + commandOracle, err := commandoracle.ReadDiagnostic(root) + if err != nil { + return SelfEvidenceSnapshot{}, err + } + if err := commandOracleValidateCurrent(context.Background(), root, commandOracle); err != nil { + return SelfEvidenceSnapshot{}, err + } proofReceiptReport, err := readSelfEvidenceDocument[selfEvidenceReport](root, proofReceiptReportPath) if err != nil { return SelfEvidenceSnapshot{}, err @@ -594,6 +617,7 @@ func readSelfEvidenceSnapshot(root string) (SelfEvidenceSnapshot, error) { cliContractCommands: cliContractCommands, ciProvenance: ciProvenance, coverageMetrics: coverageMetrics, + commandOracle: commandOracle, execution: execution, proofReceiptReport: proofReceiptReport, proofReceipts: proofReceipts, @@ -619,6 +643,7 @@ func readSelfEvidenceDocument[T any](root string, path string) (selfEvidenceDocu func (snapshot SelfEvidenceSnapshot) valid() bool { return coverageMetricsRecordMatches(snapshot.coverageMetrics.value, snapshot.cliContractCommands) && coverageMetricsMatchExecution(snapshot.coverageMetrics.value, snapshot.execution) && + coverageMetricsMatchCommandOracle(snapshot.coverageMetrics.value, snapshot.commandOracle, snapshot.cliContractCommands) && proofReceiptReportMatchesDocument(snapshot.proofReceiptReport, snapshot.proofReceipts) && proofReceiptDocumentMatches(snapshot.proofReceipts) && receiptProducerReportMatchesDocument(snapshot.producerReport, snapshot.producerPolicy) && @@ -629,6 +654,37 @@ func (snapshot SelfEvidenceSnapshot) valid() bool { snapshot.receiptDigestsConsistent(snapshot.execution) } +func coverageMetricsMatchCommandOracle(metrics coverageMetricsEvidence, evidence commandoracle.Evidence, cliContractCommands []string) bool { + oracle := evidence.Record + routes := metrics.CommandRoutes + expectedCommandRefs := make([]string, 0, len(cliContractCommands)) + for _, command := range cliContractCommands { + expectedCommandRefs = append(expectedCommandRefs, "proofkit.cli."+command) + } + sort.Strings(expectedCommandRefs) + oracleCommandRefs := commandoracle.ExecutionCommandRefs(evidence) + return oracle.ArtifactKind == commandoracle.ArtifactKind && + oracle.CommandID == commandoracle.CommandID && + oracle.SchemaVersion == commandoracle.SchemaVersion && + oracle.State == "passed" && + len(oracle.Entries) > 0 && + len(oracle.ExecutionCommands) > 0 && + len(oracle.NonClaims) > 0 && + reflect.DeepEqual(oracleCommandRefs, expectedCommandRefs) && + isSHA256(oracle.CandidateSetDigest) && + isSHA256(oracle.CounterfeitCorpusDigest) && + isSHA256(oracle.SourceSnapshotDigest) && + isSHA256(evidence.RecordDigest) && + routes.ExecutionBackedSemanticRouteEntryCount != nil && + *routes.ExecutionBackedSemanticRouteEntryCount == len(oracle.Entries) && + routes.CommandOracleCandidateSetDigest == oracle.CandidateSetDigest && + routes.CommandOracleCounterfeitCorpusDigest == oracle.CounterfeitCorpusDigest && + routes.CommandOracleRecordDigest == evidence.RecordDigest && + routes.CommandOracleSourceSnapshotDigest == oracle.SourceSnapshotDigest && + metrics.Provenance.SourceRevision == oracle.SourceRevision && + metrics.Provenance.SourceSnapshotDigest == oracle.SourceSnapshotDigest +} + func coverageMetricsMatchExecution(record coverageMetricsEvidence, execution packageartifactrecord.Record) bool { generatedAt, err := time.Parse(time.RFC3339Nano, record.Provenance.GeneratedAt) if err != nil { @@ -879,8 +935,8 @@ func packageArtifactCommandDigest(execution packageartifactrecord.Record) (strin } func coverageMetricsRecordMatches(record coverageMetricsEvidence, cliContractCommands []string) bool { - return record.SchemaVersion == 1 && - record.ArtifactKind == "proofkit.coverage-metrics.v1" && + return record.SchemaVersion == 2 && + record.ArtifactKind == "proofkit.coverage-metrics.v2" && coverageCLIContractMatches(record.CLIContract, cliContractCommands) && len(record.NonClaims) > 0 && len(record.ProofBindings) > 0 && @@ -994,8 +1050,11 @@ func commandRouteDefectsEmpty(routes coverageCommandRouteMetrics, cliContractCom positiveInt(routes.ProofRouteCandidateInventoryEntryCount) && positiveInt(routes.ProofRouteCandidateRouteCount) && nonNegativeInt(routes.DeclaredSemanticFalsifierRouteEntryCount) && + positiveInt(routes.ExecutionBackedSemanticRouteEntryCount) && commandRouteMetricsProducerReachable(routes, cliContractCommands) && countMatchesStringSlice(routes.CommandWithoutDeclaredSemanticFalsifierRouteCount, routes.CommandsWithoutDeclaredSemanticFalsifierRoute) && + zeroInt(routes.CommandWithoutExecutionBackedSemanticRouteCount) && + zeroInt(routes.UnknownExecutionBackedSemanticRouteCommandRefCount) && zeroInt(routes.CommandWithoutProofRouteCandidateCount) && zeroInt(routes.ContractOnlyCommandCount) && zeroInt(routes.RouteOnlyCommandCount) && @@ -1005,7 +1064,13 @@ func commandRouteDefectsEmpty(routes coverageCommandRouteMetrics, cliContractCom emptyStringSlice(routes.ContractOnlyCommands) && emptyStringSlice(routes.RouteOnlyCommands) && emptyStringSlice(routes.UnknownProofRouteCandidateRefs) && - emptyStringSlice(routes.UnknownDeclaredSemanticRouteCommandRefs) + emptyStringSlice(routes.UnknownDeclaredSemanticRouteCommandRefs) && + emptyStringSlice(routes.CommandsWithoutExecutionBackedSemanticRoute) && + emptyStringSlice(routes.UnknownExecutionBackedSemanticRouteCommandRefs) && + isSHA256(routes.CommandOracleCandidateSetDigest) && + isSHA256(routes.CommandOracleCounterfeitCorpusDigest) && + isSHA256(routes.CommandOracleRecordDigest) && + isSHA256(routes.CommandOracleSourceSnapshotDigest) } func commandRouteMetricsProducerReachable(routes coverageCommandRouteMetrics, cliContractCommands []string) bool { @@ -1019,7 +1084,12 @@ func commandRouteMetricsProducerReachable(routes coverageCommandRouteMetrics, cl routes.ProofRouteCandidateInventoryEntryCount == nil || routes.ProofRouteCandidateRouteCount == nil || routes.DeclaredSemanticFalsifierRouteEntryCount == nil || - routes.CommandWithoutDeclaredSemanticFalsifierRouteCount == nil { + routes.CommandWithoutDeclaredSemanticFalsifierRouteCount == nil || + routes.ExecutionBackedSemanticRouteEntryCount == nil || + routes.CommandWithoutExecutionBackedSemanticRouteCount == nil || + routes.CommandsWithoutExecutionBackedSemanticRoute == nil || + routes.UnknownExecutionBackedSemanticRouteCommandRefCount == nil || + routes.UnknownExecutionBackedSemanticRouteCommandRefs == nil { return false } admitted := *routes.AdmittedInventoryEntryCount @@ -1031,12 +1101,16 @@ func commandRouteMetricsProducerReachable(routes coverageCommandRouteMetrics, cl declaredEntries := *routes.DeclaredSemanticFalsifierRouteEntryCount missingCandidates := *routes.CommandWithoutProofRouteCandidateCount missingDeclared := *routes.CommandWithoutDeclaredSemanticFalsifierRouteCount + executionBacked := *routes.ExecutionBackedSemanticRouteEntryCount + missingExecutionBacked := *routes.CommandWithoutExecutionBackedSemanticRouteCount + unknownExecutionBacked := *routes.UnknownExecutionBackedSemanticRouteCommandRefCount if admitted < 0 || commands < 0 || routeCount < 0 || routeSmokeCount < 0 || candidateEntries < 0 || candidateRoutes < 0 || declaredEntries < 0 || - missingCandidates < 0 || missingDeclared < 0 || + missingCandidates < 0 || missingDeclared < 0 || executionBacked < 0 || missingExecutionBacked < 0 || unknownExecutionBacked < 0 || admitted != routeCount || candidateEntries != candidateRoutes || candidateEntries > admitted || routeSmokeCount != admitted-candidateEntries || declaredEntries != 0 || missingCandidates > commands || missingDeclared != commands || + executionBacked != candidateEntries || missingExecutionBacked != 0 || unknownExecutionBacked != 0 || candidateRoutes < commands-missingCandidates || len(*routes.Commands) != commands || !reflect.DeepEqual(*routes.Commands, *routes.CommandsWithoutDeclaredSemanticFalsifierRoute) || !reflect.DeepEqual(*routes.Commands, cliContractCommands) { @@ -1048,7 +1122,21 @@ func commandRouteMetricsProducerReachable(routes coverageCommandRouteMetrics, cl sortedUniqueStringSlice(routes.ContractOnlyCommands) && sortedUniqueStringSlice(routes.RouteOnlyCommands) && sortedUniqueStringSlice(routes.UnknownProofRouteCandidateRefs) && - sortedUniqueStringSlice(routes.UnknownDeclaredSemanticRouteCommandRefs) + sortedUniqueStringSlice(routes.UnknownDeclaredSemanticRouteCommandRefs) && + sortedUniqueStringSlice(routes.CommandsWithoutExecutionBackedSemanticRoute) && + sortedUniqueStringSlice(routes.UnknownExecutionBackedSemanticRouteCommandRefs) +} + +func isSHA256(value string) bool { + if len(value) != 64 { + return false + } + for _, character := range value { + if (character < '0' || character > '9') && (character < 'a' || character > 'f') { + return false + } + } + return true } func admittedCLIContractCommands(contract cliContractEvidence) ([]string, error) { diff --git a/internal/tools/releasecloseoutinput/main_test.go b/internal/tools/releasecloseoutinput/main_test.go index 3210e20..83ea650 100644 --- a/internal/tools/releasecloseoutinput/main_test.go +++ b/internal/tools/releasecloseoutinput/main_test.go @@ -2,6 +2,7 @@ package main import ( "bytes" + "context" "crypto/sha1" "crypto/sha256" "crypto/sha512" @@ -12,18 +13,21 @@ import ( "os" "os/exec" "path/filepath" + "runtime" "sort" "strings" "sync" "testing" "time" + "github.com/research-engineering/agentic-proofkit/internal/app" "github.com/research-engineering/agentic-proofkit/internal/command/completioncriteria" "github.com/research-engineering/agentic-proofkit/internal/command/proofreceiptadmission" "github.com/research-engineering/agentic-proofkit/internal/command/receiptproduceradmission" "github.com/research-engineering/agentic-proofkit/internal/command/specproofbundleadmission" "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" "github.com/research-engineering/agentic-proofkit/internal/kernel/digest" + "github.com/research-engineering/agentic-proofkit/internal/tools/commandoracle" "github.com/research-engineering/agentic-proofkit/internal/tools/packageartifactrecord" "github.com/research-engineering/agentic-proofkit/internal/tools/releasechange" ) @@ -42,6 +46,7 @@ var ( ) func TestMain(m *testing.M) { + commandOracleValidateCurrent = func(context.Context, string, commandoracle.Evidence) error { return nil } exitCode := m.Run() if completeFixtureBaseRoot != "" { _ = os.RemoveAll(completeFixtureBaseRoot) @@ -488,6 +493,43 @@ func TestBuildInputFailsClosedForEachBlockingEvidenceClass(t *testing.T) { writeFile(t, filepath.Join(root, "artifacts", "proofkit", "coverage-metrics.json"), "{") }, }, + { + name: "candidate-only coverage v1", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + record["artifactKind"] = "proofkit.coverage-metrics.v1" + record["schemaVersion"] = 1 + writeJSON(t, filepath.Join(root, filepath.FromSlash(coverageMetricsPath)), record) + }, + }, + { + name: "missing command oracle diagnostic", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + if err := os.Remove(filepath.Join(root, filepath.FromSlash(commandOracleRecordPath))); err != nil { + t.Fatal(err) + } + }, + }, + { + name: "command oracle digest substitution", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := readJSONMap(t, filepath.Join(root, filepath.FromSlash(coverageMetricsPath))) + record["commandRoutes"].(map[string]any)["commandOracleRecordDigest"] = strings.Repeat("9", 64) + writeJSON(t, filepath.Join(root, filepath.FromSlash(coverageMetricsPath)), record) + }, + }, + { + name: "command oracle source identity substitution", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := readJSONMap(t, filepath.Join(root, filepath.FromSlash(commandOracleRecordPath))) + record["sourceSnapshotDigest"] = strings.Repeat("9", 64) + writeJSON(t, filepath.Join(root, filepath.FromSlash(commandOracleRecordPath)), record) + }, + }, { name: "self evidence report with unrelated passed rule", criterionID: "proofkit.release_closeout.self_evidence", @@ -871,26 +913,35 @@ func producerReachableCommandRouteMetricsFixture() coverageCommandRouteMetrics { missingDeclared := []string{"proofkit.cli.a", "proofkit.cli.b"} commands := []string{"proofkit.cli.a", "proofkit.cli.b"} return coverageCommandRouteMetrics{ - AdmittedInventoryEntryCount: testIntPointer(3), - CommandCount: testIntPointer(2), - Commands: &commands, - CommandWithoutProofRouteCandidateCount: testIntPointer(0), - CommandsWithoutProofRouteCandidate: &empty, - ContractOnlyCommandCount: testIntPointer(0), - ContractOnlyCommands: &empty, - CommandWithoutDeclaredSemanticFalsifierRouteCount: testIntPointer(2), - CommandsWithoutDeclaredSemanticFalsifierRoute: &missingDeclared, - RouteCount: testIntPointer(3), - RouteOnlyCommandCount: testIntPointer(0), - RouteOnlyCommands: &empty, - RouteSmokeCount: testIntPointer(1), - ProofRouteCandidateInventoryEntryCount: testIntPointer(2), - ProofRouteCandidateRouteCount: testIntPointer(2), - DeclaredSemanticFalsifierRouteEntryCount: testIntPointer(0), - UnknownProofRouteCandidateRefCount: testIntPointer(0), - UnknownProofRouteCandidateRefs: &empty, - UnknownDeclaredSemanticRouteCommandRefCount: testIntPointer(0), - UnknownDeclaredSemanticRouteCommandRefs: &empty, + AdmittedInventoryEntryCount: testIntPointer(3), + CommandCount: testIntPointer(2), + Commands: &commands, + CommandWithoutProofRouteCandidateCount: testIntPointer(0), + CommandsWithoutProofRouteCandidate: &empty, + ContractOnlyCommandCount: testIntPointer(0), + ContractOnlyCommands: &empty, + CommandWithoutDeclaredSemanticFalsifierRouteCount: testIntPointer(2), + CommandsWithoutDeclaredSemanticFalsifierRoute: &missingDeclared, + RouteCount: testIntPointer(3), + RouteOnlyCommandCount: testIntPointer(0), + RouteOnlyCommands: &empty, + RouteSmokeCount: testIntPointer(1), + ProofRouteCandidateInventoryEntryCount: testIntPointer(2), + ProofRouteCandidateRouteCount: testIntPointer(2), + DeclaredSemanticFalsifierRouteEntryCount: testIntPointer(0), + UnknownProofRouteCandidateRefCount: testIntPointer(0), + UnknownProofRouteCandidateRefs: &empty, + UnknownDeclaredSemanticRouteCommandRefCount: testIntPointer(0), + UnknownDeclaredSemanticRouteCommandRefs: &empty, + CommandOracleCandidateSetDigest: strings.Repeat("1", 64), + CommandOracleCounterfeitCorpusDigest: strings.Repeat("2", 64), + CommandOracleRecordDigest: strings.Repeat("3", 64), + CommandOracleSourceSnapshotDigest: strings.Repeat("4", 64), + CommandWithoutExecutionBackedSemanticRouteCount: testIntPointer(0), + CommandsWithoutExecutionBackedSemanticRoute: &empty, + ExecutionBackedSemanticRouteEntryCount: testIntPointer(2), + UnknownExecutionBackedSemanticRouteCommandRefCount: testIntPointer(0), + UnknownExecutionBackedSemanticRouteCommandRefs: &empty, } } @@ -965,6 +1016,67 @@ func TestSelfEvidenceRejectsArbitraryValidJSON(t *testing.T) { assertCloseoutOutcome(t, input, 1, "failed") } +func TestSelfEvidenceRejectsProducerUnreachableCommandOracleRef(t *testing.T) { + root := completeFixture(t) + evidence, err := commandoracle.ReadDiagnostic(root) + if err != nil { + t.Fatal(err) + } + record := evidence.Record + record.Entries = append([]commandoracle.JoinedEntry(nil), record.Entries...) + record.Entries[0].Candidate.CommandRef = "proofkit.cli.counterfeit.command" + candidates := make([]app.CommandCoverageOracleCandidate, 0, len(record.Entries)) + for _, entry := range record.Entries { + candidates = append(candidates, entry.Candidate) + } + record.CandidateSetDigest, err = commandoracle.CandidateSetDigest(candidates) + if err != nil { + t.Fatal(err) + } + mutated, err := commandoracle.EvidenceForRecord(record) + if err != nil { + t.Fatalf("counterfeit oracle must remain internally valid: %v", err) + } + if err := commandoracle.WriteDiagnostic(root, mutated); err != nil { + t.Fatal(err) + } + coveragePath := filepath.Join(root, filepath.FromSlash(coverageMetricsPath)) + coverage := readJSONMap(t, coveragePath) + routes := coverage["commandRoutes"].(map[string]any) + routes["commandOracleCandidateSetDigest"] = mutated.Record.CandidateSetDigest + routes["commandOracleRecordDigest"] = mutated.RecordDigest + writeJSON(t, coveragePath, coverage) + + input, err := buildInput(root) + if err != nil { + t.Fatal(err) + } + assertCriterionStatus(t, input, "proofkit.release_closeout.self_evidence", "missing_evidence") +} + +func TestSelfEvidenceInvokesCurrentCommandOracleOwner(t *testing.T) { + root := completeFixture(t) + previous := commandOracleValidateCurrent + called := false + commandOracleValidateCurrent = func(_ context.Context, gotRoot string, evidence commandoracle.Evidence) error { + called = true + if gotRoot != root || evidence.Record.CommandID != commandoracle.CommandID { + t.Fatalf("current command oracle owner received root=%q record=%#v", gotRoot, evidence.Record) + } + return fmt.Errorf("counterfeit currentness failure") + } + t.Cleanup(func() { commandOracleValidateCurrent = previous }) + + input, err := buildInput(root) + if err != nil { + t.Fatal(err) + } + if !called { + t.Fatal("release closeout did not invoke the current command oracle owner") + } + assertCriterionStatus(t, input, "proofkit.release_closeout.self_evidence", "missing_evidence") +} + func TestSelfEvidenceRequiresCurrentMatchingPackageArtifactExecution(t *testing.T) { cases := []struct { name string @@ -1181,7 +1293,7 @@ func populateCompleteFixture(t *testing.T, root string) { writeJSON(t, filepath.Join(root, "proofkit", "requirement-bindings.json"), map[string]any{"schemaVersion": 1}) writeJSON(t, filepath.Join(root, "proofkit", "witness-plan.json"), map[string]any{"schemaVersion": 1}) writeJSON(t, filepath.Join(root, filepath.FromSlash(cliContractPath)), map[string]any{ - "commands": []any{map[string]any{"command": "proofkit.coverage.command"}}, + "commands": []any{map[string]any{"command": "coverage.command"}}, }) writeFile(t, filepath.Join(root, "source.txt"), "source-v1\n") writeJSON(t, filepath.Join(root, "package.json"), map[string]any{ @@ -1243,7 +1355,16 @@ func writeLocalSelfEvidence(t *testing.T, root string, execution packageartifact "generatedAt": "2026-07-01T10:00:02Z", "sourceRevision": execution.SourceRevision, }) + oracle := commandOracleFixture(execution) + if err := commandoracle.WriteDiagnostic(root, oracle); err != nil { + t.Fatal(err) + } coverage := coverageMetricsFixture() + coverageRoutes := coverage["commandRoutes"].(map[string]any) + coverageRoutes["commandOracleCandidateSetDigest"] = oracle.Record.CandidateSetDigest + coverageRoutes["commandOracleCounterfeitCorpusDigest"] = oracle.Record.CounterfeitCorpusDigest + coverageRoutes["commandOracleRecordDigest"] = oracle.RecordDigest + coverageRoutes["commandOracleSourceSnapshotDigest"] = execution.SourceSnapshotDigest coverage["provenance"] = coverageMetricsProvenanceFixture(execution) writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), coverage) writeJSON(t, filepath.Join(root, "artifacts/proofkit/self-hosting-proof-receipt-admission-report.json"), selfEvidenceReportFixture("proofkit.proof-receipt-admission", "proofkit.self-hosting.proof-receipts", "proofkit.proof-receipt-admission.boundary", "proofkit.proof-receipt-admission.receipts")) @@ -1452,33 +1573,42 @@ func refreshBundleChildReports(bundle map[string]any) { func coverageMetricsFixture() map[string]any { return map[string]any{ - "artifactKind": "proofkit.coverage-metrics.v1", - "schemaVersion": 1, + "artifactKind": "proofkit.coverage-metrics.v2", + "schemaVersion": 2, "requirements": map[string]any{"blocking": 1}, "proofBindings": map[string]any{"boundRequirementCount": 1}, "witnessPlan": map[string]any{"commandCount": 1}, - "cliContract": map[string]any{"commandCount": 1, "commands": []any{"proofkit.coverage.command"}}, + "cliContract": map[string]any{"commandCount": 1, "commands": []any{"coverage.command"}}, "commandRoutes": map[string]any{ - "admittedInventoryEntryCount": 1, - "commandCount": 1, - "commands": []any{"proofkit.coverage.command"}, - "commandWithoutProofRouteCandidateCount": 0, - "commandsWithoutProofRouteCandidate": []any{}, - "commandWithoutDeclaredSemanticFalsifierRouteCount": 1, - "commandsWithoutDeclaredSemanticFalsifierRoute": []any{"proofkit.coverage.command"}, - "contractOnlyCommandCount": 0, - "contractOnlyCommands": []any{}, - "routeCount": 1, - "routeOnlyCommandCount": 0, - "routeOnlyCommands": []any{}, - "routeSmokeCount": 0, - "proofRouteCandidateInventoryEntryCount": 1, - "proofRouteCandidateRouteCount": 1, - "declaredSemanticFalsifierRouteEntryCount": 0, - "unknownProofRouteCandidateRefCount": 0, - "unknownProofRouteCandidateRefs": []any{}, - "unknownDeclaredSemanticRouteCommandRefCount": 0, - "unknownDeclaredSemanticRouteCommandRefs": []any{}, + "admittedInventoryEntryCount": 1, + "commandCount": 1, + "commands": []any{"coverage.command"}, + "commandWithoutProofRouteCandidateCount": 0, + "commandsWithoutProofRouteCandidate": []any{}, + "commandWithoutDeclaredSemanticFalsifierRouteCount": 1, + "commandsWithoutDeclaredSemanticFalsifierRoute": []any{"coverage.command"}, + "contractOnlyCommandCount": 0, + "contractOnlyCommands": []any{}, + "routeCount": 1, + "routeOnlyCommandCount": 0, + "routeOnlyCommands": []any{}, + "routeSmokeCount": 0, + "proofRouteCandidateInventoryEntryCount": 1, + "proofRouteCandidateRouteCount": 1, + "declaredSemanticFalsifierRouteEntryCount": 0, + "unknownProofRouteCandidateRefCount": 0, + "unknownProofRouteCandidateRefs": []any{}, + "unknownDeclaredSemanticRouteCommandRefCount": 0, + "unknownDeclaredSemanticRouteCommandRefs": []any{}, + "commandOracleCandidateSetDigest": strings.Repeat("1", 64), + "commandOracleCounterfeitCorpusDigest": strings.Repeat("2", 64), + "commandOracleRecordDigest": strings.Repeat("3", 64), + "commandOracleSourceSnapshotDigest": strings.Repeat("0", 64), + "commandWithoutExecutionBackedSemanticRouteCount": 0, + "commandsWithoutExecutionBackedSemanticRoute": []any{}, + "executionBackedSemanticRouteEntryCount": 1, + "unknownExecutionBackedSemanticRouteCommandRefCount": 0, + "unknownExecutionBackedSemanticRouteCommandRefs": []any{}, }, "deadZones": map[string]any{ "bindingWithoutRequirementIds": []any{}, @@ -1496,6 +1626,54 @@ func coverageMetricsFixture() map[string]any { } } +func commandOracleFixture(execution packageartifactrecord.Record) commandoracle.Evidence { + candidate := app.CommandCoverageOracleCandidate{ + AssertionOracleID: "proofkit.oracle.fixture", + CommandRef: "proofkit.cli.coverage.command", + ExpectedPublicOutcome: "Fixture command rejects its declared counterfeit input.", + FalsificationEventID: "proofkit.falsifier.fixture", + NegativeCaseID: "proofkit.negative.fixture", + OracleKind: "semantic_route_falsifier", + OwnerInvariantID: "proofkit.invariant.fixture", + PackagePath: "./internal/sample", + Selector: "internal/sample/sample_test.go::TestFixture", + SourceMarker: "proofkit.command_coverage.source_oracle.v1.000000000000000000000000000000000000000000000000000000000000000000000000000001", + SourcePath: "internal/sample/sample_test.go", + TestID: "proofkit.test.fixture", + TestName: "TestFixture", + WrongImplementationClass: "proofkit.wrong.fixture", + } + candidates := []app.CommandCoverageOracleCandidate{candidate} + candidateDigest, err := commandoracle.CandidateSetDigest(candidates) + if err != nil { + panic(err) + } + record := commandoracle.Record{ + ArtifactKind: commandoracle.ArtifactKind, + CandidateSetDigest: candidateDigest, + CommandID: commandoracle.CommandID, + CounterfeitCorpusDigest: strings.Repeat("2", 64), + Entries: []commandoracle.JoinedEntry{{ + Candidate: candidate, + ExecutionState: "passed", + PackageImportPath: "example.test/proofkit/internal/sample", + }}, + ExecutionCommands: commandoracle.ExecutionCommandsForCandidates(candidates), + GoVersion: runtime.Version(), + NonClaims: commandoracle.RecordNonClaims(), + Platform: runtime.GOOS + "/" + runtime.GOARCH, + SchemaVersion: commandoracle.SchemaVersion, + SourceRevision: execution.SourceRevision, + SourceSnapshotDigest: execution.SourceSnapshotDigest, + State: "passed", + } + evidence, err := commandoracle.EvidenceForRecord(record) + if err != nil { + panic(err) + } + return evidence +} + func coverageMetricsProvenanceFixture(execution packageartifactrecord.Record) map[string]any { finishedAt, err := time.Parse(time.RFC3339Nano, execution.FinishedAt) if err != nil { diff --git a/internal/tools/repositorysnapshot/filesystem.go b/internal/tools/repositorysnapshot/filesystem.go new file mode 100644 index 0000000..96e0c17 --- /dev/null +++ b/internal/tools/repositorysnapshot/filesystem.go @@ -0,0 +1,218 @@ +package repositorysnapshot + +import ( + "context" + "crypto/sha256" + "encoding/binary" + "encoding/hex" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "sort" + "strings" +) + +func admitEmptyDestination(root, destination string, sourceRoot, destinationRoot *os.Root) error { + rootResolved, err := resolvedPath(root) + if err != nil { + return err + } + destinationResolved, err := resolvedPath(destination) + if err != nil { + return err + } + rootInfo, err := os.Stat(rootResolved) + if err != nil { + return fmt.Errorf("stat repository snapshot source root failed") + } + rootHandleInfo, err := sourceRoot.Stat(".") + if err != nil || !os.SameFile(rootInfo, rootHandleInfo) { + return fmt.Errorf("repository snapshot source root changed during admission") + } + destinationInfo, err := os.Stat(destinationResolved) + if err != nil { + return fmt.Errorf("stat repository snapshot destination failed") + } + destinationHandleInfo, err := destinationRoot.Stat(".") + if err != nil || !os.SameFile(destinationInfo, destinationHandleInfo) { + return fmt.Errorf("repository snapshot destination changed during admission") + } + for current := destinationResolved; ; current = filepath.Dir(current) { + info, err := os.Stat(current) + if err != nil { + return fmt.Errorf("stat repository snapshot destination ancestor failed") + } + if os.SameFile(rootInfo, info) { + return fmt.Errorf("repository snapshot destination must be outside the source root") + } + if parent := filepath.Dir(current); parent == current { + break + } + } + entries, err := fs.ReadDir(destinationRoot.FS(), ".") + if err != nil { + return fmt.Errorf("read repository snapshot destination failed") + } + if len(entries) != 0 { + return fmt.Errorf("repository snapshot destination must be empty") + } + return nil +} + +func resolvedPath(path string) (string, error) { + absolute, err := filepath.Abs(path) + if err != nil { + return "", fmt.Errorf("repository snapshot path resolution failed") + } + resolved, err := filepath.EvalSymlinks(absolute) + if err != nil { + return "", fmt.Errorf("repository snapshot path resolution failed") + } + return filepath.Clean(resolved), nil +} + +func digestPaths(ctx context.Context, root string, paths []string) (string, error) { + rootFS, err := os.OpenRoot(root) + if err != nil { + return "", fmt.Errorf("open repository snapshot root failed") + } + defer rootFS.Close() + hash := sha256.New() + totalBytes := int64(0) + for _, path := range paths { + if err := ctx.Err(); err != nil { + return "", fmt.Errorf("repository snapshot operation canceled: %w", err) + } + normalized, err := normalizedPath(path) + if err != nil { + return "", err + } + info, content, err := readSourceFile(rootFS, normalized) + if err != nil { + return "", err + } + totalBytes += int64(len(content)) + if totalBytes > maxSnapshotBytes { + return "", fmt.Errorf("repository snapshot exceeds total byte limit") + } + writeDigestField(hash, []byte(normalized)) + writeDigestField(hash, []byte("regular")) + writeDigestField(hash, []byte(normalizedMode(info.Mode()))) + writeDigestField(hash, content) + } + return hex.EncodeToString(hash.Sum(nil)), nil +} + +func materializedPaths(ctx context.Context, root string) ([]string, error) { + paths := []string{} + err := filepath.WalkDir(root, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return fmt.Errorf("walk materialized repository snapshot failed") + } + if err := ctx.Err(); err != nil { + return fmt.Errorf("repository snapshot operation canceled: %w", err) + } + if path == root || entry.IsDir() { + return nil + } + info, err := entry.Info() + if err != nil { + return fmt.Errorf("inspect materialized repository snapshot entry failed") + } + if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() { + return fmt.Errorf("materialized repository snapshot contains non-regular file") + } + relative, err := filepath.Rel(root, path) + if err != nil { + return fmt.Errorf("relativize materialized repository snapshot entry failed") + } + normalized, err := normalizedPath(filepath.ToSlash(relative)) + if err != nil { + return err + } + paths = append(paths, normalized) + if len(paths) > maxSnapshotFiles { + return fmt.Errorf("materialized repository snapshot exceeds file-count limit") + } + return nil + }) + if err != nil { + return nil, err + } + sort.Strings(paths) + return paths, nil +} + +func readSourceFile(root *os.Root, path string) (fs.FileInfo, []byte, error) { + localPath := filepath.FromSlash(path) + info, err := root.Lstat(localPath) + if err != nil { + return nil, nil, fmt.Errorf("repository snapshot source entry lstat failed") + } + if info.Mode()&os.ModeSymlink != 0 { + return nil, nil, fmt.Errorf("repository snapshot source entry symlinks are not admitted") + } + if !info.Mode().IsRegular() { + return nil, nil, fmt.Errorf("repository snapshot source entry non-regular files are not admitted") + } + if info.Size() > maxSourceFileBytes { + return nil, nil, fmt.Errorf("repository snapshot source entry exceeds resource limit") + } + file, err := root.Open(localPath) + if err != nil { + return nil, nil, fmt.Errorf("open repository snapshot source entry failed") + } + defer file.Close() + opened, err := file.Stat() + if err != nil || !os.SameFile(info, opened) || !opened.Mode().IsRegular() { + return nil, nil, fmt.Errorf("repository snapshot source entry changed during admission") + } + content, err := io.ReadAll(io.LimitReader(file, maxSourceFileBytes+1)) + if err != nil { + return nil, nil, fmt.Errorf("read repository snapshot source entry failed") + } + if int64(len(content)) > maxSourceFileBytes { + return nil, nil, fmt.Errorf("repository snapshot source entry exceeds resource limit") + } + after, err := file.Stat() + if err != nil || !os.SameFile(opened, after) || opened.Size() != after.Size() || after.Size() != int64(len(content)) { + return nil, nil, fmt.Errorf("repository snapshot source entry changed while being read") + } + return opened, content, nil +} + +func normalizedPath(path string) (string, error) { + normalized := filepath.ToSlash(filepath.Clean(filepath.FromSlash(path))) + if normalized == "." || filepath.IsAbs(filepath.FromSlash(path)) || normalized == ".." || strings.HasPrefix(normalized, "../") { + return "", fmt.Errorf("snapshot path must be normalized and repository-relative") + } + if path != normalized { + return "", fmt.Errorf("snapshot path is not normalized") + } + return normalized, nil +} + +func normalizedMode(mode fs.FileMode) string { + return fmt.Sprintf("%04o", mode.Perm()) +} + +func writeDigestField(hash interface{ Write([]byte) (int, error) }, value []byte) { + var size [8]byte + binary.BigEndian.PutUint64(size[:], uint64(len(value))) + _, _ = hash.Write(size[:]) + _, _ = hash.Write(value) +} + +func isSHA256(value string) bool { + if len(value) != 64 { + return false + } + for _, character := range value { + if (character < '0' || character > '9') && (character < 'a' || character > 'f') { + return false + } + } + return true +} diff --git a/internal/tools/repositorysnapshot/git_inventory.go b/internal/tools/repositorysnapshot/git_inventory.go new file mode 100644 index 0000000..bcd6e5a --- /dev/null +++ b/internal/tools/repositorysnapshot/git_inventory.go @@ -0,0 +1,178 @@ +package repositorysnapshot + +import ( + "context" + "fmt" + "os/exec" + "path/filepath" + "sort" + "strings" + "sync" + "time" +) + +const ( + maxGitOutputBytes = 16 << 20 + processWaitDelay = 5 * time.Second +) + +func gitPaths(ctx context.Context, root string) ([]string, error) { + paths, err := gitNullPaths(ctx, root, "ls-files", "-z", "--cached", "--others", "--exclude-standard") + if err != nil { + return nil, err + } + deleted, err := gitNullPaths(ctx, root, "ls-files", "-z", "--deleted") + if err != nil { + return nil, err + } + deletedSet := make(map[string]struct{}, len(deleted)) + for _, path := range deleted { + deletedSet[path] = struct{}{} + } + current := make([]string, 0, len(paths)) + for _, path := range paths { + if _, removed := deletedSet[path]; !removed { + current = append(current, path) + } + } + sort.Strings(current) + if len(current) == 0 { + return nil, fmt.Errorf("repository snapshot source inventory is empty") + } + if len(current) > maxSnapshotFiles { + return nil, fmt.Errorf("repository snapshot exceeds file-count limit") + } + for index := 1; index < len(current); index++ { + if current[index] == current[index-1] { + return nil, fmt.Errorf("repository snapshot contains duplicate path") + } + } + return current, nil +} + +func gitNullPaths(ctx context.Context, root string, args ...string) ([]string, error) { + output, err := gitOutput(ctx, root, args...) + if err != nil { + return nil, err + } + parts := strings.Split(output, "\x00") + paths := make([]string, 0, len(parts)) + for _, path := range parts { + if path == "" { + continue + } + normalized, err := normalizedPath(filepath.ToSlash(path)) + if err != nil { + return nil, err + } + paths = append(paths, normalized) + } + return paths, nil +} + +func sourceRevision(ctx context.Context, root, digest string) (string, error) { + head, err := gitOutput(ctx, root, "rev-parse", "HEAD") + if err != nil { + return "", err + } + status, err := gitOutput(ctx, root, "status", "--porcelain=v1", "--untracked-files=all") + if err != nil { + return "", err + } + revision := strings.TrimSpace(head) + if !isGitObjectID(revision) { + return "", fmt.Errorf("git revision identity is invalid") + } + if strings.TrimSpace(status) != "" { + revision += "+worktree.sha256:" + digest + } + return revision, nil +} + +func gitOutput(ctx context.Context, root string, args ...string) (string, error) { + command := exec.CommandContext(ctx, "git", args...) + command.Dir = root + command.WaitDelay = processWaitDelay + configureProcessGroup(command) + stdout := newBoundedBuffer() + stderr := newBoundedBuffer() + command.Stdout = stdout + command.Stderr = stderr + if err := command.Start(); err != nil { + return "", fmt.Errorf("git %s failed to start", strings.Join(args, " ")) + } + waitDone := make(chan error, 1) + go func() { waitDone <- command.Wait() }() + var waitErr error + overflowed := false + stdoutExceeded := stdout.Exceeded() + stderrExceeded := stderr.Exceeded() + contextDone := ctx.Done() + waitComplete := false + for !waitComplete { + select { + case waitErr = <-waitDone: + waitComplete = true + case <-stdoutExceeded: + overflowed = true + stdoutExceeded = nil + _ = terminateProcessGroup(command) + case <-stderrExceeded: + overflowed = true + stderrExceeded = nil + _ = terminateProcessGroup(command) + case <-contextDone: + contextDone = nil + _ = terminateProcessGroup(command) + } + } + if ctx.Err() != nil { + return "", fmt.Errorf("repository snapshot operation canceled: %w", ctx.Err()) + } + if overflowed || stdout.Overflowed() || stderr.Overflowed() { + return "", fmt.Errorf("git output exceeds resource limit") + } + if waitErr != nil { + return "", fmt.Errorf("git %s failed", strings.Join(args, " ")) + } + if len(stderr.content) != 0 { + return "", fmt.Errorf("git %s emitted diagnostics", strings.Join(args, " ")) + } + return string(stdout.content), nil +} + +type boundedBuffer struct { + content []byte + exceeded chan struct{} + once sync.Once +} + +func newBoundedBuffer() *boundedBuffer { + return &boundedBuffer{exceeded: make(chan struct{})} +} + +func (buffer *boundedBuffer) Write(value []byte) (int, error) { + remaining := maxGitOutputBytes - len(buffer.content) + if remaining > 0 { + count := len(value) + if count > remaining { + count = remaining + } + buffer.content = append(buffer.content, value[:count]...) + } + if len(value) > remaining { + buffer.once.Do(func() { close(buffer.exceeded) }) + } + return len(value), nil +} + +func (buffer *boundedBuffer) Exceeded() <-chan struct{} { return buffer.exceeded } + +func (buffer *boundedBuffer) Overflowed() bool { + select { + case <-buffer.exceeded: + return true + default: + return false + } +} diff --git a/internal/tools/repositorysnapshot/process_unix.go b/internal/tools/repositorysnapshot/process_unix.go new file mode 100644 index 0000000..1aaa1e1 --- /dev/null +++ b/internal/tools/repositorysnapshot/process_unix.go @@ -0,0 +1,28 @@ +//go:build darwin || linux + +package repositorysnapshot + +import ( + "errors" + "os" + "os/exec" + "syscall" +) + +func configureProcessGroup(command *exec.Cmd) { + command.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + command.Cancel = func() error { + return terminateProcessGroup(command) + } +} + +func terminateProcessGroup(command *exec.Cmd) error { + if command.Process == nil { + return nil + } + err := syscall.Kill(-command.Process.Pid, syscall.SIGKILL) + if errors.Is(err, os.ErrProcessDone) || errors.Is(err, syscall.ESRCH) { + return nil + } + return err +} diff --git a/internal/tools/repositorysnapshot/snapshot.go b/internal/tools/repositorysnapshot/snapshot.go new file mode 100644 index 0000000..65bef66 --- /dev/null +++ b/internal/tools/repositorysnapshot/snapshot.go @@ -0,0 +1,177 @@ +package repositorysnapshot + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "fmt" + "os" + "path/filepath" + "strings" +) + +const ( + maxSnapshotFiles = 20_000 + maxSnapshotBytes = 512 << 20 + maxSourceFileBytes = 64 << 20 +) + +type Snapshot struct { + Digest string + Paths []string + Revision string +} + +func Capture(root string) (Snapshot, error) { + return CaptureContext(context.Background(), root) +} + +func CaptureContext(ctx context.Context, root string) (Snapshot, error) { + paths, err := gitPaths(ctx, root) + if err != nil { + return Snapshot{}, err + } + digest, err := digestPaths(ctx, root, paths) + if err != nil { + return Snapshot{}, err + } + revision, err := sourceRevision(ctx, root, digest) + if err != nil { + return Snapshot{}, err + } + return Snapshot{Digest: digest, Paths: append([]string(nil), paths...), Revision: revision}, nil +} + +func Materialize(root, destination string) (Snapshot, error) { + return MaterializeContext(context.Background(), root, destination) +} + +func MaterializeContext(ctx context.Context, root, destination string) (Snapshot, error) { + sourceRoot, err := os.OpenRoot(root) + if err != nil { + return Snapshot{}, fmt.Errorf("open repository snapshot source failed") + } + defer sourceRoot.Close() + destinationRoot, err := os.OpenRoot(destination) + if err != nil { + return Snapshot{}, fmt.Errorf("open repository snapshot destination failed") + } + defer destinationRoot.Close() + if err := admitEmptyDestination(root, destination, sourceRoot, destinationRoot); err != nil { + return Snapshot{}, err + } + paths, err := gitPaths(ctx, root) + if err != nil { + return Snapshot{}, err + } + + hash := sha256.New() + totalBytes := int64(0) + for _, path := range paths { + if err := ctx.Err(); err != nil { + return Snapshot{}, fmt.Errorf("repository snapshot operation canceled: %w", err) + } + normalized, err := normalizedPath(path) + if err != nil { + return Snapshot{}, err + } + info, content, err := readSourceFile(sourceRoot, normalized) + if err != nil { + return Snapshot{}, err + } + totalBytes += int64(len(content)) + if totalBytes > maxSnapshotBytes { + return Snapshot{}, fmt.Errorf("repository snapshot exceeds total byte limit") + } + if err := destinationRoot.MkdirAll(filepath.Dir(filepath.FromSlash(normalized)), 0o755); err != nil { + return Snapshot{}, fmt.Errorf("create materialized snapshot directory failed") + } + file, err := destinationRoot.OpenFile(filepath.FromSlash(normalized), os.O_WRONLY|os.O_CREATE|os.O_EXCL, info.Mode().Perm()) + if err != nil { + return Snapshot{}, fmt.Errorf("create materialized snapshot file failed") + } + if _, err := file.Write(content); err != nil { + file.Close() + return Snapshot{}, fmt.Errorf("write materialized snapshot file failed") + } + if err := file.Chmod(info.Mode().Perm()); err != nil { + file.Close() + return Snapshot{}, fmt.Errorf("set materialized snapshot file mode failed") + } + if err := file.Close(); err != nil { + return Snapshot{}, fmt.Errorf("close materialized snapshot file failed") + } + writeDigestField(hash, []byte(normalized)) + writeDigestField(hash, []byte("regular")) + writeDigestField(hash, []byte(normalizedMode(info.Mode()))) + writeDigestField(hash, content) + } + digest := hex.EncodeToString(hash.Sum(nil)) + revision, err := sourceRevision(ctx, root, digest) + if err != nil { + return Snapshot{}, err + } + return Snapshot{Digest: digest, Paths: append([]string(nil), paths...), Revision: revision}, nil +} + +func ValidateMaterialized(root string, snapshot Snapshot) error { + return ValidateMaterializedContext(context.Background(), root, snapshot) +} + +func ValidateMaterializedContext(ctx context.Context, root string, snapshot Snapshot) error { + if len(snapshot.Paths) == 0 || !isSHA256(snapshot.Digest) || !ValidRevision(snapshot.Revision) { + return fmt.Errorf("repository snapshot identity is incomplete") + } + paths, err := materializedPaths(ctx, root) + if err != nil { + return err + } + if !equalStrings(paths, snapshot.Paths) { + return fmt.Errorf("materialized repository snapshot inventory is stale") + } + digest, err := digestPaths(ctx, root, snapshot.Paths) + if err != nil { + return err + } + if digest != snapshot.Digest { + return fmt.Errorf("materialized repository snapshot digest is stale") + } + return nil +} + +func ValidRevision(value string) bool { + const dirtySeparator = "+worktree.sha256:" + parts := strings.Split(value, dirtySeparator) + if len(parts) > 2 || !isGitObjectID(parts[0]) { + return false + } + return len(parts) == 1 || isSHA256(parts[1]) +} + +func isGitObjectID(value string) bool { + if len(value) != 40 && len(value) != 64 { + return false + } + for _, character := range value { + if (character < '0' || character > '9') && (character < 'a' || character > 'f') { + return false + } + } + return true +} + +func EqualIdentity(left, right Snapshot) bool { + return left.Revision == right.Revision && left.Digest == right.Digest && equalStrings(left.Paths, right.Paths) +} + +func equalStrings(left, right []string) bool { + if len(left) != len(right) { + return false + } + for index := range left { + if left[index] != right[index] { + return false + } + } + return true +} diff --git a/internal/tools/repositorysnapshot/snapshot_test.go b/internal/tools/repositorysnapshot/snapshot_test.go new file mode 100644 index 0000000..99fd61b --- /dev/null +++ b/internal/tools/repositorysnapshot/snapshot_test.go @@ -0,0 +1,246 @@ +package repositorysnapshot + +import ( + "context" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" +) + +func TestMaterializeBindsCopiedBytesAndRejectsLiveMutation(t *testing.T) { + root := initializeRepository(t) + writeFile(t, filepath.Join(root, "a.txt"), "alpha") + if err := os.Chmod(filepath.Join(root, "a.txt"), 0o760); err != nil { + t.Fatal(err) + } + runGit(t, root, "add", "a.txt") + runGit(t, root, "commit", "-m", "initial") + destination := t.TempDir() + + snapshot, err := Materialize(root, destination) + if err != nil { + t.Fatalf("Materialize() error = %v", err) + } + if err := ValidateMaterialized(destination, snapshot); err != nil { + t.Fatalf("ValidateMaterialized() error = %v", err) + } + writeFile(t, filepath.Join(root, "a.txt"), "mutated") + current, err := Capture(root) + if err != nil { + t.Fatalf("Capture() error = %v", err) + } + if EqualIdentity(snapshot, current) { + t.Fatal("live mutation preserved snapshot identity") + } + content, err := os.ReadFile(filepath.Join(destination, "a.txt")) + if err != nil { + t.Fatalf("read materialized file: %v", err) + } + if string(content) != "alpha" { + t.Fatalf("materialized content = %q, want alpha", content) + } + info, err := os.Stat(filepath.Join(destination, "a.txt")) + if err != nil { + t.Fatal(err) + } + if got := info.Mode().Perm(); got != 0o760 { + t.Fatalf("materialized mode = %04o, want 0760", got) + } +} + +func TestMaterializeRejectsSymlinkAndNonEmptyDestination(t *testing.T) { + root := initializeRepository(t) + writeFile(t, filepath.Join(root, "target.txt"), "target") + const callerPath = "caller-path-sentinel" + if err := os.Symlink("target.txt", filepath.Join(root, callerPath)); err != nil { + t.Fatalf("symlink: %v", err) + } + runGit(t, root, "add", "target.txt", callerPath) + runGit(t, root, "commit", "-m", "initial") + if _, err := Materialize(root, t.TempDir()); err == nil || !strings.Contains(err.Error(), "symlinks are not admitted") { + t.Fatalf("Materialize() symlink error = %v", err) + } else if strings.Contains(err.Error(), callerPath) { + t.Fatalf("Materialize() leaked caller-owned path: %v", err) + } + + plainRoot := initializeRepository(t) + writeFile(t, filepath.Join(plainRoot, "a.txt"), "alpha") + runGit(t, plainRoot, "add", "a.txt") + runGit(t, plainRoot, "commit", "-m", "initial") + destination := t.TempDir() + writeFile(t, filepath.Join(destination, "occupied"), "x") + if _, err := Materialize(plainRoot, destination); err == nil || !strings.Contains(err.Error(), "must be empty") { + t.Fatalf("Materialize() non-empty destination error = %v", err) + } +} + +func TestMaterializeRejectsSymlinkedDestinationInsideSource(t *testing.T) { + root := initializeRepository(t) + writeFile(t, filepath.Join(root, "tracked.txt"), "tracked") + runGit(t, root, "add", "tracked.txt") + runGit(t, root, "commit", "-m", "initial") + insideSource := filepath.Join(root, "snapshot-output") + if err := os.Mkdir(insideSource, 0o755); err != nil { + t.Fatal(err) + } + destination := filepath.Join(t.TempDir(), "destination") + if err := os.Symlink(insideSource, destination); err != nil { + t.Fatal(err) + } + + if _, err := Materialize(root, destination); err == nil || !strings.Contains(err.Error(), "must be outside the source root") { + t.Fatalf("Materialize() error = %v, want physical source-containment rejection", err) + } + entries, err := os.ReadDir(insideSource) + if err != nil { + t.Fatal(err) + } + if len(entries) != 0 { + t.Fatalf("source-contained destination was mutated: %v", entries) + } +} + +func TestCaptureIncludesUntrackedAndIgnoresIgnoredFiles(t *testing.T) { + root := initializeRepository(t) + writeFile(t, filepath.Join(root, ".gitignore"), "ignored.txt\n") + writeFile(t, filepath.Join(root, "tracked.txt"), "tracked") + runGit(t, root, "add", ".gitignore", "tracked.txt") + runGit(t, root, "commit", "-m", "initial") + writeFile(t, filepath.Join(root, "untracked.txt"), "untracked") + writeFile(t, filepath.Join(root, "ignored.txt"), "ignored") + + snapshot, err := Capture(root) + if err != nil { + t.Fatalf("Capture() error = %v", err) + } + joined := strings.Join(snapshot.Paths, ",") + if !strings.Contains(joined, "untracked.txt") || strings.Contains(joined, "ignored.txt") { + t.Fatalf("snapshot paths = %v", snapshot.Paths) + } +} + +func TestValidRevisionAdmitsOnlyGitObjectIdentityAndOptionalSnapshotDigest(t *testing.T) { + valid := []string{ + strings.Repeat("a", 40), + strings.Repeat("b", 64), + strings.Repeat("c", 40) + "+worktree.sha256:" + strings.Repeat("d", 64), + } + for _, value := range valid { + if !ValidRevision(value) { + t.Fatalf("ValidRevision(%q) = false", value) + } + } + invalid := []string{ + "", + "caller-secret-sentinel", + strings.Repeat("A", 40), + strings.Repeat("a", 39), + strings.Repeat("a", 40) + "+worktree.sha256:invalid", + strings.Repeat("a", 40) + "+worktree.sha256:" + strings.Repeat("b", 64) + "+worktree.sha256:" + strings.Repeat("c", 64), + } + for _, value := range invalid { + if ValidRevision(value) { + t.Fatalf("ValidRevision(%q) = true", value) + } + } +} + +func TestValidateMaterializedRejectsSurplusFile(t *testing.T) { + root := initializeRepository(t) + writeFile(t, filepath.Join(root, "tracked.txt"), "tracked") + runGit(t, root, "add", "tracked.txt") + runGit(t, root, "commit", "-m", "initial") + destination := t.TempDir() + snapshot, err := Materialize(root, destination) + if err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(destination, "surplus.txt"), "surplus") + if err := ValidateMaterialized(destination, snapshot); err == nil || !strings.Contains(err.Error(), "inventory is stale") { + t.Fatalf("ValidateMaterialized() error = %v, want surplus-file rejection", err) + } +} + +func TestCaptureContextTerminatesCanceledGitProcessGroup(t *testing.T) { + bin := t.TempDir() + gitPath := filepath.Join(bin, "git") + if err := os.WriteFile(gitPath, []byte("#!/bin/sh\n/bin/sleep 10\n"), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin) + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + started := time.Now() + if _, err := CaptureContext(ctx, t.TempDir()); err == nil || !strings.Contains(err.Error(), "canceled") { + t.Fatalf("CaptureContext() error = %v, want cancellation", err) + } + if elapsed := time.Since(started); elapsed > 2*time.Second { + t.Fatalf("CaptureContext() cancellation took %s", elapsed) + } +} + +func TestCaptureContextTerminatesGitProcessGroupOnOutputOverflow(t *testing.T) { + bin := t.TempDir() + gitPath := filepath.Join(bin, "git") + script := "#!/bin/sh\n/bin/dd if=/dev/zero bs=17825792 count=1 1>&2\n/bin/sleep 10\n" + if err := os.WriteFile(gitPath, []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin) + started := time.Now() + if _, err := CaptureContext(context.Background(), t.TempDir()); err == nil || !strings.Contains(err.Error(), "output exceeds resource limit") { + t.Fatalf("CaptureContext() error = %v, want output-limit rejection", err) + } + if elapsed := time.Since(started); elapsed > 3*time.Second { + t.Fatalf("CaptureContext() output-limit termination took %s", elapsed) + } +} + +func TestCaptureRejectsSuccessfulGitDiagnosticsWithoutEcho(t *testing.T) { + bin := t.TempDir() + gitPath := filepath.Join(bin, "git") + const diagnostic = "sensitive diagnostic sentinel" + script := "#!/bin/sh\nprintf 'tracked.txt\\0'\nprintf '" + diagnostic + "' 1>&2\n" + if err := os.WriteFile(gitPath, []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin) + _, err := Capture(t.TempDir()) + if err == nil || !strings.Contains(err.Error(), "emitted diagnostics") { + t.Fatalf("Capture() error = %v, want diagnostics rejection", err) + } + if strings.Contains(err.Error(), diagnostic) { + t.Fatalf("Capture() leaked git diagnostics: %v", err) + } +} + +func initializeRepository(t *testing.T) string { + t.Helper() + root := t.TempDir() + runGit(t, root, "init", "-q") + runGit(t, root, "config", "user.email", "test@example.invalid") + runGit(t, root, "config", "user.name", "Test") + return root +} + +func runGit(t *testing.T, root string, args ...string) { + t.Helper() + command := exec.Command("git", args...) + command.Dir = root + if output, err := command.CombinedOutput(); err != nil { + t.Fatalf("git %v: %v\n%s", args, err, output) + } +} + +func writeFile(t *testing.T, path, content string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatalf("mkdir: %v", err) + } + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatalf("write %s: %v", path, err) + } +} diff --git a/proofkit/cli-contract.v2.json b/proofkit/cli-contract.v2.json index 5a99633..5a96297 100644 --- a/proofkit/cli-contract.v2.json +++ b/proofkit/cli-contract.v2.json @@ -2231,7 +2231,7 @@ "nativeSources": [ { "path": "internal/app", - "canonicalDigest": "sha256:b106e4c87900417f84f8bd999ddc67ac1d47bc54c2cb0bcb0e85993433761c82", + "canonicalDigest": "sha256:6e888333db0edc8a6eaeb91dad38b876f0bbee22a43c1bd2424e155e5a58dc88", "evidenceClass": "source_checkout" }, { @@ -5470,7 +5470,7 @@ "rootDefinitionDigest": "sha256:3c842174dff5361e7f83166469b832805e05aa314b073c16234b5b64e346281e", "nativeSource": { "path": "internal/app", - "canonicalDigest": "sha256:b106e4c87900417f84f8bd999ddc67ac1d47bc54c2cb0bcb0e85993433761c82", + "canonicalDigest": "sha256:6e888333db0edc8a6eaeb91dad38b876f0bbee22a43c1bd2424e155e5a58dc88", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -5499,7 +5499,7 @@ "rootDefinitionDigest": "sha256:0ea95e277ebe44cd2de42c29b47c38686ac0b6b390d8965367437b3fe138e209", "nativeSource": { "path": "internal/app", - "canonicalDigest": "sha256:b106e4c87900417f84f8bd999ddc67ac1d47bc54c2cb0bcb0e85993433761c82", + "canonicalDigest": "sha256:6e888333db0edc8a6eaeb91dad38b876f0bbee22a43c1bd2424e155e5a58dc88", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { diff --git a/proofkit/requirement-bindings.json b/proofkit/requirement-bindings.json index f306044..607e664 100644 --- a/proofkit/requirement-bindings.json +++ b/proofkit/requirement-bindings.json @@ -469,7 +469,7 @@ "claimLevel": "blocking", "proofState": "witness_backed", "nonClaims": [ - "This requirement does not claim semantic command coverage, line coverage, dead-code absence, exhaustive semantic completeness for any test, native witness execution, receipt freshness, or merge satisfaction." + "Successful selected tests do not prove assertion-branch execution, mutation adequacy, line coverage, dead-code absence, exhaustive command semantics, malicious-test resistance, producer authentication, receipt freshness, merge satisfaction, or production readiness." ] }, { @@ -2335,6 +2335,38 @@ "local-go" ] }, + { + "requirementId": "REQ-PROOFKIT-QUALITY-010", + "scenarioId": "proofkit.supply-chain-quality.artifact-file-boundary", + "witnessId": "proofkit.artifact-file.boundary", + "witnessKind": "contract", + "witnessPath": "internal/tools/artifactfile/file_test.go", + "witnessSelectors": [ + { + "selector": "TestOperationsRejectFinalSymlinkWithoutTargetMutation", + "command": "go test ./internal/tools/artifactfile -run '^TestOperationsRejectFinalSymlinkWithoutTargetMutation$'" + }, + { + "selector": "TestOperationsRejectSymlinkComponentsWithoutOutsideMutation", + "command": "go test ./internal/tools/artifactfile -run '^TestOperationsRejectSymlinkComponentsWithoutOutsideMutation$'" + }, + { + "selector": "TestReadBoundedRejectsUnrepresentableLimit", + "command": "go test ./internal/tools/artifactfile -run '^TestReadBoundedRejectsUnrepresentableLimit$'" + }, + { + "selector": "TestWriteReadAndRemoveRoundTrip", + "command": "go test ./internal/tools/artifactfile -run '^TestWriteReadAndRemoveRoundTrip$'" + } + ], + "commandIds": [ + "proofkit.coverage-metrics", + "proofkit.go-test" + ], + "environmentClasses": [ + "local-go" + ] + }, { "requirementId": "REQ-PROOFKIT-QUALITY-010", "scenarioId": "proofkit.supply-chain-quality.coverage-metrics", @@ -2349,6 +2381,126 @@ { "selector": "TestEachLinkageDeadZoneConjunctHasIndependentFalsifier", "command": "go test ./internal/tools/coveragemetrics -run '^TestEachLinkageDeadZoneConjunctHasIndependentFalsifier$'" + }, + { + "selector": "TestInvalidateMetricsFileRejectsSymlinkParentWithoutDeletingOutsideFile", + "command": "go test ./internal/tools/coveragemetrics -run '^TestInvalidateMetricsFileRejectsSymlinkParentWithoutDeletingOutsideFile$'" + }, + { + "selector": "TestWriteMetricsFileRejectsSymlinkEscapeWithoutMutation", + "command": "go test ./internal/tools/coveragemetrics -run '^TestWriteMetricsFileRejectsSymlinkEscapeWithoutMutation$'" + } + ], + "commandIds": [ + "proofkit.coverage-metrics", + "proofkit.go-test" + ], + "environmentClasses": [ + "local-go" + ] + }, + { + "requirementId": "REQ-PROOFKIT-QUALITY-010", + "scenarioId": "proofkit.supply-chain-quality.command-oracle-counterfeit-corpus", + "witnessId": "proofkit.command-oracle.counterfeit-corpus", + "witnessKind": "contract", + "witnessPath": "internal/tools/commandoracle/corpus_test.go", + "witnessSelectors": [ + { + "selector": "TestCounterfeitCorpusClosesRequiredAxes", + "command": "go test ./internal/tools/commandoracle -run '^TestCounterfeitCorpusClosesRequiredAxes$'" + }, + { + "selector": "TestCounterfeitCorpusClosureRejectsMissingRequiredAxes", + "command": "go test ./internal/tools/commandoracle -run '^TestCounterfeitCorpusClosureRejectsMissingRequiredAxes$'" + }, + { + "selector": "TestEachCounterfeitCaseProducesItsCheckedInDecision", + "command": "go test ./internal/tools/commandoracle -run '^TestEachCounterfeitCaseProducesItsCheckedInDecision$'" + } + ], + "commandIds": [ + "proofkit.coverage-metrics", + "proofkit.go-test" + ], + "environmentClasses": [ + "local-go" + ] + }, + { + "requirementId": "REQ-PROOFKIT-QUALITY-010", + "scenarioId": "proofkit.supply-chain-quality.command-oracle-execution-ledger", + "witnessId": "proofkit.command-oracle.execution-ledger", + "witnessKind": "contract", + "witnessPath": "internal/tools/commandoracle/execute_test.go", + "witnessSelectors": [ + { + "selector": "TestExecuteBindsMaterializedSourceCandidatesAndRuntimeEvents", + "command": "go test ./internal/tools/commandoracle -run '^TestExecuteBindsMaterializedSourceCandidatesAndRuntimeEvents$'" + }, + { + "selector": "TestRunGoTestsTerminatesOnContextDeadline", + "command": "go test ./internal/tools/commandoracle -run '^TestRunGoTestsTerminatesOnContextDeadline$'" + }, + { + "selector": "TestRunGoTestCommandTerminatesImmediatelyWhenStderrExceedsBound", + "command": "go test ./internal/tools/commandoracle -run '^TestRunGoTestCommandTerminatesImmediatelyWhenStderrExceedsBound$'" + }, + { + "selector": "TestRunGoTestsDoesNotExecuteCrossPackageNameMatches", + "command": "go test ./internal/tools/commandoracle -run '^TestRunGoTestsDoesNotExecuteCrossPackageNameMatches$'" + }, + { + "selector": "TestValidateCurrentRejectsProducerUnreachableCandidateProjection", + "command": "go test ./internal/tools/commandoracle -run '^TestValidateCurrentRejectsProducerUnreachableCandidateProjection$'" + } + ], + "commandIds": [ + "proofkit.coverage-metrics", + "proofkit.go-test" + ], + "environmentClasses": [ + "local-go" + ] + }, + { + "requirementId": "REQ-PROOFKIT-QUALITY-010", + "scenarioId": "proofkit.supply-chain-quality.command-oracle-source-snapshot", + "witnessId": "proofkit.command-oracle.source-snapshot", + "witnessKind": "contract", + "witnessPath": "internal/tools/repositorysnapshot/snapshot_test.go", + "witnessSelectors": [ + { + "selector": "TestCaptureContextTerminatesCanceledGitProcessGroup", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestCaptureContextTerminatesCanceledGitProcessGroup$'" + }, + { + "selector": "TestCaptureContextTerminatesGitProcessGroupOnOutputOverflow", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestCaptureContextTerminatesGitProcessGroupOnOutputOverflow$'" + }, + { + "selector": "TestCaptureRejectsSuccessfulGitDiagnosticsWithoutEcho", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestCaptureRejectsSuccessfulGitDiagnosticsWithoutEcho$'" + }, + { + "selector": "TestMaterializeBindsCopiedBytesAndRejectsLiveMutation", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestMaterializeBindsCopiedBytesAndRejectsLiveMutation$'" + }, + { + "selector": "TestMaterializeRejectsSymlinkAndNonEmptyDestination", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestMaterializeRejectsSymlinkAndNonEmptyDestination$'" + }, + { + "selector": "TestMaterializeRejectsSymlinkedDestinationInsideSource", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestMaterializeRejectsSymlinkedDestinationInsideSource$'" + }, + { + "selector": "TestValidRevisionAdmitsOnlyGitObjectIdentityAndOptionalSnapshotDigest", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestValidRevisionAdmitsOnlyGitObjectIdentityAndOptionalSnapshotDigest$'" + }, + { + "selector": "TestValidateMaterializedRejectsSurplusFile", + "command": "go test ./internal/tools/repositorysnapshot -run '^TestValidateMaterializedRejectsSurplusFile$'" } ], "commandIds": [ @@ -2639,6 +2791,14 @@ { "selector": "TestBuildInputFailsClosedForEachBlockingEvidenceClass", "command": "go test ./internal/tools/releasecloseoutinput -run '^TestBuildInputFailsClosedForEachBlockingEvidenceClass$'" + }, + { + "selector": "TestSelfEvidenceInvokesCurrentCommandOracleOwner", + "command": "go test ./internal/tools/releasecloseoutinput -run '^TestSelfEvidenceInvokesCurrentCommandOracleOwner$'" + }, + { + "selector": "TestSelfEvidenceRejectsProducerUnreachableCommandOracleRef", + "command": "go test ./internal/tools/releasecloseoutinput -run '^TestSelfEvidenceRejectsProducerUnreachableCommandOracleRef$'" } ], "commandIds": [ diff --git a/proofkit/witness-plan.json b/proofkit/witness-plan.json index 95ad99b..9e2283e 100644 --- a/proofkit/witness-plan.json +++ b/proofkit/witness-plan.json @@ -594,6 +594,11 @@ "credentialClass": "none", "cachePolicy": "disabled", "expectedArtifacts": [ + { + "kind": "report", + "path": "artifacts/proofkit/command-oracle-execution.json", + "required": true + }, { "kind": "report", "path": "artifacts/proofkit/coverage-metrics.json", @@ -920,12 +925,16 @@ "internal/app/command_registry.go", "internal/command", "internal/testsupport/commandcoverage", + "internal/tools/artifactfile", + "internal/tools/commandoracle", "internal/tools/coveragemetrics", + "internal/tools/repositorysnapshot", "proofkit/cli-contract.v2.json", "proofkit/requirement-bindings.json", "proofkit/witness-plan.json" ], "outputSelectors": [ + "artifacts/proofkit/command-oracle-execution.json", "artifacts/proofkit/coverage-metrics.json" ], "resourceReads": [