diff --git a/ADOPTION.md b/ADOPTION.md index 0bbaca3..b15f146 100644 --- a/ADOPTION.md +++ b/ADOPTION.md @@ -163,7 +163,7 @@ caller-owned capability or test observations Use `test-evidence-inventory --projection discovery-draft` only for explicit caller-owned test discovery facts. The command does not scan repositories, -execute tests, emit `semantic_falsifier` evidence, or close coverage. It emits +execute tests, emit execution-backed semantic evidence, or close coverage. It emits candidate inventory guidance with a non-strict candidate authority so an agent can ask the right owner questions and materialize strict inventory rows later. Candidate inventory diagnostics are rejected by strict inventory admission until diff --git a/BACKLOG.md b/BACKLOG.md index 18e81be..41f518d 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -48,7 +48,13 @@ records, generated release manifests, or the owning docs named above. | Status | ID | Scope | Completion condition | |---|---|---|---| -| NEXT | COVERAGE-01 | Replace command proof-route candidates with an owner-admitted executable oracle ledger; static route records are explicitly non-semantic. | `CommandCoverageInventory` consumes a separate execution-backed owner ledger whose rows bind `commandRef`, selector, concrete falsification event, assertion oracle, expected public outcome, and owner invariant; until then route metadata, prose, source markers, test existence, and failure-capable AST nodes emit only `proof_route_candidate`, candidate-route closure remains blocking, and missing `semantic_falsifier` evidence remains an explicit non-blocking metric and non-claim. | +| NEXT | COVERAGE-01 | Replace command proof-route candidates with an owner-admitted executable oracle ledger; static route records are explicitly non-semantic. | `CommandCoverageInventory` consumes a separate execution-backed owner ledger whose rows bind `commandRef`, selector, concrete falsification event, assertion oracle, expected public outcome, and owner invariant; an independently authored versioned counterfeit corpus covers every shipped policy, evidence class, identity coordinate, and single or correlated substitution axis with positive controls and exact expected decisions; until then route metadata, prose, legacy source markers, test existence, and failure-capable AST nodes emit only declared routes or `proof_route_candidate`, candidate-route closure remains blocking, and semantic execution evidence remains an explicit non-claim. | +| NEXT | COMPACT-01 | Replace compact proof caller labels and synthetic counts with an honest declaration-only schema/profile. | One atomic schema/profile cutover prefixes caller-owned proof and mutation fields with `declared`, removes checked/finding counts without independent evidence owners, rejects collapsed witness roles, updates every compact producer and consumer, and proves exact role and round-trip closure without claiming execution or assurance. | +| NEXT | SOURCE-MODEL-01 | Define one representation-neutral typed requirement-source v2 model before selecting a source syntax. | A private bounded model admits atomic requirement identities, grouped authoring, premises, scenarios, definitions, vocabulary, lifecycle, references, and deterministic normalization; an independently authored field/variant completeness manifest plus mutant corpus proves every normative field reaches each required downstream owner, with no production parser or persisted normalized mirror. | +| BLOCKED | SOURCE-CODEC-01 | Select at most one compact source codec without creating dual authority. | After `SOURCE-MODEL-01`, one versioned experiment manifest freezes disjoint role sets: the flat-v1 baseline control, grouped-model ablations, and exactly complete grouped-JSON plus at most one complete restricted-DSL codec candidate over the same model. Only codec candidates can win the predeclared replacement relation; controls and ablations measure causality and cannot become production grammars. A newly discovered candidate requires a new manifest version and complete experiment. A frozen corpus and strict `Replace(candidate, grouped-json)` predicate cover grammar completeness, safety, semantic parity, diagnostics, canonical bytes, review accuracy, token cost, diff amplification, parse/format cost, and unknowns. Every metric is classified exactly once by a versioned registry with role, direction, baseline pair, aggregation, material threshold, primary decision requirement, and missing-observation semantics; duplicate or unclassified metrics fail admission, hard constraints cannot trade off, report-only metrics cannot decide replacement, promised byte/token reductions must be materially better, and bounded diff/parse costs must be noninferior. If grouped JSON fails its hard gate, retain the current flat v1 source and perform no v2 cutover; otherwise select the restricted text candidate only when it is the unique strict replacement, while a tie, unknown, incomparability, or non-material improvement selects grouped JSON. The losing parser and formatter are deleted before experiment closeout, and production admits exactly one grammar. | +| BLOCKED | SOURCE-CUTOVER-01 | Migrate self-hosted requirement sources only after one codec, the typed v2 model, nested structural contracts, and the complete evidence counterfeit corpus pass their gates. | `COVERAGE-01` and `SCHEMA-01` are complete; a digest-bound clause ledger proves representation-only equality or owner-reviewed semantic decomposition for every legacy requirement; all bindings/scenarios/contracts/context/diff/graph/browser owners cut over atomically; v1 admission and the losing codec are removed; and active-v1 inventory is zero. | +| BLOCKED | SCHEMA-01 | Replace root-shape-only public contracts with one independent complete nested structural-contract owner. | A versioned schema owner covers nested fields, variants, cardinalities, bounds, enums, defaults, duplicate and unknown-field policy, and cross-field constraints; generated artifacts pass parity against an independently authored completeness manifest and mutant corpus without becoming semantic or policy authority. | +| BLOCKED | SOURCE-PILOT-01 | Validate the selected source-v2 model and agent routing against heterogeneous external repositories without mutating them. | At least two independent repository classes complete no-push dual runs whose frozen inputs compare incumbent and candidate mapping, diagnostics, token cost, authoring accuracy, proof-route gaps, and rollback; unresolved parity or authority gaps keep incumbent owners active. | | BLOCKED | GOVERNANCE-01 | Evaluate a generic explicit-inventory governance-observation command without promoting one consumer's policy into Proofkit; detailed candidate contract is retained in [issue #64](https://github.com/research-engineering/agentic-proofkit/issues/64). | A sanitized reproducible fixture detects one named failure without classifying its false-positive counterexample, existing owners are proven insufficient, and either a second independent consumer reproduces the predicate or the owner admits recurring first-consumer cost; otherwise retire the candidate. | | DEFERRED | VALUE-01 | Admit exact value-evidence comparisons only after a real producer and downstream consumer establish the public record boundary; detailed candidate contract is retained in [issue #65](https://github.com/research-engineering/agentic-proofkit/issues/65). | A real execution-receipt projection, baseline producer, and downstream consumer prove an exact producer-output-to-admission round trip plus compact/full-graph inclusion or an intentional omission non-claim; otherwise no public command is added. | | BLOCKED | RELOCATION-01 | Add provenance-bounded witness relocation candidates without introducing a second binding path or trusting a caller-authored prior digest; detailed candidate contract is retained in [issue #66](https://github.com/research-engineering/agentic-proofkit/issues/66). | An owner-admitted content-addressed baseline binds witness id, prior path and digest, source revision, evidence class, authentication non-claims, and freshness non-claims; the scanner then proves the zero/one/many match partition while remaining non-current until fresh execution evidence exists. | diff --git a/docs/proofkit-contract-map.md b/docs/proofkit-contract-map.md index d639d1e..501c154 100644 --- a/docs/proofkit-contract-map.md +++ b/docs/proofkit-contract-map.md @@ -42,7 +42,7 @@ owner boundaries. It is not a second command-family inventory. | Adoption and scaffolding | `init`, `adoption-contract-envelope`, `adoption-workflow-plan`, `adoption-checklist`, `adoption-doctor`, `gradual-adoption`, `gradual-adoption-bootstrap`, `gradual-adoption-guidance`, `capability-map-admission`, `pilot-admission`, `scaffold-profile-plan`, `scaffold-project-structure`, `stack-preset` | adoption intent, aggregate adoption contract envelope, checklist facts, target paths, owner routes, caller-extracted stale authority vocabulary facts, explicit pre-spec capability observations, pilot records, stack preset id, optional init preset id | dry-run route selection, aggregate contract-envelope admission, deterministic starter plans, checklist/report admission, bounded guidance envelopes, dry-run manifests, pre-spec trust-mode admission, adoption gap and stale-authority classification, pilot shape admission | final files, final requirements, rollout policy, text extraction from files, code observation extraction, pilot truth | selected child output, plan, report, seed packet, or agent envelope | | Requirement source | `capability-map-admission`, `requirement-authoring-plan`, `requirement-source-admission`, `requirement-source-transition`, `spec-overview-claims`, `requirement-spec-tree`, `requirement-spec-tree-view`, `requirement-source-view`, `requirement-browser-server` | `requirements.v1.json`, caller-owned capability maps, caller-owned authoring facts, overview claim extraction, explicit spec hierarchy, view options | candidate seed admission, candidate-only authoring packets, source-shape admission, lifecycle checks, explicit tree topology/source-ref admission, shared safe renderer fragments, presentation-only views | requirement meaning, extraction completeness, Markdown extraction completeness, hierarchy ownership, proof adequacy, file materialization | capability map report, authoring packet, source report, spec-tree report, rendered view, or browser presentation | | Requirement proof binding | `requirement-bindings`, `binding-partition`, `proof-slice`, `evidence-graph`, `requirement-proof-resolver`, `requirement-proof-source-set`, `requirement-proof-view`, `spec-proof-bundle-admission` | requirement records, bindings, witness commands, source-set facts, receipt reports, partition policy | graph validation, binding partition projection, compact slices, typed compact proof contract projections, resolver projections, bundle linkage checks | test semantics, witness execution, proof freshness, merge policy | proof report, partition report, slice, lookup graph, or view | -| Test inventory and coverage | `test-evidence-inventory`, `test-evidence-inventory --projection discovery-draft`, `test-evidence-inventory --normalized-inventory`, `requirement-coverage-input-compose`, `requirement-coverage-view`, `requirement-browser-server --view coverage` | caller-owned direct or source-set test inventory, caller-owned explicit test discovery facts, declared quality findings, requirement source, proof binding or compact proof contract, coverage universe, optional owner-invariant registry, aggregate coverage compose input | strict inventory/source-set admission, candidate-only discovery draft projection, fail-closed normalized inventory projection, deterministic coverage-view input composition from explicit facts, weak-oracle and declared-quality classification, bounded agent action guidance, requirement/test/command/owner-invariant joins, nonsemantic command-evidence classification, stable coverage failure/warning classifications, presentation-only coverage view | inventory completeness, weak-test truth, test discovery extraction, native test execution, receipt freshness, producer trust, merge policy | candidate inventory guidance, inventory report, normalized inventory data product, coverage-view input, coverage view, or browser presentation | +| Test inventory and coverage | `test-evidence-inventory`, `test-evidence-inventory --projection discovery-draft`, `test-evidence-inventory --normalized-inventory`, `requirement-coverage-input-compose`, `requirement-coverage-view`, `requirement-browser-server --view coverage` | caller-owned direct or source-set test inventory, caller-owned explicit test discovery facts, declared quality findings, requirement source, proof binding or compact proof contract, coverage universe, optional owner-invariant registry, aggregate coverage compose input | strict inventory/source-set admission, candidate-only discovery draft projection, fail-closed normalized inventory projection, deterministic coverage-view input composition from explicit facts, missing declared assertion-signal and declared-quality classification, bounded agent action guidance, requirement/test/command/owner-invariant joins, nonsemantic command-evidence classification, stable coverage failure/warning classifications, presentation-only coverage view | inventory completeness, oracle quality, test quality, test discovery extraction, native test execution, receipt freshness, producer trust, merge policy | candidate inventory guidance, inventory report, normalized inventory data product, coverage-view input, coverage view, or browser presentation | | Selective planning | `changed-path-set`, `requirement-impact-input-compose`, `impact`, `selective-gate-plan`, `selective-gate-evidence`, `selective-gate-obligation-decision-input`, `proof-obligation-algebra`, `obligation-decision` | changed paths, base/current requirement sources, base/current single-binding-per-requirement proof contracts, generated-artifact policy, local environment policy, proof-like path policy, scan obligation ownership, planned receipts, obligation routes, obligation algebra records | fail-closed impact input composition, fail-closed planning, receipt comparison, obligation algebra admission, bounded agent packets | git diff truth, repository scanning, command execution, producer trust, final admission | composed impact input, plan, evidence report, obligation algebra report, or obligation input | | Receipts and producers | `proof-receipt-admission`, `receipt-producer-admission`, `receipt-currentness-scope`, `receipt-trust-class`, `producer-policy-self-proof` | receipt sets, producer policy, scope/currentness facts, trust classes | receipt shape, producer/receipt compatibility, self-proof diagnostics | producer authentication, freshness policy, CI trust roots | receipt/provenance report | | Release and deployment | `release-authority`, `external-consumer`, `registry-consumer-proof-input-compose`, `registry-consumer`, `deployment-evidence-admission`, `completion-criteria`, `branch-authority`, `readiness-closeout` | package facts, tarball/registry facts, explicit primitive registry/install/smoke facts, deployment evidence, criteria, branch facts | artifact/channel boundary checks, registry-consumer input composition, release diagnostics, falsifiable criteria shape | package publication, registry fetch, package-manager execution, deployment, rollback, approval | composed input, release/deployment/readiness report | @@ -131,7 +131,7 @@ Semantic context routes are `requirement-context-compose`, | An agent needs a bounded semantic subset instead of whole specification files. | `requirement-context-compose --repo-root ` over an explicit catalog, then `requirement-context-slice` over the materialized snapshot. | The snapshot and slice are content-bound derived projections. Stop before inferring hierarchy, scanning ambient paths, treating omissions as absence, or promoting the slice to requirement, proof, freshness, or merge authority. | | Overview prose may contain durable claims. | `spec-overview-claims` | Escalate when normative claims are not tied to `REQ-*` records. | | Requirements have no verified proof route. | `requirement-bindings`, `witness-plan` from either an explicit `witness_command_catalog` or a complete `binding_witness_plan_input`, `proof-slice`, or `requirement-proof-resolver` | Stop before claiming proof adequacy; native witness semantics stay with the consumer. A binding-derived witness plan still needs caller-owned vocabulary and conservative command policy. | -| Tests or proof evidence need inventory. | `test-evidence-inventory`; use `--projection discovery-draft` only for explicit discovered-test facts, then `requirement-coverage-input-compose` when an aggregate `coverage_compose_input` exists, then `requirement-coverage-view` when a `coverage_view_input` exists | Compose only from explicit caller-owned facts. Discovery drafts are candidate-only and cannot close coverage. Use `failureClassifications[]`, `warningClassifications[]`, and `agentActionPlan[]` for machine routing. Escalate when tests are route-only, weak-oracle, unbound, or outside the caller-owned coverage universe. | +| Tests or proof evidence need inventory. | `test-evidence-inventory`; use `--projection discovery-draft` only for explicit discovered-test facts, then `requirement-coverage-input-compose` when an aggregate `coverage_compose_input` exists, then `requirement-coverage-view` when a `coverage_view_input` exists | Compose only from explicit caller-owned facts. Discovery drafts are candidate-only and cannot close coverage. Use `failureClassifications[]`, `warningClassifications[]`, and `agentActionPlan[]` for machine routing. Escalate when tests are route-only, lack a declared assertion signal, are unbound, or are outside the caller-owned coverage universe. Proofkit reports declared signals; the consumer owns oracle-quality and test-quality conclusions. | | A change set is known. | `changed-path-set`, optionally `requirement-impact-input-compose`, `impact`, then `selective-gate-plan --agent-envelope` | Raw `knownChangedPaths` in `agent-route` are diagnostic only. Materialize a caller-owned `changed_path_set`, compose a caller-owned `impact_input` before `impact`, and compose a caller-owned `selective_gate_plan_input` before `selective-gate-plan`. Use `scanObligation` to name whether a `text-policy`, `secret-scan`, or caller-owned external scanner is required. Fail closed on unknown scope, dynamic edges, missing owner routes, unbound proof-like paths, or full-gate escalation. | | Caller-owned file contents need secret-like text detection. | `secret-scan` | Provide explicit sorted file inventory with content. Stop before claiming repository-wide discovery, credential validity, provider ingestion, merge readiness, or replacement of GitHub secret scanning. | | Does a TypeScript package public API match a caller-owned manifest? | `agent-route` with `goal: "verify_typescript_public_api"` and explicit `typescript_public_api_manifest` plus `typescript_public_api_repo_root`, then `typescript-public-api-surfaces --repo-root ` | The manifest must name each referenced `package.json`, sorted-unique export conditions, and a non-JSX `.ts`, `.mts`, or `.cts` `sourcePath` whose canonical target has the same admitted extension class. The bounded scanner accepts only the fail-closed export grammar in `proofkit/cli-contract.v2.json`; it does not parse unrestricted TypeScript or TSX, infer conventional layouts, or prove compiler output provenance, checkout freshness, package-manager truth, or merge readiness. | diff --git a/docs/specs/proofkit-consumer-infra-retirement/overview.md b/docs/specs/proofkit-consumer-infra-retirement/overview.md index 5aafb2a..307e1fa 100644 --- a/docs/specs/proofkit-consumer-infra-retirement/overview.md +++ b/docs/specs/proofkit-consumer-infra-retirement/overview.md @@ -30,9 +30,11 @@ approval, rollout approval, and production decisions. gradual adoption, and release-channel scenarios to existing Proofkit primitives through bounded structured command refs. - `REQ-PROOFKIT-RETIRE-006`: migration parity admission validates - caller-provided parity evidence shape, source/target closure, typed - equivalence dimensions, and matched digest equality without owning evidence - authenticity, freshness, semantic correctness, or retirement approval. + caller-provided parity-claim shape, source/target closure, typed equivalence + dimensions, consistency between caller-declared status and supplied digests, + and lossless record-level non-claim projection without owning digest + computation, evidence authenticity, freshness, semantic correctness, or + retirement approval. - `REQ-PROOFKIT-RETIRE-007`: gradual adoption guidance keeps caller-provided candidate boundaries advisory, exposes owner-review questions, routes mode and checked-scope semantics through one private owner, and fails closed for diff --git a/docs/specs/proofkit-consumer-infra-retirement/requirements.v1.json b/docs/specs/proofkit-consumer-infra-retirement/requirements.v1.json index 7f77b78..dd984d7 100644 --- a/docs/specs/proofkit-consumer-infra-retirement/requirements.v1.json +++ b/docs/specs/proofkit-consumer-infra-retirement/requirements.v1.json @@ -143,7 +143,7 @@ { "requirementId": "REQ-PROOFKIT-RETIRE-006", "ownerId": "proofkit.consumer-infra-retirement", - "invariant": "Migration parity admission validates caller-provided parity evidence shape, source/target closure, typed equivalence dimensions, and matched digest equality without owning evidence authenticity, freshness, semantic correctness, or retirement approval.", + "invariant": "Migration parity admission validates caller-provided parity-claim shape, source/target closure, typed equivalence dimensions, consistency between caller-declared status and caller-supplied digests, and lossless record-level non-claim projection; statuses, admitted parity claim refs, and status-derived parity counters remain explicitly caller-declared, structural and failure counts remain Proofkit-computed admission facts, and the report does not own digest computation, evidence authenticity, freshness, semantic correctness, or retirement approval.", "claimLevel": "blocking", "riskClass": "high", "proofBindingRefs": [ diff --git a/docs/specs/proofkit-spec-proof-core/overview.md b/docs/specs/proofkit-spec-proof-core/overview.md index 31b70c5..0d442da 100644 --- a/docs/specs/proofkit-spec-proof-core/overview.md +++ b/docs/specs/proofkit-spec-proof-core/overview.md @@ -49,8 +49,21 @@ execution receipts, and merge policy. owners, scope widening, fabricated normalized inventory, non-strict discovery candidate inventory, nonsemantic command evidence, and declared dead zones in selected-owner or full-repository scopes, and do not treat - proof routes, discovery drafts, nonsemantic command evidence, normalized - projections, composed inputs, or rendered views as semantic test coverage. + caller-authored route, falsifier, oracle-signal, or falsifier-supersession + metadata as selector resolution, oracle review, execution, freshness, trust, + sensitivity, dominance proof, verified coverage, or assurance closure; + discovery classifies only declared assertion-signal presence and coverage + re-admission requires the versioned exact root and row shapes, canonical row + order, exact row metadata, command-owned non-claims, exact missing-inventory + diagnostics, owner-scoped compact scenario identities, and a coverage basis + that commits to selected owners, full-repository out-of-scope source + identities, and the complete retained inventory projection; it retains + complete inventory entries without selected parents, replays owner-scope and + inventory-owned failure semantics, and rejects missing projection fields, + contradictory repeated projections, one-sided retained parent references, + compact scenario-to-route drift, and unowned or inconsistent state/class + vocabulary. The coverage basis proves internal projection consistency, not + source authenticity or producer identity. - `REQ-PROOFKIT-SPEC-007`: caller-owned inputs are admitted into canonical immutable records before any proof, policy, route, rendering, or report decision consumes them. @@ -80,9 +93,10 @@ execution receipts, and merge policy. policy. - `REQ-PROOFKIT-SPEC-012`: requirement authoring plans package caller-provided design, implementation, PR, code, test, and clarification facts into - candidate-only requirement updates, delegate structural checks to requirement - source admission and transition, and keep owner review/materialization outside - Proofkit. + candidate-only requirement updates, preserve complete canonically ordered + admitted authoring references, consume requirement records only through + canonical owner projections, delegate structural checks to requirement source + admission and transition, and keep owner review/materialization outside Proofkit. - `REQ-PROOFKIT-SPEC-013`: repeated cross-command proof vocabulary has one private owner for proof and selective-edge state while command-local and caller-owned vocabularies stay outside the shared owner. diff --git a/docs/specs/proofkit-spec-proof-core/requirements.v1.json b/docs/specs/proofkit-spec-proof-core/requirements.v1.json index 3da0d43..b767b31 100644 --- a/docs/specs/proofkit-spec-proof-core/requirements.v1.json +++ b/docs/specs/proofkit-spec-proof-core/requirements.v1.json @@ -143,7 +143,7 @@ { "requirementId": "REQ-PROOFKIT-SPEC-006", "ownerId": "proofkit.spec-proof-core", - "invariant": "Test evidence inventory, explicit test-discovery draft projections, proof-binding-derived inventory projections, normalized inventory projections, coverage-view input composition, and requirement coverage views classify or assemble caller-owned test-to-requirement, command, witness, owner-invariant, quality-finding, and declared-surface routes with stable machine-readable failure and warning classifications plus bounded agent action guidance while preserving source-set entry provenance, failing closed on structured selector/sourcePath drift, command-ref collisions, missing requirement owners, scope widening, fabricated normalized inventory, non-strict discovery candidate inventory, nonsemantic command evidence, and declared dead zones in selected-owner or full-repository scopes, and without treating proof routes, discovery drafts, nonsemantic command evidence, normalized projections, composed inputs, generated guidance, or rendered views as semantic test coverage.", + "invariant": "Test evidence inventory, explicit test-discovery draft projections, proof-binding-derived inventory projections, normalized inventory projections, coverage-view input composition, and requirement coverage views classify or assemble caller-owned test-to-requirement, command, witness, owner-invariant, quality-finding, and declared-surface routes with stable machine-readable failure and warning classifications plus bounded agent action guidance while preserving source-set entry provenance, failing closed on structured selector/sourcePath drift, command-ref collisions, missing requirement owners, scope widening, fabricated normalized inventory, non-strict discovery candidate inventory, nonsemantic command evidence, and declared dead zones in selected-owner or full-repository scopes; caller-authored route, falsifier, oracle-signal, and falsifier-supersession metadata emits only explicit declaration or mapping states, discovery classifies only the presence of admitted assertion-signal categories, coverage output re-admission requires the versioned exact root and row shapes, canonical row order, exact row metadata, command-owned non-claims, exact missing-inventory diagnostics, owner-scoped compact scenario identities, and a coverage basis committing to selected owners, full-repository out-of-scope source identities, and the complete retained inventory projection, retains complete inventory entries even when no selected parent references them, replays owner-scope and inventory-owned failure semantics, and rejects missing projected fields, contradictory repeated projections, one-sided retained parent references, compact scenario-to-route projection drift, and unowned or inconsistent state/class vocabulary, and no such metadata or coverage-basis commitment can emit source authenticity, producer identity, selector-resolution, oracle-review, execution, freshness, trust, sensitivity, dominance-proof, verified, covered, or assurance-closure facts.", "claimLevel": "blocking", "riskClass": "high", "proofBindingRefs": [ @@ -153,7 +153,7 @@ "NC-PROOFKIT-SPEC-006" ], "nonClaims": [ - "This requirement does not claim repository scanning, native test execution, proof freshness, receipt authenticity, merge approval, or global coverage completeness outside the caller-provided universe." + "This requirement does not claim repository scanning, source authenticity, producer identity, selector resolution, oracle quality, native test execution, proof freshness, receipt authenticity, trust, sensitivity, assurance closure, merge approval, or global coverage completeness outside the caller-provided universe." ], "lifecycle": { "state": "active", @@ -305,7 +305,7 @@ { "requirementId": "REQ-PROOFKIT-SPEC-012", "ownerId": "proofkit.spec-proof-core", - "invariant": "Requirement authoring plans admit caller-provided design, implementation, pull request, code, test, and clarification facts into candidate-only requirement updates, delegate candidate source and lifecycle checks to requirement-source admission and transition, and emit owner-review plans without reading repositories, writing files, inferring requirement meaning, or approving promotion.", + "invariant": "Requirement authoring plans admit caller-provided design, implementation, pull request, code, test, and clarification facts into candidate-only requirement updates, preserve every admitted authoring reference as a canonically ordered lossless projection of refId, kind, path, optional digest, summary, and nonClaims, consume current and candidate requirement records only through canonical requirement-source owner projections, delegate candidate source and lifecycle checks to requirement-source admission and transition, and emit owner-review plans without reading repositories, writing files, inferring requirement meaning, or approving promotion.", "claimLevel": "blocking", "riskClass": "high", "proofBindingRefs": [ diff --git a/docs/specs/proofkit-supply-chain-quality/overview.md b/docs/specs/proofkit-supply-chain-quality/overview.md index d75f1d0..87a81e9 100644 --- a/docs/specs/proofkit-supply-chain-quality/overview.md +++ b/docs/specs/proofkit-supply-chain-quality/overview.md @@ -102,7 +102,12 @@ vulnerability absence, or consumer rollout safety by itself. fields after validation. - `REQ-PROOFKIT-QUALITY-015`: the package gate includes an admitted release closeout completion-criteria report so unit tests alone cannot satisfy - release closeout. + release closeout, and coverage-metric re-admission uses overflow-safe exact + producer relations plus the complete command inventory, and compares both + coverage command projections with the actual `cli-contract.v2.json` command + inventory from the same source snapshot, so neither an impossible count + partition nor a coordinated same-size command substitution can satisfy + closeout. - `REQ-PROOFKIT-QUALITY-016`: release platform targets use one private owner that projects platform suffixes, Go build targets, npm OS/CPU metadata, package tar entries, Python wheel tags, PyPI candidate completeness, diff --git a/docs/specs/proofkit-supply-chain-quality/requirements.v1.json b/docs/specs/proofkit-supply-chain-quality/requirements.v1.json index 79c4048..399b71b 100644 --- a/docs/specs/proofkit-supply-chain-quality/requirements.v1.json +++ b/docs/specs/proofkit-supply-chain-quality/requirements.v1.json @@ -129,7 +129,7 @@ { "requirementId": "REQ-PROOFKIT-QUALITY-010", "ownerId": "proofkit.supply-chain-quality", - "invariant": "Coverage metrics report requirement, binding, witness, CLI inventory linkage, and descriptor-owned command proof-route candidates from admitted test-evidence-inventory rows, failing closed independently for each requirement/proof linkage dead-zone class and each missing-candidate, unknown-candidate-ref, unknown-semantic-ref, contract-only, or route-only command-route class; source-checkout witness selectors must resolve to valid functions in active Go test files with exact executable commands and a failure-capable assertion candidate outside literal-dead branches, uninvoked closures, and skip-bearing helper paths, critical anti-vacuity scenarios must retain their exact closed selector inventories, and failed command-route inventory admission also fails closed, while static route metadata, prose, source markers, test existence, and failure-capable syntax remain candidate evidence and cannot satisfy semantic_falsifier coverage.", + "invariant": "Coverage metrics report requirement, binding, witness, CLI inventory linkage, and descriptor-owned command proof-route candidates from admitted test-evidence-inventory rows, failing closed independently for each requirement/proof linkage dead-zone class and each missing-candidate, unknown-candidate-ref, unknown-declared-semantic-route-ref, contract-only, or route-only command-route class; source-checkout witness selectors must resolve to valid functions in active Go test files with exact executable commands and a failure-capable assertion candidate outside literal-dead branches, uninvoked closures, and skip-bearing helper paths, critical anti-vacuity scenarios must retain their exact closed selector inventories, and failed command-route inventory admission also fails closed, while static route metadata, prose, source markers, test existence, and failure-capable syntax remain candidate evidence and cannot become execution-backed semantic evidence.", "claimLevel": "blocking", "riskClass": "medium", "proofBindingRefs": ["proofkit/requirement-bindings.json"], @@ -194,7 +194,7 @@ { "requirementId": "REQ-PROOFKIT-QUALITY-015", "ownerId": "proofkit.supply-chain-quality", - "invariant": "The package gate includes an admitted release closeout completion-criteria report that classifies package artifacts, Python wrappers, release manifest, SBOM, channel scope, and self-hosting evidence instead of allowing unit tests alone to satisfy release closeout.", + "invariant": "The package gate includes an admitted release closeout completion-criteria report that classifies package artifacts, Python wrappers, release manifest, SBOM, channel scope, and self-hosting evidence instead of allowing unit tests alone to satisfy release closeout; coverage-metric re-admission uses overflow-safe relations to require the exact producer projection between admitted entries, routes, route classes, proof-route candidates, the complete command inventory, and commands missing declaration-only routes, and compares both coverage command projections with the actual cli-contract.v2.json command inventory from the same source snapshot.", "claimLevel": "blocking", "riskClass": "high", "proofBindingRefs": ["proofkit/requirement-bindings.json"], diff --git a/internal/app/cli_abi_test.go b/internal/app/cli_abi_test.go index a5b630a..1266896 100644 --- a/internal/app/cli_abi_test.go +++ b/internal/app/cli_abi_test.go @@ -142,13 +142,16 @@ func TestCLIABIGoldenCorpus(t *testing.T) { }, }, { - name: "requirement authoring plan emits candidate-only preview JSON", - args: []string{"requirement-authoring-plan", "--input", "-"}, - stdin: cliRequirementAuthoringPlanInput(), - wantStatus: 0, - wantStdoutJSON: true, + name: "requirement authoring plan emits candidate-only preview JSON", + args: []string{"requirement-authoring-plan", "--input", "-"}, + stdin: cliRequirementAuthoringPlanInput(), + wantStatus: 0, + wantStdoutJSON: true, + wantOutputVariant: "01-root", wantStdoutHas: []string{ `"planKind": "proofkit.requirement-authoring-plan"`, + `"schemaVersion": 2`, + `"authoringRefs": [`, `"authority": "candidate_only"`, `"ownerReviewRequired": true`, `"state": "passed"`, @@ -336,7 +339,7 @@ func TestCLIABIGoldenCorpus(t *testing.T) { `"candidate_only:test.cli.discovery.test_missing_auth"`, }, wantStdoutNotHas: []string{ - `"evidenceClass": "semantic_falsifier"`, + `"evidenceClass": "declared_semantic_falsifier_route"`, }, }, { @@ -348,7 +351,7 @@ func TestCLIABIGoldenCorpus(t *testing.T) { wantInputVariant: "01-direct-inventory", wantOutputVariant: "03-report", wantStdoutHas: []string{ - `"missing_semantic_anchor:test.cli.semantic"`, + `"missing_declared_route_anchor:test.cli.semantic"`, `"state": "failed"`, }, }, @@ -375,7 +378,7 @@ func TestCLIABIGoldenCorpus(t *testing.T) { wantOutputVariant: "02-report", wantStdoutHas: []string{ `"viewKind": "proofkit.requirement-coverage-view"`, - `"covered_by_semantic_falsifier"`, + `"mapped_to_declared_semantic_falsifier_route"`, `"state": "passed"`, }, }, @@ -425,7 +428,7 @@ func TestCLIABIGoldenCorpus(t *testing.T) { wantStatus: 1, wantStdoutJSON: true, wantStdoutHas: []string{ - `"classificationId": "missing_semantic_test"`, + `"classificationId": "missing_declared_test_route"`, `"missing_test_inventory:REQ-PROOFKIT-CLI-COVERAGE-001"`, `"state": "failed"`, }, @@ -515,6 +518,55 @@ func TestCLIABIGoldenCorpus(t *testing.T) { } } +func TestRequirementAuthoringPlanOutputUsesVersionedRootShape(t *testing.T) { + var input map[string]any + if err := json.Unmarshal([]byte(cliRequirementAuthoringPlanInput()), &input); err != nil { + t.Fatalf("decode authoring plan fixture: %v", err) + } + input["authoringRefs"] = append(input["authoringRefs"].([]any), map[string]any{ + "refId": "proofkit.cli.authoring.test-summary", "kind": "test_summary", + "path": "artifacts/cli-authoring-tests.json", "digest": nil, + "summary": "The CLI test summary supplies undigested candidate context.", + "nonClaims": []any{"CLI authoring test-summary ref is label-only."}, + }) + encodedInput, err := json.Marshal(input) + if err != nil { + t.Fatalf("encode authoring plan fixture: %v", err) + } + raw := runCLI(t, []string{"requirement-authoring-plan", "--input", "-"}, string(encodedInput)) + var output any + if err := json.Unmarshal(raw, &output); err != nil { + t.Fatalf("requirement authoring plan output must be JSON: %v\n%s", err, raw) + } + assertPublicCLIRootVariant(t, "requirement-authoring-plan", "output", "01-root", output) + record := jsonObject(t, output) + assertNumberField(t, record, "schemaVersion", 2) + refs := jsonArrayField(t, record, "authoringRefs") + if len(refs) != 2 { + t.Fatalf("authoringRefs=%#v, want exactly two ordered admitted refs", refs) + } + ref := jsonObject(t, refs[0]) + assertStringField(t, ref, "digest", "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa") + assertStringField(t, ref, "kind", "design_doc") + assertStringField(t, ref, "path", "docs/designs/cli-authoring.md") + assertStringField(t, ref, "refId", "proofkit.cli.authoring.design") + assertStringField(t, ref, "summary", "The CLI fixture design proposes one durable candidate invariant.") + if len(jsonArrayField(t, ref, "nonClaims")) != 1 { + t.Fatalf("authoring ref nonClaims=%#v, want exact projected boundary", ref["nonClaims"]) + } + second := jsonObject(t, refs[1]) + if second["digest"] != nil { + t.Fatalf("second authoring ref digest=%#v, want explicit null", second["digest"]) + } + assertStringField(t, second, "kind", "test_summary") + assertStringField(t, second, "path", "artifacts/cli-authoring-tests.json") + assertStringField(t, second, "refId", "proofkit.cli.authoring.test-summary") + assertStringField(t, second, "summary", "The CLI test summary supplies undigested candidate context.") + if len(jsonArrayField(t, second, "nonClaims")) != 1 { + t.Fatalf("second authoring ref nonClaims=%#v, want exact projected boundary", second["nonClaims"]) + } +} + func TestSelfCheckOutputUsesExactRootShape(t *testing.T) { var stdout bytes.Buffer var stderr bytes.Buffer @@ -1086,19 +1138,7 @@ func TestRequirementBrowserOneShotCLIOutputVariants(t *testing.T) { go func() { result <- Run(ctx, args, bytes.NewReader(input), &submittedStdout, &submittedStderr) }() - var browserURL string - deadline := time.Now().Add(3 * time.Second) - for browserURL == "" && time.Now().Before(deadline) { - content, readErr := os.ReadFile(urlFile) - if readErr == nil { - browserURL = strings.TrimSpace(string(content)) - break - } - time.Sleep(10 * time.Millisecond) - } - if browserURL == "" { - t.Fatal("browser launcher did not receive one-shot URL") - } + browserURL := waitForBrowserLauncherURL(t, urlFile, result, &submittedStdout, &submittedStderr) response, err := http.Get(browserURL) if err != nil { t.Fatal(err) @@ -1161,6 +1201,29 @@ func TestRequirementBrowserOneShotCLIOutputVariants(t *testing.T) { assertPublicCLIRootVariant(t, "requirement-browser-server", "output", "02-one-shot-terminal", terminal) } +func waitForBrowserLauncherURL(t *testing.T, urlFile string, result <-chan int, stdout, stderr *bytes.Buffer) string { + t.Helper() + ticker := time.NewTicker(10 * time.Millisecond) + defer ticker.Stop() + timer := time.NewTimer(10 * time.Second) + defer timer.Stop() + for { + select { + case status := <-result: + t.Fatalf("one-shot CLI exited before browser launch: status=%d stdout=%s stderr=%s", status, stdout.String(), stderr.String()) + case <-ticker.C: + content, err := os.ReadFile(urlFile) + if err == nil { + if browserURL := strings.TrimSpace(string(content)); browserURL != "" { + return browserURL + } + } + case <-timer.C: + t.Fatal("browser launcher did not receive one-shot URL") + } + } +} + func TestAdoptionDoctorCLIABI(t *testing.T) { commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.018061704360080936767110902235369680299779598229946624164723476412925521966788") cases := []struct { @@ -2250,7 +2313,7 @@ func cliImpactDemo(demoID string, stackDiverse bool) map[string]any { } func cliTestEvidenceInventory() string { - return `{"schemaVersion":1,"inventoryId":"proofkit.cli.inventory","authority":"caller_owned_inventory","entries":[{"testId":"test.cli.semantic","selector":"go test ./internal/app -run TestCLI","sourcePath":"internal/app/cli_abi_test.go","ownerId":"proofkit.cli","evidenceClass":"semantic_falsifier","requirementRefs":["REQ-PROOFKIT-CLI-001"],"ownerInvariantRefs":[],"commandRefs":["proofkit.cli.command"],"witnessRefs":["proofkit.cli.witness"],"falsifier":{"falsifierId":"falsifier.cli.semantic","negativeCaseId":"case.cli.semantic","wrongImplementationClassId":"wrong.cli.semantic","dominanceGroup":"cli.semantic","supersedes":[]},"oracle":{"oracleId":"oracle.cli.semantic","oracleKind":"negative_exit_and_diagnostic","expectedPublicOutcome":"failed report with diagnostic","assertionSummary":"The CLI emits a failed report on invalid semantic coverage."},"nonClaims":[]}],"nonClaims":["CLI ABI fixture does not execute native tests."]}` + return `{"schemaVersion":1,"inventoryId":"proofkit.cli.inventory","authority":"caller_owned_inventory","entries":[{"testId":"test.cli.semantic","selector":"go test ./internal/app -run TestCLI","sourcePath":"internal/app/cli_abi_test.go","ownerId":"proofkit.cli","evidenceClass":"declared_semantic_falsifier_route","requirementRefs":["REQ-PROOFKIT-CLI-001"],"ownerInvariantRefs":[],"commandRefs":["proofkit.cli.command"],"witnessRefs":["proofkit.cli.witness"],"falsifier":{"falsifierId":"falsifier.cli.semantic","negativeCaseId":"case.cli.semantic","wrongImplementationClassId":"wrong.cli.semantic","dominanceGroup":"cli.semantic","supersedes":[]},"oracle":{"oracleId":"oracle.cli.semantic","oracleKind":"negative_exit_and_diagnostic","expectedPublicOutcome":"failed report with diagnostic","assertionSummary":"The CLI emits a failed report on invalid semantic coverage."},"nonClaims":[]}],"nonClaims":["CLI ABI fixture does not execute native tests."]}` } func cliSecretScanInput(content string) string { @@ -2288,7 +2351,7 @@ func cliTestDiscoveryDraftInput() string { } func cliCoverageInventory() string { - return `{"schemaVersion":1,"inventoryId":"proofkit.cli.coverage.inventory","authority":"caller_owned_inventory","entries":[{"testId":"test.cli.coverage.semantic","selector":"go test ./internal/app -run TestCoverage","sourcePath":"internal/app/cli_abi_test.go","ownerId":"proofkit.cli.coverage","evidenceClass":"semantic_falsifier","requirementRefs":["REQ-PROOFKIT-CLI-COVERAGE-001"],"ownerInvariantRefs":[],"commandRefs":["proofkit.cli.coverage.command"],"witnessRefs":["proofkit.cli.coverage.witness"],"falsifier":{"falsifierId":"falsifier.cli.coverage","negativeCaseId":"case.cli.coverage","wrongImplementationClassId":"wrong.cli.coverage","dominanceGroup":"cli.coverage","supersedes":[]},"oracle":{"oracleId":"oracle.cli.coverage","oracleKind":"negative_exit_and_diagnostic","expectedPublicOutcome":"failed report with diagnostic","assertionSummary":"Missing inventory leaves coverage failed."},"nonClaims":[]}],"nonClaims":["CLI coverage inventory fixture does not execute native tests."]}` + return `{"schemaVersion":1,"inventoryId":"proofkit.cli.coverage.inventory","authority":"caller_owned_inventory","entries":[{"testId":"test.cli.coverage.semantic","selector":"go test ./internal/app -run TestCoverage","sourcePath":"internal/app/cli_abi_test.go","ownerId":"proofkit.cli.coverage","evidenceClass":"declared_semantic_falsifier_route","requirementRefs":["REQ-PROOFKIT-CLI-COVERAGE-001"],"ownerInvariantRefs":[],"commandRefs":["proofkit.cli.coverage.command"],"witnessRefs":["proofkit.cli.coverage.witness"],"falsifier":{"falsifierId":"falsifier.cli.coverage","negativeCaseId":"case.cli.coverage","wrongImplementationClassId":"wrong.cli.coverage","dominanceGroup":"cli.coverage","supersedes":[]},"oracle":{"oracleId":"oracle.cli.coverage","oracleKind":"negative_exit_and_diagnostic","expectedPublicOutcome":"failed report with diagnostic","assertionSummary":"Missing inventory leaves coverage failed."},"nonClaims":[]}],"nonClaims":["CLI coverage inventory fixture does not execute native tests."]}` } func cliCoverageInput(inventory string) string { diff --git a/internal/app/cli_contract_test.go b/internal/app/cli_contract_test.go index 75894b2..33abc5c 100644 --- a/internal/app/cli_contract_test.go +++ b/internal/app/cli_contract_test.go @@ -22,7 +22,7 @@ import ( ) const ( - cliContractPublicABISHA256 = "472517950d99dc3c6ce772ac6f63ca5be2a4ff3692aac3025dd7f4adb09289b3" + cliContractPublicABISHA256 = "6a3b6504546419e9198af89ea90af19d543dac26af975aeef1075b4ff7d88491" maxAggregateFileReadBytesForContractTest = 64 << 20 maxPackageManifestBytesForContractTest = 256 << 10 maxSourceFileBytesForContractTest = 8 << 20 @@ -1981,16 +1981,48 @@ func TestRequirementCoverageViewContractDescribesMachineClassifications(t *testi "declared_dead_zone", "failed_test_inventory", "missing_requirement_binding", - "missing_semantic_test", + "missing_declared_test_route", "nonsemantic_command_evidence", "nonsemantic_governance_evidence", "not_applicable_with_reason", "owner_scope_violation", + "proof_route_candidate_only", "routing_smoke_only", + "unclassified_gap", "unknown_reference", }, "requirement coverage classification ids") } +func TestRequirementCoverageViewBreakingRootUsesVersionedOutputContract(t *testing.T) { + contract := readCLIContract(t) + definitions := cliContractDefinitionMap(t, contract.ContractDefinitions) + for _, command := range contract.Commands { + if command.Command != "requirement-coverage-view" { + continue + } + output := canonicalJSONValue(t, command.OutputContract).(map[string]any) + if output["contractId"] != "proofkit.requirement-coverage-view.output.v2" || output["schemaVersion"] != float64(2) { + t.Fatalf("requirement coverage output identity=%#v, want versioned v2 contract", output) + } + definitionID := output["rootDefinitionRef"].(string) + definition := definitions[definitionID] + if definitionID != "proofkit.requirement-coverage-view.output.v2.root-shape" || definition["schemaVersion"] != float64(2) { + t.Fatalf("requirement coverage root definition=%#v, want versioned v2 root", definition) + } + for _, rawVariant := range definition["fieldTree"].(map[string]any)["variants"].([]any) { + variant := rawVariant.(map[string]any) + required := stringsFromAny(variant["requiredFields"].([]any)) + if variant["variantId"] == "02-report" && + slices.Contains(required, "coverageBasis") && + slices.Contains(required, "unmappedTests") { + return + } + } + t.Fatal("requirement coverage v2 report root must require coverageBasis and unmappedTests") + } + t.Fatal("requirement-coverage-view missing from CLI contract") +} + func TestTestEvidenceInventoryContractDescribesMachineClassifications(t *testing.T) { contract := readCLIContract(t) var inventory *cliContractCommand @@ -2038,7 +2070,7 @@ func TestTestEvidenceInventoryContractDescribesMachineClassifications(t *testing "invalid_falsifier_supersession", "missing_edge", "missing_executable_command_ref", - "missing_semantic_anchor", + "missing_declared_route_anchor", "mock_tests_mock", "over_broad_integration", "proof_route_candidate", @@ -2047,7 +2079,7 @@ func TestTestEvidenceInventoryContractDescribesMachineClassifications(t *testing "snapshot_without_oracle", "tautology", "unasserted_diagnostic", - "weak_or_empty_oracle", + "incomplete_declared_oracle_metadata", "wrong_boundary", "wrong_evidence_boundary", }, "test evidence inventory classification ids") diff --git a/internal/app/cli_output_witness_contract_test.go b/internal/app/cli_output_witness_contract_test.go index d4d3afb..3363d66 100644 --- a/internal/app/cli_output_witness_contract_test.go +++ b/internal/app/cli_output_witness_contract_test.go @@ -378,6 +378,14 @@ func rootDistinctOutputContractExpectations() []rootDistinctOutputContractExpect SelectorTest: "TestStandaloneMultiVariantCommandsUseExactRootShapes", ExecutableCommand: "go test ./internal/app -run '^TestStandaloneMultiVariantCommandsUseExactRootShapes$'", }, + { + Command: "requirement-authoring-plan", + NativeSourceForm: "nativeSource", + NativeSourcePaths: []string{"internal/command/requirementauthoringplan"}, + SelectorPath: "internal/app/cli_abi_test.go", + SelectorTest: "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", + ExecutableCommand: "go test ./internal/app -run '^TestRequirementAuthoringPlanOutputUsesVersionedRootShape$'", + }, { Command: "self-check", NativeSourceForm: "nativeSource", @@ -396,6 +404,11 @@ func rootDistinctOutputBindingMappings() []rootDistinctOutputBindingMapping { ScenarioID: "proofkit.package-boundary.cli-output-root-witnesses", SelectorTest: "TestAdoptionContractEnvelopeCLIABI", }, + { + RequirementID: "REQ-PROOFKIT-PACKAGE-002", + ScenarioID: "proofkit.package-boundary.cli-output-root-witnesses", + SelectorTest: "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", + }, { RequirementID: "REQ-PROOFKIT-PACKAGE-002", ScenarioID: "proofkit.package-boundary.cli-output-root-witnesses", @@ -411,6 +424,11 @@ func rootDistinctOutputBindingMappings() []rootDistinctOutputBindingMapping { ScenarioID: "proofkit.supply-chain-quality.cli-abi-golden", SelectorTest: "TestAdoptionContractEnvelopeCLIABI", }, + { + RequirementID: "REQ-PROOFKIT-QUALITY-004", + ScenarioID: "proofkit.supply-chain-quality.cli-abi-golden", + SelectorTest: "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", + }, { RequirementID: "REQ-PROOFKIT-QUALITY-004", ScenarioID: "proofkit.supply-chain-quality.cli-abi-golden", diff --git a/internal/app/command_contract_generated.go b/internal/app/command_contract_generated.go index cd858f9..5aa3f98 100644 --- a/internal/app/command_contract_generated.go +++ b/internal/app/command_contract_generated.go @@ -1,7 +1,7 @@ // Code generated by internal/tools/commandcontractgen; DO NOT EDIT. package app -const commandContractSourceSHA256 = "a6d6faaa1da035747847d7e2f9232857aa9e342413a1f7c2408ce2ca4cbe832b" +const commandContractSourceSHA256 = "3d2fab7d4b8da24fb1c105ce8814b531926c50dee9a594eacad953f8af897132" type generatedCommandContractMetadata struct { InputContractSHA256 string @@ -15,7 +15,7 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "adoption-contract-envelope": {InputContractSHA256: "sha256:33204c558b8d2d41ede42cbbebaa079ea73cdbd9568b45c0cfd5c61f7a45ccd5", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.adoption-contract-envelope.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e66dcc00c0a1aee44c4a221f96e2f009918b67cfd9da4b352e087bcf780e4c7c", FlagChoices: map[string][]string{}}, "adoption-doctor": {InputContractSHA256: "sha256:43c741dab52b0e0204eace2b693c091aeb9c280b4d8c27e956dc524188bd9f42", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.adoption-doctor.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:b7d03277879eaec1d928b870a51242d399f17ee03e16dd7fc355af1109b4831f", FlagChoices: map[string][]string{}}, "adoption-workflow-plan": {InputContractSHA256: "sha256:b32ae67179d7b6dcf1ea66cb6b2b2691c8367ce2e2be367619b65973166da55c", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.adoption-workflow-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:8d64cb53ebd0307e3cebc3435286a3d2a1ee8a0ad6f7514fc0fb3285db0f565b", FlagChoices: map[string][]string{}}, - "agent-route": {InputContractSHA256: "sha256:ad0a30069885ee74212d7cd0bed6461ff6d835127b7a7d10f24cb9f941ad2d54", InputSchemaSummary: []string{"availableInputs", "browserMode", "goal", "knownChangedPaths", "mode", "nonClaims", "observedReports", "openBrowser", "routeId", "schemaVersion", "root-shape-only definition proofkit.agent-route.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:805ffb86061551b826671fdab70f3c7396d6ba26d07c44eadb9853d63a35083b", FlagChoices: map[string][]string{}}, + "agent-route": {InputContractSHA256: "sha256:e18596c4c440322c5596f2d4981a2f3f91bef5136d3c41bd7bf800e627d46f7e", InputSchemaSummary: []string{"availableInputs", "browserMode", "goal", "knownChangedPaths", "mode", "nonClaims", "observedReports", "openBrowser", "routeId", "schemaVersion", "root-shape-only definition proofkit.agent-route.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:a39e222f92f21a54f5bad5dec84fb68dadd6b06d83cb9b46d0b942fe944288b3", FlagChoices: map[string][]string{}}, "binding-partition": {InputContractSHA256: "sha256:366ad082045af52b2ac6604f18626d0f285b2db73b45d9a82687b8d3b0d2b3fd", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.binding-partition.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:52840879e13a00ef9a4abaad6cdb33000511674d5f9003fb56f387fdf58fadc8", FlagChoices: map[string][]string{}}, "branch-authority": {InputContractSHA256: "sha256:8a3ed74978898593fbdbf1f7fa684dae450fbd9019edcd60d07f818d63363ed4", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.branch-authority.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:3c7dc74842299b92cd5baf57cc8666e9415963091359e5faf654e28da89561f1", FlagChoices: map[string][]string{}}, "capability-map-admission": {InputContractSHA256: "sha256:efe87497cf6a32cfd1f9c9c191393cac8781d1e9211914a8a7b6aaa44e7f371d", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.capability-map-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:bfb32535fd51bed168111912acb1651527897fe59a592119dc0032ce9286c9db", FlagChoices: map[string][]string{}}, @@ -25,7 +25,7 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "custom-rule-boundary": {InputContractSHA256: "sha256:18119bc01230b9cb01cbb56bab531c1d2153c689aea3863b24de4f345eec5ede", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.custom-rule-boundary.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:8a80f33974c60f960daa4f9803f2b5dfb34ff323418f5f0ab75d7b348653a34a", FlagChoices: map[string][]string{}}, "deployment-evidence-admission": {InputContractSHA256: "sha256:c3472b87497be14c107dcf95f2c98594d21ec88e514919fe8fb7da858206421d", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.deployment-evidence-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:818fb2b7dbf67be1da27cc62e78c7bd7fdaddaa38fa5ec80f0cad64fbcb60d90", FlagChoices: map[string][]string{}}, "document-lifecycle-boundary": {InputContractSHA256: "sha256:393836f0ce8e23df351e01d7dec221908ae76fa9b1a045b96fce229eecc844ed", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.document-lifecycle-boundary.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:65c7c0338df60d83d71df9fce3a7e60ce0b0d86819a38e2a6ffeac58b4abc934", FlagChoices: map[string][]string{}}, - "evidence-graph": {InputContractSHA256: "sha256:82c4f14a91ae1b77d1e338e4a1390eec45093cb184830c5dab11e0d2d192c55e", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.evidence-graph.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:254ee56ab897df021856db23b62e9366ed2dfe26b821efa0da9eff4ef83a2592", FlagChoices: map[string][]string{}}, + "evidence-graph": {InputContractSHA256: "sha256:93732159b1e76b44a2e3ddf9344aaf10f073308e2c816b973ef28e7db814d360", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.evidence-graph.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:b261b9dd0c25c297b7542c4fb7486c02f1ece4308428f26f3e964db8af63ef31", FlagChoices: map[string][]string{}}, "external-consumer": {InputContractSHA256: "sha256:195d4f09c6fb03004f5ba3d6bad9aea7728ecd9aa876e7e30d521ff961a3e931", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.external-consumer.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:2615a60f4cd21a865ed604aca94bf5066adb4bc8f7a3a2345bd53d181429822f", FlagChoices: map[string][]string{}}, "gradual-adoption": {InputContractSHA256: "sha256:b82598422d2e18c7160d827836e964db6e9b0c2b14a119c1d4f5c4d18779c17f", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.gradual-adoption.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:75310c4f53b1fd3838d03e8988e7eb5faef61e757149a6f556033e23ddc488ff", FlagChoices: map[string][]string{}}, "gradual-adoption-bootstrap": {InputContractSHA256: "sha256:453bab6dc3f3c84ab9cd7d375966b1cd881b53f8e8414da4aa837f8215db6f19", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.gradual-adoption-bootstrap.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:c00c79676575f2c5903321e87b4b494a3b6d77219b19552701cecde469db5427", FlagChoices: map[string][]string{}}, @@ -34,15 +34,15 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "impact": {InputContractSHA256: "sha256:c67d90fde422b44e765e46712e99a2e32af64caf3eaf98abc3992ec805204153", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.impact.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f3a528a18045c6c47b9b4eb199272c7b9e322657c8307e051eec49533140d4c6", FlagChoices: map[string][]string{}}, "init": {InputContractSHA256: "", InputSchemaSummary: []string(nil), OutputContractSHA256: "sha256:3e59a3002327c759e5e747f8baacaa63a4d6784e1a1c520f0a54e01af3f2faa0", FlagChoices: map[string][]string{}}, "json-report-cli-adapter-source": {InputContractSHA256: "", InputSchemaSummary: []string(nil), OutputContractSHA256: "sha256:5fcab8ffc599c54045ee04e5eb5abbe0175791e74d5f7edb2940e89424c5b4aa", FlagChoices: map[string][]string{}}, - "migration-parity-admission": {InputContractSHA256: "sha256:767c6e333db22837c4accb3e3d44b259fb5b2f54c783351e148ec7b0fbd86802", InputSchemaSummary: []string{"schemaVersion=1", "paritySetId", "sourceProofOwners[]", "targetProofkitRefs[]", "parityRecords[]", "nonClaims[]", "root-shape-only definition proofkit.migration-parity-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:2cba2943433dfbc2efc1350cede9340452b609158d24c5c622f0a68ef1fc43b6", FlagChoices: map[string][]string{}}, + "migration-parity-admission": {InputContractSHA256: "sha256:0b36c0e68da3b857dac4b13e7b3bd523052459106133aa8c908a4352682e6c05", InputSchemaSummary: []string{"schemaVersion=1", "paritySetId", "sourceProofOwners[]", "targetProofkitRefs[]", "parityRecords[]", "nonClaims[]", "root-shape-only definition proofkit.migration-parity-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:8e0f8af2b205817f018b0fe133fe789661caa29007695e036bfcab63c1830f47", FlagChoices: map[string][]string{}}, "migration-plan": {InputContractSHA256: "sha256:58a62759a634101ce2ca9218184175134bbe5633328e1b23797b94c19fc9b11a", InputSchemaSummary: []string{"schemaVersion=1", "migrationId", "sourceProofOwners[]", "targetProofkitRefs[]", "parityEvidenceRefs[]", "retainedOwners[]", "retirementCandidates[]", "followUpCommands[]", "nonClaims[]", "root-shape-only definition proofkit.migration-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f14f0381e9dc241357c346315b95b03ef5b23f1d1bbc3b00f111fbe1515ed3ff", FlagChoices: map[string][]string{}}, "obligation-decision": {InputContractSHA256: "sha256:1dea2ed5c5066451d6d49b815cea99df2cdae2ef05d42fed16c8aeb45eb7f445", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.obligation-decision.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:96dc074f611bcc12e511bc803c548e4df623e2de869d3add29a3ea6386e04330", FlagChoices: map[string][]string{}}, "package-runtime-dependency-admission": {InputContractSHA256: "sha256:fc85887af9b8fcd899d245f0db30b2f2f68609822fc268126bf999082bb4115f", InputSchemaSummary: []string{"schemaVersion=1", "reportId", "expectedDependencySpec", "expectedLockfileIntegrity", "expectedPackageName", "expectedPackageVersion", "admissibleLocations{}", "packageResolution{}", "nonClaims[]", "root-shape-only definition proofkit.package-runtime-dependency-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:c012032e8c8212fd50bc2e85669cc610609ca2124ebc992c9e88f44a1ad2d5fc", FlagChoices: map[string][]string{}}, - "pilot-admission": {InputContractSHA256: "sha256:6c4d6fb7ba99cb8da806826584807a723891955c83c04d0054a848e303e5ca9a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.pilot-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:d280e4144b31e45d8a47043acfc0821874fca8f518c55ab63c10041e3a15cb29", FlagChoices: map[string][]string{}}, + "pilot-admission": {InputContractSHA256: "sha256:6c4d6fb7ba99cb8da806826584807a723891955c83c04d0054a848e303e5ca9a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.pilot-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:1603d385db8a493287b4394a3bdad99e027cb87cc043c6c42b132c2616b1a804", FlagChoices: map[string][]string{}}, "producer-policy-self-proof": {InputContractSHA256: "sha256:d48e18826000c8d415f3c44b6c686e1da6ed962ef7ca36c9f705de8c68d034f9", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.producer-policy-self-proof.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e82a3989a743f8babc6069f7af82b1dd1ea62bad8dbb18d95e105b36f74e4276", FlagChoices: map[string][]string{}}, "proof-obligation-algebra": {InputContractSHA256: "sha256:4f176b6bc9bdbd0d96d65c071d66447d246665bda7a23269e7927f1d0b80b043", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.proof-obligation-algebra.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f9ee9e56b349756c55856a2dab198e1ad85db70a468c38e3aeca73cfe2ed66f6", FlagChoices: map[string][]string{}}, "proof-receipt-admission": {InputContractSHA256: "sha256:7cb4c4fb60c8b5a37109bbd8c00d567749f7d181bbc905d8bc58155f139c44cb", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.proof-receipt-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:3f802ac3fac6762ede51f0e0a151f16dc10b4a20344a3887b3ee8bae43ce94f2", FlagChoices: map[string][]string{}}, - "proof-slice": {InputContractSHA256: "sha256:c6abf98e38371a2afdd005b7d17317aa97fbae7a5aa9672367ee8ea4a53b9258", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.proof-slice.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:73e5f9db8bf966c9818b6581b10c6b20167af4ab1c4b33fa619c040a165125c7", FlagChoices: map[string][]string{}}, + "proof-slice": {InputContractSHA256: "sha256:07d7a2433ecfb3dc6a5016a39cc0d874d8589e23d9d155e83acfc2b542a2bf31", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.proof-slice.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:95d0c2dd22dc6ac632f51e28010184d55400521589c2e853403056e2c69b8a72", FlagChoices: map[string][]string{}}, "readiness-closeout": {InputContractSHA256: "sha256:4f427c1d0cefb00133d0d9fdb15f75ca9d12a26746632914e82bf72710883b9b", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.readiness-closeout.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:427a227aa59d60739bc7bdea03363ece95070520dc154fff1f063e174027cc5e", FlagChoices: map[string][]string{}}, "receipt-currentness-scope": {InputContractSHA256: "sha256:a3787eaacabc8902e90a39fe7fa179df3464a93829991006f720d2bd30572f06", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.receipt-currentness-scope.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:007e38673d5e8bb4a5c9447a835a7692e544b0f6e1f077d6f4f109fd08e3bf2a", FlagChoices: map[string][]string{}}, "receipt-producer-admission": {InputContractSHA256: "sha256:676aa03b2331a094e287dd3f2dfad3a74403b6f8dadb17e420b446aac0b3592c", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.receipt-producer-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:626b910bb8110901d6769c0ec2a14d216d9ca422aaf2f73ef5fc851c60b0847a", FlagChoices: map[string][]string{}}, @@ -52,19 +52,19 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "release-authority": {InputContractSHA256: "sha256:807f53ceab20f949fda99c10448f8490e16b22ae5b8915c9f471337321f5ffef", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.release-authority.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:7b01b165b79c37bbee95f272b2ab6e7707ec9c3fd542a2a5a820b676810adb86", FlagChoices: map[string][]string{}}, "rendered-artifact-freshness": {InputContractSHA256: "sha256:be4f53ef1307b4c16bb15a945f8021473b5a215961f3d38f6f591a0240da91f3", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.rendered-artifact-freshness.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:c9a763142d11daca913672b0bab517767d35e275cd7c8ccb2bb360e6fc8f7425", FlagChoices: map[string][]string{}}, "repo-profile-admission": {InputContractSHA256: "sha256:3a7331d66195dbdc9f672d380efe8fdb9d1d2e36a764b8bc912dccdd81b0e965", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.repo-profile-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:36d2116fa144aa86d7b9d0ac59b89ad04fb97c85a11f3f65efb7311506761fbd", FlagChoices: map[string][]string{}}, - "requirement-authoring-plan": {InputContractSHA256: "sha256:9167f6ea1e888c196c14799f30a7657cf7e74a4af53f15ef61b44b9ae82cc5ba", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-authoring-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:6cca126c3295537f0c7a73892e8c8778bbd10e442290e75dd45b833373312a46", FlagChoices: map[string][]string{}}, - "requirement-bindings": {InputContractSHA256: "sha256:c9fc55b5b8d67849adb8a10b441b73f33a9eb1b6b6afeee7ec698874e1f9cd50", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-bindings.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f61039fafcdb2ced7655c1be0bd9b46d689878f27aebdf12975c8e9f32cbd14e", FlagChoices: map[string][]string{}}, + "requirement-authoring-plan": {InputContractSHA256: "sha256:208d7d47109dee1ec355ae3970937690ae528a9cc0cb0eb885d7cc72d843f1e8", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-authoring-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e995d289c4a44310add784bbafa5a3a50ec305c3809a89506dd3b49914fbe28f", FlagChoices: map[string][]string{}}, + "requirement-bindings": {InputContractSHA256: "sha256:df5df45c82742c81ef7c24d32fea3802911b9f6b56c011393dcc6a89ca5b1908", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-bindings.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e6405501bc78168bf44f2cacff77cd3c86f8bbe5b4d3c7a00f6f71b11f0db706", FlagChoices: map[string][]string{}}, "requirement-browser-server": {InputContractSHA256: "sha256:29df21bfe8abc6fade2951d9080280bfc93d490ead2f432510be7bdf573567f8", InputSchemaSummary: []string{"workspace mode: schemaVersion=2", "workspace mode: workspaceId", "workspace mode: context=proofkit.requirement-context schemaVersion=2 with strict v1 adapter", "workspace mode: diffInput=proofkit.requirement-semantic-diff-input schemaVersion=2 (optional)", "workspace mode: graphInput=proofkit.requirement-traceability-graph-input schemaVersion=2 (optional)", "--session-mode values: browse|one-shot-question", "one-shot-question requires --view workspace --serve --open", "--session-timeout-seconds is 1..7200 and requires one-shot-question", "source|proof|coverage|spec-tree modes retain their owner input contracts", "root-shape-only definition proofkit.requirement-browser-server.input.v2.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:16e741d88e5ede4271c5e769c724e72164fcba461f0c6c17d285f318f8e03005", FlagChoices: map[string][]string{}}, "requirement-context-compose": {InputContractSHA256: "sha256:04603ae61b50c44e47dde2d11f4b730bb5e91f53645bcda6dbda80b708ef2781", InputSchemaSummary: []string{"schemaVersion=1", "catalogId", "specTree.path", "requirementSources[] (non-empty)", "requirementSources[].nodeId", "requirementSources[].path", "expectedSourceDigest (optional sha256 ref)", "proofBinding.path (optional)", "coverage.path (optional)", "exact catalog paths only; no discovery", "root-shape-only definition proofkit.requirement-context-compose.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:499413fd829de7f275f624a13f2de51afca9a6c201066eb69d95487d9abcab0b", FlagChoices: map[string][]string{}}, "requirement-context-slice": {InputContractSHA256: "sha256:4962dbf8bdde43384ceb375620e6b00f759157a4e6ceb1614d6e58a9f33ee583", InputSchemaSummary: []string{"schemaVersion=1", "sliceId", "context=proofkit.requirement-context schemaVersion=2 with strict v1 adapter", "query.profile=routing|specification|proof|coverage|review", "query.nodeIds[]|requirementIds[]|ownerIds[]|lifecycleStates[]", "query.maxDepth=0..512", "query.maxNodes=1..4096", "query.maxRequirements=1..16384", "root-shape-only definition proofkit.requirement-context-slice.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:93460f60611eeaf2a13d6e8779841fe78746e1fde9db7fa5e41c91c3b6b49ca4", FlagChoices: map[string][]string{}}, "requirement-coverage-input-compose": {InputContractSHA256: "sha256:26e46a1438f5633a835f4afaef293bed5417b13f4cd4c0a752d02659fcd094c6", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-coverage-input-compose.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:4e246b49fd99038d4b1df9ce484ef2214c3129769df2aaf67b398367a451dd86", FlagChoices: map[string][]string{}}, - "requirement-coverage-view": {InputContractSHA256: "sha256:a503fa4070bb4eae8200412a6cd6516ceefea07fb50453e17103f2d6bab5960c", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-coverage-view.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:b8b8ab6d7f0d78a33e81b3e6b37c8f7cb93f6d24ff745e6435128d38abb3c395", FlagChoices: map[string][]string{}}, + "requirement-coverage-view": {InputContractSHA256: "sha256:68341bf9ea46251209de4a51d5f7477e7b9be82472fdc0f49e647f385a41b64c", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-coverage-view.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e27fdd2c6771c75947670555c4c1aeb15024baf4b9913b8e10f4f27d7a3411e0", FlagChoices: map[string][]string{}}, "requirement-impact-input-compose": {InputContractSHA256: "sha256:9c85648f3ec078390a807a97e5184ebab82438ec3a0915fb6049ab741f35c3bb", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-impact-input-compose.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:27efae46bedaa3f7f9132f2542344b0d16db195d5c78f09a28079d42ce4037e5", FlagChoices: map[string][]string{}}, - "requirement-proof-resolver": {InputContractSHA256: "sha256:6b032c893c770f260d72976b1ead28455e2775988771eee85cb8065bcdb7a91d", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-proof-resolver.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:99cf48212bfbfc4667a4951c0b4f4480701bfe5715028b6b05db28ecc0b0ae50", FlagChoices: map[string][]string{}}, + "requirement-proof-resolver": {InputContractSHA256: "sha256:704ff9da8feb8fa5a251c48aca7c044000667793f5fc35f61ebf61ff37c4a59e", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-proof-resolver.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:eedaea9d9070198652b748f0d23685a19ff08377b3296c06706e15bfa6d51652", FlagChoices: map[string][]string{}}, "requirement-proof-source-set": {InputContractSHA256: "sha256:3ffe27b751445e4c525513ef2953f33fda2db49b4c84f6b9e93efc67b645a814", InputSchemaSummary: []string{"canonicalEnvelope", "sourceSet", "sources", "root-shape-only definition proofkit.requirement-proof-source-set.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:c74a57fd4ada0a06c0fafd31ea1aedb20035c67e8e7ef0fc413ae8bf086143b3", FlagChoices: map[string][]string{}}, "requirement-proof-view": {InputContractSHA256: "sha256:b0c2455d77a9b47380e0d6551f0f67bfe098b8ec22239b035164263446b29e1c", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-proof-view.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:59ab83ea52a0736f7435ceee8b2d324308e7c088c9dae56ebb289bf3750babe9", FlagChoices: map[string][]string{}}, "requirement-semantic-diff": {InputContractSHA256: "sha256:e0829b5576210214e67d86911b6b7c1313f4cacdfbf476fa19503702e4d03031", InputSchemaSummary: []string{"schemaVersion=2", "diffId", "baseContext=proofkit.requirement-context schemaVersion=2", "currentContext=proofkit.requirement-context schemaVersion=2", "strict schemaVersion=1 adapter requires two v1 contexts", "query.requirementIds[] (optional)", "query.ownerIds[] (optional)", "query.maxChanges=1..8192 (optional)", "root-shape-only definition proofkit.requirement-semantic-diff.input.v2.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:581b7585767d7cf55f0e2f891155e04f26ec6a0871b9b97cda8aa0b9a52ea770", FlagChoices: map[string][]string{}}, - "requirement-source-admission": {InputContractSHA256: "sha256:748da7b4f6bba55877cfe51efcbd85fbdab2f2329a9d9668ea2905713c80c417", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-source-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:2f6a2db4163b1829bd4cdf27890477abd77697d70dfd591943d47b7be798a90e", FlagChoices: map[string][]string{}}, + "requirement-source-admission": {InputContractSHA256: "sha256:f9afe7bb6ef0b2a1f32209f02805f91ee17bd7db9f072aa54c3975afb9a37180", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-source-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:aed5c873f09004435a68490ad33b0d289c286a3e67d3d150389ce638fad2b727", FlagChoices: map[string][]string{}}, "requirement-source-transition": {InputContractSHA256: "sha256:fad33de2e47d31a81b2dd1a84214d061dd2b7ec88a5ba0153287aebb29b19bd8", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-source-transition.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:aedf88574794d4916fd1d8b234538dbfbf358461b4d117ea4d0380e7890182b7", FlagChoices: map[string][]string{}}, "requirement-source-view": {InputContractSHA256: "sha256:0819889f9bfaddefe0555250612ef5f4d9172899b04d427d48b0420d765c00ad", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.requirement-source-view.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:2f29bd1edea7c3e930a143c2f96c229a0a199e52c89367302c414759bf660c44", FlagChoices: map[string][]string{}}, "requirement-spec-tree": {InputContractSHA256: "sha256:96876589778a1cf1bc3f41fa33ad86de502db05886ba620ecbee59225060e315", InputSchemaSummary: []string{"schemaVersion", "treeId", "rootNodeId", "callerAnnotations", "nodes", "edges", "overlays", "root-shape-only definition proofkit.requirement-spec-tree.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:a027461411b8614288186df2c014a9e5303f905f0fb4cfc9b3ed6439790ae547", FlagChoices: map[string][]string{}}, @@ -76,11 +76,11 @@ var generatedCommandContractMetadataByName = map[string]generatedCommandContract "selective-gate-evidence": {InputContractSHA256: "sha256:8aa178ab7ca7c475c23707bc4e15fd3f9f8d57acf6f6dcf279677e7769a45586", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.selective-gate-evidence.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:723569262bb85d9674b2a78d3bcb6e9f4cab229b71e8c784ff1b804a7fcade71", FlagChoices: map[string][]string{}}, "selective-gate-obligation-decision-input": {InputContractSHA256: "sha256:85761fcbc0ea94239d55bf379d0592a6ca814e6612a2d609a651f6cdaf8ca10a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.selective-gate-obligation-decision-input.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:ab9dddabe975238d7019266c43350afa2df1a61d4c2eb7bc23afd520b588a2da", FlagChoices: map[string][]string{}}, "selective-gate-plan": {InputContractSHA256: "sha256:5293a5a4c7d8426cf637e6f8d252095ca0eb1714365bb89bec83307b778c678a", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.selective-gate-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:d7bffed853af5595af08b03859be01c283a3bdff1b3502d94ddc190889977647", FlagChoices: map[string][]string{}}, - "self-check": {InputContractSHA256: "sha256:f452e8e75fa5a4ce349274b295d66ebf5aa92556fb1331cda32131e94eb251a5", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.self-check.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:9b23a41a960fe3b7e717635f77f43870bdceec2921f08bc0a788eebd33a5ff86", FlagChoices: map[string][]string{}}, + "self-check": {InputContractSHA256: "sha256:dee7c7a0b11b427553814e3bc1925391118f079018c6c97b940cf9fc2ae8450f", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.self-check.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:7062eb2e46470c0bbcbdea1239e747dc3e3de76b782d3ea509caafe2d0653f29", FlagChoices: map[string][]string{}}, "spec-overview-claims": {InputContractSHA256: "sha256:2490dcd34ba7485e13f8f33e8a288a0463c4c52cc6b0d82c57777466927e49a4", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.spec-overview-claims.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:554f3a7020e9820ccb90672629fd769c52b2f298f356040aa3b0a817666cbfbf", FlagChoices: map[string][]string{}}, "spec-proof-bundle-admission": {InputContractSHA256: "sha256:6b6c2875b6476e63a1911e7d6112d9999df2babbee969f84abc4c9e4b470c933", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.spec-proof-bundle-admission.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:e9e0eb66cebca3b99fe5036fb2e7327a9284934ed76f58818d18094d0546fc52", FlagChoices: map[string][]string{}}, "stack-preset": {InputContractSHA256: "", InputSchemaSummary: []string(nil), OutputContractSHA256: "sha256:f495e9ade4e1e7af7a8f8b2059f7611cc016e6080363afccf76d8dfc2dbc6d2d", FlagChoices: map[string][]string{"--preset": []string{"agentic_runtime_repo", "generated_docs_contract_repo", "python_service", "python_typescript_service", "typescript_monorepo", "typescript_workspace"}}}, - "test-evidence-inventory": {InputContractSHA256: "sha256:9f2217986ec85f017b16075749c53ae1b01b348bc9a6b66db423a6c3564e4528", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.test-evidence-inventory.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:4db9063936db590c97f33fd47b6fbafcf974ec3dab1f21ffaad37f7b9549e68c", FlagChoices: map[string][]string{}}, + "test-evidence-inventory": {InputContractSHA256: "sha256:f785149c9fac0077796d7307761963294d81225ad2f068ee74006881cb17e481", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.test-evidence-inventory.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:f6f2c606457ec7d3fa6a6b6c907487d2ae10ce5858aeaa6a13cd8ad82a556494", FlagChoices: map[string][]string{}}, "text-policy": {InputContractSHA256: "sha256:d686238fdba68d22a701030ecbfaa5d6fcb7229b3d12638b45c5473670eadd35", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.text-policy.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:ca8635ae272627e37453e03c4d535904757c73fea96d4c55defbc1d647b1f80e", FlagChoices: map[string][]string{}}, "typescript-public-api-surfaces": {InputContractSHA256: "sha256:e8e55a53390c50fe0d8e669907318474c237327f2f094b3c78908b4fd6941035", InputSchemaSummary: []string{"schemaVersion=1", "machineContract=public_api_surfaces", "entries[].packageManifestPath", "entries[].packageName", "entries[].exportKey", "entries[].exportConditions[] (non-empty, sorted unique by condition)", "entries[].exportConditions[].condition", "entries[].exportConditions[].path", "entries[].exportConditions[].sourcePath (declared and canonical target .ts/.mts/.cts)", "entries[].runtimeExports[]", "entries[].typeExports[]", "entries[].deniedExportKeys[] (optional)", "sourceGrammar=fail_closed_restricted_typescript_exports_v1", "maxSourceFileBytes=8388608", "maxPackageManifestBytes=262144", "maxAggregateFileReadBytes=67108864", "root-shape-only definition proofkit.typescript-public-api-surfaces.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:2f05ea8073e7a2d4a08de962461f5fb31a78f14e77f9ce2406e3ce40ec7beb68", FlagChoices: map[string][]string{}}, "witness-plan": {InputContractSHA256: "sha256:7814c5d27487a361bac77045afe32c6449c24881f04d5496da7182b3f2c0c1ee", InputSchemaSummary: []string{"schemaVersion=1", "root-shape-only definition proofkit.witness-plan.input.v1.root-shape; nested fields, types, and cardinalities are non-claims"}, OutputContractSHA256: "sha256:722c577bf5cf1dab8c9da8d18fc0634e1b9a6471aa5ca0b05f55b730cdd0d303", FlagChoices: map[string][]string{}}, diff --git a/internal/app/command_coverage_routes.go b/internal/app/command_coverage_routes.go index df755bd..a382c42 100644 --- a/internal/app/command_coverage_routes.go +++ b/internal/app/command_coverage_routes.go @@ -9,6 +9,9 @@ import ( "strings" ) +// These private marker names predate the declaration-only public contract. +// Their projection remains proof_route_candidate until COVERAGE-01 supplies an +// independently admitted execution-backed oracle ledger. type commandCoverageRoute struct { file string kind string @@ -44,7 +47,6 @@ type CommandCoverageSummary struct { CommandRef string ProofRouteCandidateCount int RouteCount int - SemanticRouteCount int RouteSmokeCount int } @@ -84,7 +86,7 @@ var commandCoverageRoutes = map[string][]commandCoverageRoute{ "impact": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/impact/impact_test.go", "TestBuildRoutesChangedRecordToObligationAndRejectsUnboundProofChange", semanticRouteProof("impact.build_routes_changed_record_to_obligation_and_rejects_unbound_proof_change", commandCoverageExpectedPublicOutcome), "Impact analysis must route changed requirement records to obligations and reject unbound proof-like changes.")}, "init": {directCLIRoute("internal/app/cli_abi_test.go", "TestCLIABIGoldenCorpus", semanticRouteProof("cli_abi.init_golden_corpus", commandCoverageExpectedPublicOutcome), "Init CLI ABI must emit dry-run route guidance without reading stdin, scanning, writing, or promoting repository facts.")}, "json-report-cli-adapter-source": {packageFalsifierRoute("internal/command/jsonreportcliadaptersource/json_report_cli_adapter_source_test.go", "TestGeneratedTypeScriptAdapterExecutesCoreSemantics", semanticRouteProof("json_report_cli_adapter_source.generated_type_script_adapter_executes_core_semantics", commandCoverageExpectedPublicOutcome), "JSON report CLI adapter source generation must emit executable TypeScript that preserves parser, stable JSON, subprocess exit-code, stdout, stderr, and redacted direct-main semantics.")}, - "migration-parity-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationparityadmission/migrationparityadmission_test.go", "TestBuildAdmitsMatchedParityAndRejectsDigestDrift", semanticRouteProof("migrationparityadmission.build_admits_matched_parity_and_rejects_digest_drift", commandCoverageExpectedPublicOutcome), "Migration parity admission must reject matched parity records with digest drift.")}, + "migration-parity-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationparityadmission/migrationparityadmission_test.go", "TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift", semanticRouteProof("migrationparityadmission.build_admits_caller_declared_match_and_rejects_digest_drift", commandCoverageExpectedPublicOutcome), "Migration parity admission must reject caller-declared matches whose supplied digests differ without claiming native digest verification.")}, "migration-plan": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/migrationplan/migrationplan_test.go", "TestSortedFollowUpCommandsRejectsShellControlTokens", semanticRouteProof("migrationplan.sorted_follow_up_commands_rejects_shell_control_tokens", commandCoverageExpectedPublicOutcome), "Migration plans must reject shell-control follow-up commands.")}, "obligation-decision": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/obligationdecision/obligationdecision_test.go", "TestBuildAdmitsSatisfiedBlockingObligationsAndRejectsMissingReceipt", semanticRouteProof("obligationdecision.build_admits_satisfied_blocking_obligations_and_rejects_missing_receipt", commandCoverageExpectedPublicOutcome), "Obligation decision must fail blocking obligations that lack satisfying evidence states.")}, "package-runtime-dependency-admission": {requiredInputAdmissionRoute, packageFalsifierRoute("internal/command/packageruntimedependency/package_runtime_dependency_test.go", "TestBuildAdmitsExternalRuntimeDependencyAndRejectsWorkspaceResolution", semanticRouteProof("package_runtime_dependency.build_admits_external_runtime_dependency_and_rejects_workspace_resolution", commandCoverageExpectedPublicOutcome), "Package runtime dependency admission must reject local workspace resolution.")}, @@ -137,7 +139,7 @@ var commandCoverageRoutes = map[string][]commandCoverageRoute{ "stack-preset": {directCLIRoute("internal/app/command_coverage_test.go", "TestNoInputCommandsHaveCommandSpecificBehavior", semanticRouteProof("command_coverage.no_input_commands_have_command_specific_behavior", commandCoverageExpectedPublicOutcome), "Stack preset CLI route must emit JSON and reject unknown preset flags."), packageFalsifierRoute("internal/command/stackpreset/stackpreset_test.go", "TestPresetInventoryIsCompleteDeterministicAndDefensivelyCopied", semanticRouteProof("stackpreset.preset_inventory_is_complete_deterministic_and_defensively_copied", commandCoverageExpectedPublicOutcome), "Stack preset inventory must keep preset ids aligned with complete non-empty profile records and defensive copies."), packageFalsifierRoute("internal/command/stackpreset/stackpreset_test.go", "TestUnknownPresetIsRejected", semanticRouteProof("stackpreset.unknown_preset_is_rejected", commandCoverageExpectedPublicOutcome), "Stack preset package API must reject unknown preset ids.")}, "test-evidence-inventory": { requiredInputAdmissionRoute, - packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildRejectsWeakOracleAndDuplicateFalsifier", semanticRouteProof("testevidenceinventory.build_rejects_weak_oracle_and_duplicate_falsifier", commandCoverageExpectedPublicOutcome), "Test evidence inventory must reject weak semantic oracles and duplicate falsifier equivalence claims."), + packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier", semanticRouteProof("testevidenceinventory.build_rejects_incomplete_declared_oracle_metadata_and_duplicate_falsifier", commandCoverageExpectedPublicOutcome), "Test evidence inventory must reject incomplete caller-declared oracle metadata and duplicate falsifier equivalence claims."), packageFalsifierRoute("internal/command/testevidenceinventory/testevidenceinventory_test.go", "TestBuildDiscoveryDraftEmitsCandidateOnlyInventory", semanticRouteProof("testevidenceinventory.build_discovery_draft_emits_candidate_only_inventory", commandCoverageExpectedPublicOutcome), "Test discovery draft projection must emit candidate-only inventory guidance without closing semantic coverage."), packageFalsifierRoute("internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go", "TestBuildRejectsDerivedCommandRefCollision", semanticRouteProof("proofbindingtestinventory.build_rejects_derived_command_ref_collision", commandCoverageExpectedPublicOutcome), "Proof-binding-derived inventory projection must reject command-ref collisions before emitting normalized inventory."), }, @@ -237,7 +239,7 @@ func commandCoverageInventoryFrom(routes map[string][]commandCoverageRoute) (map "Command coverage inventory does not execute tests.", "Command coverage inventory does not prove native command success, receipt freshness, or merge satisfaction.", "Routing smoke entries prove CLI input routing only and cannot satisfy semantic command coverage.", - "Static route metadata, prose, source markers, test existence, and failure-capable AST nodes are proof-route candidates only; they cannot emit semantic_falsifier evidence.", + "Static route metadata, prose, source markers, test existence, and failure-capable AST nodes are proof-route candidates only; they cannot emit execution-backed semantic evidence.", }, }, nil } diff --git a/internal/app/command_coverage_test.go b/internal/app/command_coverage_test.go index 2145840..daf1583 100644 --- a/internal/app/command_coverage_test.go +++ b/internal/app/command_coverage_test.go @@ -162,7 +162,7 @@ func TestCommandCoverageInventoryIsAdmittedAndBindsProofRouteCandidates(t *testi t.Fatalf("proof-route candidate laundered semantic evidence: %#v", entry) } candidateCommandRefs[entry.CommandRefs[0]] = struct{}{} - case "semantic_falsifier": + case "declared_semantic_falsifier_route": t.Fatalf("static command coverage route became semantic evidence: %#v", entry) case "routing_smoke_nonclaim": if len(entry.OwnerInvariantRefs) != 0 || entry.Falsifier != nil || entry.Oracle != nil { diff --git a/internal/command/agentroute/agentroute.go b/internal/command/agentroute/agentroute.go index 97440bf..cd1a34a 100644 --- a/internal/command/agentroute/agentroute.go +++ b/internal/command/agentroute/agentroute.go @@ -242,7 +242,7 @@ var routeSpecs = map[string]routeSpec{ {Command: "requirement-coverage-view", InputKind: "coverage_view_input", Why: "Coverage inspection should join admitted requirement, proof-binding, and test-inventory inputs."}, {Command: "requirement-browser-server", InputKind: "coverage_view_input", ExtraArgs: []string{"--view", "coverage"}, Why: "Browser views are presentation-only inspection surfaces over caller-owned coverage input."}, }, - StopConditions: []string{"Stop when tests are route-only, weak-oracle, unbound, or outside the caller-owned coverage universe."}, + StopConditions: []string{"Stop when tests are route-only, lack a declared assertion signal, are unbound, or are outside the caller-owned coverage universe."}, Escalations: []string{"Escalate coverage completeness and native test execution to the consuming repository."}, SliceSummary: "Route coverage inspection through admitted requirement, proof-binding, and test inventory inputs.", SliceNonClaims: []string{"The coverage slice does not claim inventory completeness outside the caller-owned universe."}, diff --git a/internal/command/agentroute/agentroute_test.go b/internal/command/agentroute/agentroute_test.go index 43659b1..3aad0f6 100644 --- a/internal/command/agentroute/agentroute_test.go +++ b/internal/command/agentroute/agentroute_test.go @@ -413,6 +413,10 @@ func TestBuildRoutesCoverageComposeInputBeforeCoverageView(t *testing.T) { if commandExists(commands, "requirement-coverage-view") { t.Fatalf("coverage compose route must not require prebuilt coverage_view_input: %#v", commands) } + stopConditions := strings.Join(toStringSlice(t, report["stopConditions"].([]any)), "\n") + if !strings.Contains(stopConditions, "lack a declared assertion signal") || strings.Contains(stopConditions, "weak-oracle") { + t.Fatalf("coverage stop conditions retain obsolete oracle-quality vocabulary: %s", stopConditions) + } } func TestBuildRoutesRequirementSourceTransitionOnlyWithTransitionInput(t *testing.T) { diff --git a/internal/command/migrationparityadmission/migrationparityadmission.go b/internal/command/migrationparityadmission/migrationparityadmission.go index efdcebb..be9b5ee 100644 --- a/internal/command/migrationparityadmission/migrationparityadmission.go +++ b/internal/command/migrationparityadmission/migrationparityadmission.go @@ -21,7 +21,7 @@ var equivalenceKinds = []string{ } var equivalenceKindSet = toSet(equivalenceKinds) -var parityStatuses = []string{"matched", "mismatched", "not_comparable", "not_run"} +var parityStatuses = []string{"caller_declared_match", "caller_declared_mismatch", "caller_declared_not_comparable", "caller_declared_not_run"} var parityStatusSet = toSet(parityStatuses) var sourceOwnerKinds = []string{"local_doc", "local_manifest", "local_script", "local_test", "other"} @@ -36,6 +36,7 @@ var boundaryNonClaims = []string{ "Migration parity admission does not approve old-owner deletion, migration exceptions, merge, release, rollout, or production readiness.", "Migration parity admission does not authenticate parity evidence.", "Migration parity admission does not compute digest values or proof freshness.", + "Migration parity statuses and status-derived parity claim counters are caller declarations, while structural and failure counts are Proofkit-computed admission facts; none are native verification results.", "Migration parity admission does not execute native commands or prove command result correctness.", "Migration parity admission does not prove semantic correctness of either legacy or Proofkit-owned infrastructure.", } @@ -73,7 +74,7 @@ type parityDiagnostic struct { Findings []string } -type admittedEvidenceRef struct { +type admittedParityClaimRef struct { EquivalenceKind string EvidenceID string EvidenceRefs []string @@ -111,11 +112,11 @@ func Build(raw any) (report.Record, int, error) { return diagnostics[left].EvidenceID < diagnostics[right].EvidenceID }) failures := []string{} - admittedRefs := []admittedEvidenceRef{} + admittedRefs := []admittedParityClaimRef{} for _, diagnostic := range diagnostics { failures = append(failures, diagnostic.Findings...) if len(diagnostic.Findings) == 0 { - admittedRefs = append(admittedRefs, admittedEvidenceRef{ + admittedRefs = append(admittedRefs, admittedParityClaimRef{ EquivalenceKind: diagnostic.EquivalenceKind, EvidenceID: diagnostic.EvidenceID, EvidenceRefs: diagnostic.EvidenceRefs, @@ -137,18 +138,18 @@ func Build(raw any) (report.Record, int, error) { ReportID: input.ParitySetID, State: state, Summary: map[string]any{ - "admittedParityEvidenceCount": len(admittedRefs), - "failureCount": len(failures), - "matchedCount": countByStatus(diagnostics, "matched"), - "mismatchedCount": countByStatus(diagnostics, "mismatched"), - "notComparableCount": countByStatus(diagnostics, "not_comparable"), - "notRunCount": countByStatus(diagnostics, "not_run"), - "parityRecordCount": len(diagnostics), - "sourceProofOwnerCount": len(input.SourceProofOwners), - "targetProofkitRefCount": len(input.TargetProofkitRefs), + "admittedParityClaimCount": len(admittedRefs), + "failureCount": len(failures), + "callerDeclaredMatchCount": countByStatus(diagnostics, "caller_declared_match"), + "callerDeclaredMismatchCount": countByStatus(diagnostics, "caller_declared_mismatch"), + "callerDeclaredNotComparableCount": countByStatus(diagnostics, "caller_declared_not_comparable"), + "callerDeclaredNotRunCount": countByStatus(diagnostics, "caller_declared_not_run"), + "parityRecordCount": len(diagnostics), + "sourceProofOwnerCount": len(input.SourceProofOwners), + "targetProofkitRefCount": len(input.TargetProofkitRefs), }, Diagnostics: []report.Diagnostic{ - {Key: "admittedParityEvidenceRefs", Value: admittedEvidenceDiagnostics(admittedRefs)}, + {Key: "admittedParityClaimRefs", Value: admittedParityClaimDiagnostics(admittedRefs)}, {Key: "failures", Value: admit.StringSliceToAny(failures)}, {Key: "migrationParity", Value: parityDiagnostics(diagnostics)}, }, @@ -398,13 +399,13 @@ func evaluateParityRecord(record parityRecord, sourceOwnerIDs map[string]struct{ if _, ok := targetIDs[record.TargetID]; !ok { findings = append(findings, fmt.Sprintf("migration parity record %s references unknown target: %s", record.EvidenceID, record.TargetID)) } - if record.Status == "matched" && record.LegacyDigest != record.ProofkitDigest { - findings = append(findings, fmt.Sprintf("migration parity record %s is matched but digests differ", record.EvidenceID)) + if record.Status == "caller_declared_match" && record.LegacyDigest != record.ProofkitDigest { + findings = append(findings, fmt.Sprintf("migration parity record %s declares a match but digests differ", record.EvidenceID)) } - if record.Status == "mismatched" && record.LegacyDigest == record.ProofkitDigest { - findings = append(findings, fmt.Sprintf("migration parity record %s is mismatched but digests are equal", record.EvidenceID)) + if record.Status == "caller_declared_mismatch" && record.LegacyDigest == record.ProofkitDigest { + findings = append(findings, fmt.Sprintf("migration parity record %s declares a mismatch but digests are equal", record.EvidenceID)) } - if record.Status != "matched" { + if record.Status != "caller_declared_match" { findings = append(findings, fmt.Sprintf("migration parity record %s is not admitted: %s", record.EvidenceID, record.Status)) } sort.Strings(findings) @@ -421,6 +422,7 @@ func parityDiagnostics(diagnostics []parityDiagnostic) []any { "findings": admit.StringSliceToAny(diagnostic.Findings), "legacyDigest": diagnostic.LegacyDigest, "legacySubjectRef": diagnostic.LegacySubjectRef, + "nonClaims": admit.StringSliceToAny(diagnostic.NonClaims), "proofkitDigest": diagnostic.ProofkitDigest, "proofkitSubjectRef": diagnostic.ProofkitSubjectRef, "reason": diagnostic.Reason, @@ -433,7 +435,7 @@ func parityDiagnostics(diagnostics []parityDiagnostic) []any { return result } -func admittedEvidenceDiagnostics(refs []admittedEvidenceRef) []any { +func admittedParityClaimDiagnostics(refs []admittedParityClaimRef) []any { result := make([]any, 0, len(refs)) for _, ref := range refs { result = append(result, map[string]any{ @@ -452,11 +454,11 @@ func ruleResults(diagnostics []parityDiagnostic) []report.RuleResult { results := make([]report.RuleResult, 0, len(diagnostics)) for _, diagnostic := range diagnostics { status := "passed" - message := "migration parity evidence record is admitted" + message := "caller-declared migration parity claim is admitted" ruleDiagnostics := []report.Diagnostic{} if len(diagnostic.Findings) > 0 { status = "failed" - message = "migration parity evidence record is not admitted" + message = "caller-declared migration parity claim is not admitted" ruleDiagnostics = []report.Diagnostic{{Key: "findings", Value: admit.StringSliceToAny(diagnostic.Findings)}} } results = append(results, report.RuleResult{ diff --git a/internal/command/migrationparityadmission/migrationparityadmission_test.go b/internal/command/migrationparityadmission/migrationparityadmission_test.go index 147043c..3c73afd 100644 --- a/internal/command/migrationparityadmission/migrationparityadmission_test.go +++ b/internal/command/migrationparityadmission/migrationparityadmission_test.go @@ -2,15 +2,19 @@ package migrationparityadmission import ( "encoding/json" - "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" + "slices" + "sort" + "strings" "testing" + + "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" ) const testDigest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" const otherDigest = "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" -func TestBuildAdmitsMatchedParityAndRejectsDigestDrift(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.030283783580216032239035834806346340389755580763262728864296008708349716751941") +func TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift(t *testing.T) { + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.087858357304530014219351528044382591532264544185224319857347811230381754931058") input := validMigrationParityInput() record, exitCode, err := Build(input) if err != nil { @@ -19,6 +23,12 @@ func TestBuildAdmitsMatchedParityAndRejectsDigestDrift(t *testing.T) { if exitCode != 0 || record.State != "passed" { t.Fatalf("Build() exit=%d state=%s, want passed", exitCode, record.State) } + if record.Summary["callerDeclaredMatchCount"] != 1 || record.Summary["admittedParityClaimCount"] != 1 { + t.Fatalf("Build() summary lost caller-declared boundary: %#v", record.Summary) + } + if _, exists := record.Summary["matchedCount"]; exists { + t.Fatalf("Build() retained verification-sounding matchedCount: %#v", record.Summary) + } input = validMigrationParityInput() input["parityRecords"].([]any)[0].(map[string]any)["proofkitDigest"] = otherDigest @@ -31,6 +41,155 @@ func TestBuildAdmitsMatchedParityAndRejectsDigestDrift(t *testing.T) { } } +func TestBuildProjectsEveryCallerDeclaredStatusAndSummaryField(t *testing.T) { + tests := []struct { + name string + status string + differentDigests bool + passed bool + wantFindings []string + }{ + {name: "match equal", status: "caller_declared_match", passed: true}, + {name: "match different", status: "caller_declared_match", differentDigests: true, wantFindings: []string{"migration parity record proofkit.test.evidence declares a match but digests differ"}}, + {name: "mismatch equal", status: "caller_declared_mismatch", wantFindings: []string{"migration parity record proofkit.test.evidence declares a mismatch but digests are equal", "migration parity record proofkit.test.evidence is not admitted: caller_declared_mismatch"}}, + {name: "mismatch different", status: "caller_declared_mismatch", differentDigests: true, wantFindings: []string{"migration parity record proofkit.test.evidence is not admitted: caller_declared_mismatch"}}, + {name: "not comparable equal", status: "caller_declared_not_comparable", wantFindings: []string{"migration parity record proofkit.test.evidence is not admitted: caller_declared_not_comparable"}}, + {name: "not comparable different", status: "caller_declared_not_comparable", differentDigests: true, wantFindings: []string{"migration parity record proofkit.test.evidence is not admitted: caller_declared_not_comparable"}}, + {name: "not run equal", status: "caller_declared_not_run", wantFindings: []string{"migration parity record proofkit.test.evidence is not admitted: caller_declared_not_run"}}, + {name: "not run different", status: "caller_declared_not_run", differentDigests: true, wantFindings: []string{"migration parity record proofkit.test.evidence is not admitted: caller_declared_not_run"}}, + } + wantSummaryKeys := []string{ + "admittedParityClaimCount", + "callerDeclaredMatchCount", + "callerDeclaredMismatchCount", + "callerDeclaredNotComparableCount", + "callerDeclaredNotRunCount", + "failureCount", + "parityRecordCount", + "sourceProofOwnerCount", + "targetProofkitRefCount", + } + wantDiagnosticKeys := []string{"admittedParityClaimRefs", "failures", "migrationParity"} + statusSummaryKeys := map[string]string{ + "caller_declared_match": "callerDeclaredMatchCount", + "caller_declared_mismatch": "callerDeclaredMismatchCount", + "caller_declared_not_comparable": "callerDeclaredNotComparableCount", + "caller_declared_not_run": "callerDeclaredNotRunCount", + } + const boundaryNonClaim = "Migration parity statuses and status-derived parity claim counters are caller declarations, while structural and failure counts are Proofkit-computed admission facts; none are native verification results." + + for _, item := range tests { + t.Run(item.name, func(t *testing.T) { + input := validMigrationParityInput() + parity := input["parityRecords"].([]any)[0].(map[string]any) + parity["status"] = item.status + if item.differentDigests { + parity["proofkitDigest"] = otherDigest + } + + record, exitCode, err := Build(input) + if err != nil { + t.Fatalf("Build() error=%v", err) + } + if item.passed != (exitCode == 0 && record.State == "passed") { + t.Fatalf("Build() exit=%d state=%s, passed=%v", exitCode, record.State, item.passed) + } + statusCountTotal := 0 + for status, summaryKey := range statusSummaryKeys { + want := 0 + if status == item.status { + want = 1 + } + if record.Summary[summaryKey] != want { + t.Fatalf("Build() %s=%#v, want %d for status %s", summaryKey, record.Summary[summaryKey], want, item.status) + } + statusCountTotal += record.Summary[summaryKey].(int) + } + if statusCountTotal != record.Summary["parityRecordCount"] { + t.Fatalf("Build() caller-declared status total=%d, parityRecordCount=%#v", statusCountTotal, record.Summary["parityRecordCount"]) + } + if got := sortedMapKeys(record.Summary); !slices.Equal(got, wantSummaryKeys) { + t.Fatalf("Build() summary keys=%v, want %v", got, wantSummaryKeys) + } + diagnosticKeys := make([]string, 0, len(record.Diagnostics)) + var admittedRefs []any + var failures []any + var migrationParity []any + for _, diagnostic := range record.Diagnostics { + diagnosticKeys = append(diagnosticKeys, diagnostic.Key) + if diagnostic.Key == "admittedParityClaimRefs" { + admittedRefs = diagnostic.Value.([]any) + } + if diagnostic.Key == "failures" { + failures = diagnostic.Value.([]any) + } + if diagnostic.Key == "migrationParity" { + migrationParity = diagnostic.Value.([]any) + } + } + sort.Strings(diagnosticKeys) + if !slices.Equal(diagnosticKeys, wantDiagnosticKeys) { + t.Fatalf("Build() diagnostic keys=%v, want %v", diagnosticKeys, wantDiagnosticKeys) + } + if len(migrationParity) != 1 || migrationParity[0].(map[string]any)["status"] != item.status { + t.Fatalf("Build() migration parity=%#v, want status %s", migrationParity, item.status) + } + if got := anyStrings(migrationParity[0].(map[string]any)["findings"].([]any)); !slices.Equal(got, item.wantFindings) { + t.Fatalf("Build() findings=%v, want %v", got, item.wantFindings) + } + wantRecordNonClaims := anyStrings(parity["nonClaims"].([]any)) + if got := anyStrings(migrationParity[0].(map[string]any)["nonClaims"].([]any)); !slices.Equal(got, wantRecordNonClaims) { + t.Fatalf("Build() record nonClaims=%v, want exact caller-declared boundary %v", got, wantRecordNonClaims) + } + for key, want := range map[string]int{ + "admittedParityClaimCount": len(admittedRefs), + "failureCount": len(failures), + "parityRecordCount": len(input["parityRecords"].([]any)), + "sourceProofOwnerCount": len(input["sourceProofOwners"].([]any)), + "targetProofkitRefCount": len(input["targetProofkitRefs"].([]any)), + } { + if record.Summary[key] != want { + t.Fatalf("Build() %s=%#v, want computed %d", key, record.Summary[key], want) + } + } + if !strings.Contains(strings.Join(anyStrings(record.NonClaims), "\n"), boundaryNonClaim) { + t.Fatalf("Build() nonClaims=%#v, want caller-declaration boundary", record.NonClaims) + } + }) + } +} + +func TestBuildRejectsVerificationSoundingLegacyStatuses(t *testing.T) { + for _, status := range []string{"matched", "mismatched", "not_comparable", "not_run"} { + t.Run(status, func(t *testing.T) { + input := validMigrationParityInput() + input["parityRecords"].([]any)[0].(map[string]any)["status"] = status + + _, _, err := Build(input) + if err == nil { + t.Fatal("Build() admitted legacy status that implies native verification") + } + }) + } +} + +func sortedMapKeys(values map[string]any) []string { + keys := make([]string, 0, len(values)) + for key := range values { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} + +func anyStrings(values []any) []string { + result := make([]string, 0, len(values)) + for _, value := range values { + result = append(result, value.(string)) + } + return result +} + func validMigrationParityInput() map[string]any { return map[string]any{ "schemaVersion": json.Number("1"), @@ -54,7 +213,7 @@ func validMigrationParityInput() map[string]any { "reason": "same admitted projection", "receiptRefs": []any{}, "sourceOwnerId": "proofkit.test.legacy", - "status": "matched", + "status": "caller_declared_match", "targetId": "proofkit.test.target", }, }, diff --git a/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go b/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go index cf8838e..6073553 100644 --- a/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go +++ b/internal/command/proofbindingtestinventory/proofbindingtestinventory_test.go @@ -54,7 +54,7 @@ func TestBuildProjectsCompactProofBindingToAdmittedInventory(t *testing.T) { if err != nil || reportExitCode != 0 { t.Fatalf("BuildReport() exit=%d err=%v", reportExitCode, err) } - if report.Summary["proofRouteCandidateCount"] != 1 || report.Summary["semanticFalsifierCount"] != 0 { + if report.Summary["proofRouteCandidateCount"] != 1 || report.Summary["declaredSemanticFalsifierRouteCount"] != 0 { t.Fatalf("downstream evidence classification summary=%#v", report.Summary) } for _, rule := range report.RuleResults { @@ -86,7 +86,7 @@ func TestBuildRejectsUnstructuredFalsificationSelector(t *testing.T) { } } -func TestBuildRejectsSemanticFalsifierWithoutVerifyCommand(t *testing.T) { +func TestBuildRejectsProofRouteCandidateWithoutVerifyCommand(t *testing.T) { input := validInput() binding := input["compactProofContract"].(map[string]any)["bindings"].([]any)[0].([]any) falsification := binding[9].([]any) diff --git a/internal/command/requirementauthoringplan/requirement_authoring_plan.go b/internal/command/requirementauthoringplan/requirement_authoring_plan.go index cda3460..bc7dc17 100644 --- a/internal/command/requirementauthoringplan/requirement_authoring_plan.go +++ b/internal/command/requirementauthoringplan/requirement_authoring_plan.go @@ -32,7 +32,7 @@ type input struct { AuthoringPlanID string AuthoringRefs []authoringRef CandidateUpdates []candidateUpdate - CurrentRequirementRaw map[string]any + CurrentRequirementValue map[string]any CurrentRequirementState requirementsourceadmission.Source Mode string NonClaims []string @@ -79,13 +79,16 @@ func Build(raw any) (map[string]any, int, error) { func buildOutput(input input) (map[string]any, int) { nextSource, compositionFailures := composeNextRequirementSource(input) sourceResult, sourceErr := requirementsourceadmission.Evaluate(nextSource) + if sourceErr == nil && sourceResult.ExitCode == 0 { + nextSource = requirementsourceadmission.SourceValue(sourceResult.Source) + } transitionInput := map[string]any{ "schemaVersion": json.Number("1"), "transitionId": input.AuthoringPlanID + ".transition", "nonClaims": []any{ "Requirement authoring plan transition proof does not promote candidates to stable repository truth.", }, - "previous": input.CurrentRequirementRaw, + "previous": input.CurrentRequirementValue, "next": nextSource, } transitionRecord, transitionExitCode, transitionErr := requirementsourcetransition.Build(transitionInput) @@ -120,6 +123,7 @@ func buildOutput(input input) (map[string]any, int) { } } output := map[string]any{ + "authoringRefs": authoringRefValues(input.AuthoringRefs), "authoringPlanId": input.AuthoringPlanID, "candidateChangeSet": candidateUpdateValues(input.CandidateUpdates, state == "passed"), "mode": input.Mode, @@ -129,7 +133,7 @@ func buildOutput(input input) (map[string]any, int) { "planKind": planKind, "promotionPreconditions": promotionPreconditions(input), "ruleResults": ruleResults(failures, sourceResult, sourceErr, transitionRecord.State, transitionErr), - "schemaVersion": 1, + "schemaVersion": 2, "state": state, "summary": map[string]any{ "authoringRefCount": len(input.AuthoringRefs), @@ -151,6 +155,25 @@ func buildOutput(input input) (map[string]any, int) { return output, 1 } +func authoringRefValues(refs []authoringRef) []any { + values := make([]any, 0, len(refs)) + for _, ref := range refs { + var digest any + if ref.Digest != nil { + digest = *ref.Digest + } + values = append(values, map[string]any{ + "digest": digest, + "kind": ref.Kind, + "nonClaims": admit.StringSliceToAny(ref.NonClaims), + "path": ref.Path, + "refId": ref.RefID, + "summary": ref.Summary, + }) + } + return values +} + func admitInput(raw any) (input, error) { record, ok := raw.(map[string]any) if !ok { @@ -197,7 +220,7 @@ func admitInput(raw any) (input, error) { AuthoringPlanID: authoringPlanID, AuthoringRefs: refs, CandidateUpdates: updates, - CurrentRequirementRaw: cloneObject(current), + CurrentRequirementValue: requirementsourceadmission.SourceValue(currentResult.Source), CurrentRequirementState: currentResult.Source, Mode: mode, NonClaims: nonClaims, @@ -327,11 +350,12 @@ func admitCandidateUpdate(raw any, admittedRefIDs map[string]struct{}) (candidat if !ok { return candidateUpdate{}, fmt.Errorf("requirement authoring plan candidateRequirement must be an object") } - candidateRequirement = cloneObject(candidateRequirement) - candidateRequirementID, err := requirementID(candidateRequirement["requirementId"], "requirement authoring plan candidateRequirement.requirementId") + admittedCandidate, err := requirementsourceadmission.AdmitRequirement(candidateRequirement) if err != nil { return candidateUpdate{}, err } + candidateRequirement = requirementsourceadmission.RequirementValue(admittedCandidate) + candidateRequirementID := admittedCandidate.RequirementID if candidateRequirementID != targetRequirementID { return candidateUpdate{}, fmt.Errorf("requirement authoring plan candidateRequirement.requirementId must equal candidateUpdate.requirementId") } @@ -396,7 +420,7 @@ func admitProofObligations(raw any) ([]proofObligation, error) { } func composeNextRequirementSource(input input) (map[string]any, []string) { - current := cloneObject(input.CurrentRequirementRaw) + current := cloneObject(input.CurrentRequirementValue) requirements := cloneArray(current["requirements"].([]any)) indexByRequirementID := map[string]int{} for index, raw := range requirements { diff --git a/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go b/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go index 7985665..d986898 100644 --- a/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go +++ b/internal/command/requirementauthoringplan/requirement_authoring_plan_test.go @@ -5,6 +5,7 @@ import ( "strings" "testing" + "github.com/research-engineering/agentic-proofkit/internal/command/requirementsourceadmission" "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" ) @@ -17,6 +18,9 @@ func TestBuildComposesCandidateSourceAcceptedByAdmissionAndTransition(t *testing if exitCode != 0 || output["state"] != "passed" { t.Fatalf("Build() exit=%d state=%v output=%#v", exitCode, output["state"], output) } + if output["schemaVersion"] != 2 { + t.Fatalf("Build() schemaVersion=%#v, want versioned output schema 2", output["schemaVersion"]) + } preview, ok := output["nonAuthoritativeAdmissionPreview"].(map[string]any) if !ok { t.Fatalf("passed authoring plan must expose wrapped candidate-only preview: %#v", output["nonAuthoritativeAdmissionPreview"]) @@ -24,10 +28,59 @@ func TestBuildComposesCandidateSourceAcceptedByAdmissionAndTransition(t *testing if preview["authority"] != "candidate_only" || preview["ownerReviewRequired"] != true || preview["candidateOnly"] != true { t.Fatalf("preview lost candidate-only metadata: %#v", preview) } - assertStableJSONEqual(t, "candidate source preview", expectedNextSource(), preview["requirementSourcePreview"]) + expectedResult, err := requirementsourceadmission.Evaluate(expectedNextSource()) + if err != nil || expectedResult.ExitCode != 0 { + t.Fatalf("admit expected next source: exit=%d err=%v", expectedResult.ExitCode, err) + } + assertStableJSONEqual(t, "candidate source preview", requirementsourceadmission.SourceValue(expectedResult.Source), preview["requirementSourcePreview"]) assertOutputContains(t, output, "Requirement authoring plans do not infer requirement meaning") assertOutputContains(t, output, "proofkit.requirement-authoring-plan.review-candidates") assertOutputContains(t, output, "run_admitted_validation") + refs, ok := output["authoringRefs"].([]any) + if !ok || len(refs) != 2 { + t.Fatalf("authoring refs were not projected: %#v", output["authoringRefs"]) + } + assertStableJSONEqual(t, "authoring ref projection", validInput()["authoringRefs"], refs) +} + +func TestBuildPreservesDigestBoundAuthoringRefIdentity(t *testing.T) { + first := validInput() + second := validInput() + secondRef := second["authoringRefs"].([]any)[0].(map[string]any) + secondRef["digest"] = "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + secondRef["summary"] = "The revised design proposes the same candidate from different source bytes." + + firstOutput, firstExit, firstErr := Build(first) + secondOutput, secondExit, secondErr := Build(second) + if firstErr != nil || secondErr != nil || firstExit != 0 || secondExit != 0 { + t.Fatalf("Build() first=(%d,%v) second=(%d,%v)", firstExit, firstErr, secondExit, secondErr) + } + assertStableJSONEqual(t, "first complete authoring ref projection", first["authoringRefs"], firstOutput["authoringRefs"]) + assertStableJSONEqual(t, "second complete authoring ref projection", second["authoringRefs"], secondOutput["authoringRefs"]) + firstRefs, err := stablejson.Marshal(firstOutput["authoringRefs"]) + if err != nil { + t.Fatalf("marshal first authoring refs: %v", err) + } + secondRefs, err := stablejson.Marshal(secondOutput["authoringRefs"]) + if err != nil { + t.Fatalf("marshal second authoring refs: %v", err) + } + if string(firstRefs) == string(secondRefs) { + t.Fatal("digest- and summary-distinct authoring refs projected identically") + } +} + +func TestAdmitInputUsesCanonicalRequirementSourceProjection(t *testing.T) { + raw := validInput() + admitted, err := admitInput(raw) + if err != nil { + t.Fatalf("admitInput() error = %v", err) + } + assertStableJSONEqual(t, "canonical current requirement source", requirementsourceadmission.SourceValue(admitted.CurrentRequirementState), admitted.CurrentRequirementValue) + raw["currentRequirementSource"].(map[string]any)["sourceId"] = "proofkit.test.mutated" + if admitted.CurrentRequirementValue["sourceId"] == "proofkit.test.mutated" { + t.Fatal("admitted current requirement source retained caller alias") + } } func TestBuildRejectsInvalidCurrentRequirementSourceBeforeComposition(t *testing.T) { @@ -104,24 +157,19 @@ func TestBuildRejectsCandidateSourceAdmissionFailure(t *testing.T) { assertOutputContains(t, output, "candidate next source must pass requirement-source-admission") } -func TestBuildOmitsInvalidCandidateRequirementPayloadFromFailedOutput(t *testing.T) { +func TestBuildRejectsSecretShapedCandidateBeforeCompositionWithoutEcho(t *testing.T) { input := validInput() sentinel := "sk-proj-aaaaaaaaaaa" candidate := firstUpdate(input)["candidateRequirement"].(map[string]any) candidate["invariant"] = "Sentinel " + sentinel + " must be rejected without echo." output, exitCode, err := Build(input) - if err != nil { - t.Fatalf("Build() error = %v", err) - } - if exitCode == 0 || output["state"] != "failed" { + if err == nil || output != nil || exitCode != 1 { t.Fatalf("Build() accepted secret-shaped candidate: exit=%d output=%#v", exitCode, output) } - encoded, _ := json.Marshal(output) - if strings.Contains(string(encoded), sentinel) { - t.Fatalf("failed output leaked secret-shaped candidate payload: %s", encoded) + if strings.Contains(err.Error(), sentinel) { + t.Fatalf("admission error leaked secret-shaped candidate payload: %s", err) } - assertOutputContains(t, output, "candidateRequirementOmitted") } func TestBuildRejectsLifecycleTransitionWithoutNewEvidence(t *testing.T) { @@ -314,6 +362,14 @@ func validInput() map[string]any { "summary": "The design proposes a durable authoring invariant.", "nonClaims": []any{"This ref does not prove implementation correctness."}, }, + map[string]any{ + "refId": "proofkit.test.test-summary", + "kind": "test_summary", + "path": "artifacts/test-summary.json", + "digest": nil, + "summary": "The test summary identifies candidate validation context.", + "nonClaims": []any{"This ref does not prove witness execution."}, + }, }, "candidateUpdates": []any{ map[string]any{ diff --git a/internal/command/requirementbinding/requirementbinding.go b/internal/command/requirementbinding/requirementbinding.go index 8b5c047..67c0cd0 100644 --- a/internal/command/requirementbinding/requirementbinding.go +++ b/internal/command/requirementbinding/requirementbinding.go @@ -8,6 +8,7 @@ import ( "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" "github.com/research-engineering/agentic-proofkit/internal/kernel/compactproofcontract" + "github.com/research-engineering/agentic-proofkit/internal/kernel/proofvocab" "github.com/research-engineering/agentic-proofkit/internal/kernel/report" "github.com/research-engineering/agentic-proofkit/internal/kernel/witnesscommand" ) @@ -18,11 +19,7 @@ var claimLevels = map[string]struct{}{ "deferred": {}, } -var proofStates = map[string]struct{}{ - "explicitly_deferred": {}, - "not_bound": {}, - "witness_backed": {}, -} +var proofStates = proofvocab.RequirementProofStateSet() var witnessKinds = map[string]struct{}{ "contract": {}, diff --git a/internal/command/requirementbrowser/server_test.go b/internal/command/requirementbrowser/server_test.go index 31452fb..2306f7e 100644 --- a/internal/command/requirementbrowser/server_test.go +++ b/internal/command/requirementbrowser/server_test.go @@ -310,7 +310,7 @@ func TestStartServerServesExplicitCoverageViews(t *testing.T) { for _, want := range []string{ "Requirement Coverage View", "REQ-BROWSER-COVERAGE-001", - "covered_by_semantic_falsifier", + "mapped_to_declared_semantic_falsifier_route", "Test evidence", "Route-only evidence remains insufficient.", } { @@ -790,7 +790,7 @@ func coverageInput(t *testing.T) any { "selector": "go test ./internal/command/requirementbrowser -run TestStartServerServesExplicitCoverageViews", "sourcePath": "internal/command/requirementbrowser/server_test.go", "ownerId": "browser.coverage", - "evidenceClass": "semantic_falsifier", + "evidenceClass": "declared_semantic_falsifier_route", "requirementRefs": ["REQ-BROWSER-COVERAGE-001"], "ownerInvariantRefs": [], "commandRefs": ["proofkit.browser.coverage.command"], diff --git a/internal/command/requirementcoverageinput/requirementcoverageinput_test.go b/internal/command/requirementcoverageinput/requirementcoverageinput_test.go index d3dd30c..21ce13d 100644 --- a/internal/command/requirementcoverageinput/requirementcoverageinput_test.go +++ b/internal/command/requirementcoverageinput/requirementcoverageinput_test.go @@ -124,7 +124,7 @@ func TestBuildPreservesNormalizedSourceSetProvenance(t *testing.T) { } } -func TestBuildPreservesChildOwnedInventorySupersessionProofRef(t *testing.T) { +func TestBuildPreservesChildOwnedInventorySupersessionDeclarationRef(t *testing.T) { input := validComposeInput(t, supersedingInventoryEntries()).(map[string]any) normalized := input["normalizedTestEvidenceInventory"].(map[string]any) delete(input, "compactProofContract") @@ -141,8 +141,8 @@ func TestBuildPreservesChildOwnedInventorySupersessionProofRef(t *testing.T) { } inventory := output["testEvidenceInventory"].(map[string]any) entries := inventory["entries"].([]any) - if !entryFalsifierHas(entries, "test.coverage.semantic_replacement", "supersessionProofRef", "proof.coverage.semantic_replacement") { - t.Fatalf("composed inventory lost supersessionProofRef: %#v", inventory) + if !entryFalsifierHas(entries, "test.coverage.semantic_replacement", "supersessionDeclarationRef", "proof.coverage.semantic_replacement") { + t.Fatalf("composed inventory lost supersessionDeclarationRef: %#v", inventory) } } @@ -542,7 +542,7 @@ func supersedingInventoryEntries() string { "selector": "internal/command/requirementcoverageinput/requirementcoverageinput_test.go::semantic_replacement", "sourcePath": "internal/command/requirementcoverageinput/requirementcoverageinput_test.go", "ownerId": "proofkit.coverage", - "evidenceClass": "semantic_falsifier", + "evidenceClass": "declared_semantic_falsifier_route", "requirementRefs": ["REQ-PROOFKIT-COVERAGE-001"], "ownerInvariantRefs": ["proof.coverage.semantic_replacement"], "commandRefs": ["proofkit.coverage.command"], @@ -553,7 +553,7 @@ func supersedingInventoryEntries() string { "wrongImplementationClassId": "wrong.coverage.semantic", "dominanceGroup": "coverage.semantic", "supersedes": ["falsifier.coverage.semantic"], - "supersessionProofRef": "proof.coverage.semantic_replacement" + "supersessionDeclarationRef": "proof.coverage.semantic_replacement" }, "oracle": { "oracleId": "oracle.coverage.semantic_replacement", @@ -571,7 +571,7 @@ func inventoryEntry(testID string, commandRef string, sourcePath string, suffix "selector": "` + sourcePath + `::` + suffix + `", "sourcePath": "` + sourcePath + `", "ownerId": "proofkit.coverage", - "evidenceClass": "semantic_falsifier", + "evidenceClass": "declared_semantic_falsifier_route", "requirementRefs": ["REQ-PROOFKIT-COVERAGE-001"], "ownerInvariantRefs": [], "commandRefs": ["` + commandRef + `"], diff --git a/internal/command/requirementcoverageview/admission.go b/internal/command/requirementcoverageview/admission.go index 819a968..c5f52de 100644 --- a/internal/command/requirementcoverageview/admission.go +++ b/internal/command/requirementcoverageview/admission.go @@ -170,6 +170,8 @@ func compactRequirementScenarios(requirement map[string]any) []scenario { return []scenario{{ EnvironmentClasses: compactRequirementEnvironmentClasses(requirement), ScenarioID: scenarioID, + VerifyCommands: stringArray(requirement["verifyCommands"]), + WitnessSelectors: compactRequirementWitnessSelectors(requirement), }} } diff --git a/internal/command/requirementcoverageview/build.go b/internal/command/requirementcoverageview/build.go index b9d7b8f..a89256e 100644 --- a/internal/command/requirementcoverageview/build.go +++ b/internal/command/requirementcoverageview/build.go @@ -41,6 +41,11 @@ func build(raw any) (map[string]any, error) { requirements := buildRequirementCoverage(input, entries, input.Inventory, &failures, &warnings) ownerInvariantCoverage := buildOwnerInvariantCoverage(input, entries, input.Inventory, &warnings) commandCoverage := buildCommandCoverage(input, entries, &failures, &warnings) + unmappedTests := unprojectedTestEntries(entries, requirements, ownerInvariantCoverage, commandCoverage) + coverageBasis, err := buildCoverageBasis(input, entries) + if err != nil { + return nil, err + } deadZones := buildDeadZones(input, entries) if input.CoverageUniverse.CompletenessDeclaration == "selected_paths_advisory" { warnings = append(warnings, deadZoneDiagnostics("dead_zone_advisory", deadZones)...) @@ -59,13 +64,14 @@ func build(raw any) (map[string]any, error) { if input.Inventory != nil { nonClaims = append(nonClaims, anyStrings(input.Inventory.Report.NonClaims)...) } - return map[string]any{ + output := map[string]any{ "authority": "lookup_only", "bindingId": input.Proof.BindingID, "commandCoverage": mapsToAny(commandCoverage), "commandCoverageCount": len(commandCoverage), "completenessDeclaration": input.CoverageUniverse.CompletenessDeclaration, "contractId": input.Proof.ContractID, + "coverageBasis": coverageBasis, "coverageUniverseId": input.CoverageUniverse.UniverseID, "deadZones": mapsToAny(deadZones), "failureClassifications": mapsToAny(diagnosticClassifications(failures, "failure")), @@ -79,16 +85,21 @@ func build(raw any) (map[string]any, error) { "proofMode": input.Proof.Mode, "requirementCoverage": mapsToAny(requirements), "requirementCoverageCount": len(requirements), - "schemaVersion": 1, + "schemaVersion": 2, "sourceId": input.Source.SourceID, "state": state, "testInventoryId": inventoryID(input.Inventory), + "unmappedTests": testEntriesToAny(unmappedTests), "viewInputId": input.ViewInputID, "viewKind": "proofkit.requirement-coverage-view", "warningClassifications": mapsToAny(diagnosticClassifications(warnings, "warning")), "warningCount": len(warnings), "warnings": admit.StringSliceToAny(warnings), - }, nil + } + if err := validateCoverageOutputSemantics(output); err != nil { + return nil, err + } + return output, nil } func buildOwnerInvariantCoverage(input compositeInput, entries []testevidenceinventory.Entry, inventory *testevidenceinventory.Result, warnings *[]string) []map[string]any { @@ -108,10 +119,10 @@ func buildOwnerInvariantCoverage(input compositeInput, entries []testevidenceinv state := "missing_test_inventory" evidenceClass := "" if inventory != nil && inventory.Report.State == "passed" { - state, evidenceClass = strongestEntryState(matches) + state, evidenceClass = strongestMappingState(matches) } invariantWarnings := []string{} - if state == "missing_test_inventory" || state == "proof_route_candidate_only" || state == "route_only_nonclaim" || state == "helper_or_testkit_nonclaim" || state == "benchmark_advisory_missing_policy" { + if ownerInvariantCoverageStateWarns(state) { warning := "missing_owner_invariant_inventory:" + invariant.OwnerInvariantID invariantWarnings = append(invariantWarnings, warning) *warnings = append(*warnings, warning) @@ -140,23 +151,25 @@ func buildRequirementCoverage(input compositeInput, entries []testevidenceinvent result := make([]map[string]any, 0, len(sourceRequirements)) for _, requirement := range sourceRequirements { proof, hasProof := input.Proof.Requirements[requirement.RequirementID] + hasProofRoute := hasProof && proof.ProofState == "witness_backed" entriesForRequirement := entriesReferencingRequirement(entries, requirement.RequirementID) commandIDs := proof.CommandIDs if hasProof && input.Proof.Mode == "compact" && len(commandIDs) == 0 { commandIDs = entryCommandRefs(entriesForRequirement) } scenarios := proof.Scenarios - state, evidenceClass := requirementState(requirement, entriesForRequirement, inventory, hasProof) + state, evidenceClass := requirementState(requirement, entriesForRequirement, inventory, hasProofRoute) requirementFailures := []string{} - if !hasProof && requirement.ClaimLevel == "blocking" && requirement.Lifecycle.State == "active" { + if !hasProofRoute && requirement.ClaimLevel == "blocking" && requirement.Lifecycle.State == "active" { requirementFailures = append(requirementFailures, "missing_proof_binding_route:"+requirement.RequirementID) } - if requirementBlocks(requirement, state, input.CoverageUniverse.CompletenessDeclaration) { + if requirementMappingBlocks(requirement.ClaimLevel, requirement.Lifecycle.State, state, input.CoverageUniverse.CompletenessDeclaration) { requirementFailures = append(requirementFailures, state+":"+requirement.RequirementID) } - if requirementWarns(requirement, state, input.CoverageUniverse.CompletenessDeclaration) { + if requirementMappingWarns(requirement.ClaimLevel, state, input.CoverageUniverse.CompletenessDeclaration) { *warnings = append(*warnings, state+":"+requirement.RequirementID) } + requirementFailures = sortedUnique(requirementFailures) *failures = append(*failures, requirementFailures...) result = append(result, map[string]any{ "claimLevel": requirement.ClaimLevel, @@ -200,38 +213,15 @@ func buildCommandCoverage(input compositeInput, entries []testevidenceinventory. for _, commandID := range commands { matches := entriesReferencingCommand(entries, commandID) state := commandState(matches) - commandFailures := []string{} - if state == "missing_command_semantic_falsifier" && input.CoverageUniverse.CompletenessDeclaration != "selected_paths_advisory" { - commandFailures = append(commandFailures, state+":"+commandID) - *failures = append(*failures, commandFailures...) - } - if state == "command_owner_nonsemantic_evidence" && input.CoverageUniverse.CompletenessDeclaration != "selected_paths_advisory" { - commandFailures = append(commandFailures, "nonsemantic_command_evidence:"+commandID) - *failures = append(*failures, commandFailures...) - } - if state == "missing_command_semantic_falsifier" && input.CoverageUniverse.CompletenessDeclaration == "selected_paths_advisory" { - *warnings = append(*warnings, state+":"+commandID) - } - if state == "command_owner_nonsemantic_evidence" && input.CoverageUniverse.CompletenessDeclaration == "selected_paths_advisory" { - *warnings = append(*warnings, "nonsemantic_command_evidence:"+commandID) - } - if state == "command_route_only_nonclaim" { - *warnings = append(*warnings, state+":"+commandID) - } - if state == "command_proof_route_candidate_only" { - diagnostic := state + ":" + commandID - if input.CoverageUniverse.CompletenessDeclaration == "selected_paths_advisory" { - *warnings = append(*warnings, diagnostic) - } else { - commandFailures = append(commandFailures, diagnostic) - *failures = append(*failures, diagnostic) - } - } + commandFailures, commandWarnings := commandCoverageDiagnostics(commandID, state, input.CoverageUniverse.CompletenessDeclaration) + *failures = append(*failures, commandFailures...) + *warnings = append(*warnings, commandWarnings...) result = append(result, map[string]any{ "commandId": commandID, "coverageState": state, "failures": admit.StringSliceToAny(commandFailures), "testIds": admit.StringSliceToAny(entryIDs(matches)), + "tests": testEntriesToAny(matches), }) } return result @@ -356,14 +346,14 @@ func inOwnerScope(ownerID string, ownerSet map[string]struct{}) bool { return ok } -func requirementState(requirement requirementsourceadmission.Requirement, entries []testevidenceinventory.Entry, inventory *testevidenceinventory.Result, hasProof bool) (string, string) { +func requirementState(requirement requirementsourceadmission.Requirement, entries []testevidenceinventory.Entry, inventory *testevidenceinventory.Result, hasProofRoute bool) (string, string) { if requirement.ClaimLevel == "deferred" { return "deferred_with_owner", "" } if requirement.Lifecycle.State == "removed" { return "not_applicable", "" } - if !hasProof { + if !hasProofRoute { return "missing_proof_binding_route", "" } if inventory == nil { @@ -372,74 +362,19 @@ func requirementState(requirement requirementsourceadmission.Requirement, entrie if inventory.Report.State != "passed" { return "missing_test_inventory", "" } - return strongestEntryState(entries) + return strongestMappingState(entries) } -func strongestEntryState(entries []testevidenceinventory.Entry) (string, string) { - priority := []struct { - class string - state string - }{ - {"semantic_falsifier", "covered_by_semantic_falsifier"}, - {"property_or_fuzz", "covered_by_property_or_fuzz"}, - {"contract_admission", "covered_by_contract_admission"}, - {"governance_or_release", "covered_by_governance_invariant_nonproduct"}, - {"benchmark", "benchmark_advisory_missing_policy"}, - {"proof_route_candidate", "proof_route_candidate_only"}, - {"routing_smoke_nonclaim", "route_only_nonclaim"}, - {"helper_or_testkit", "helper_or_testkit_nonclaim"}, - } - for _, candidate := range priority { - for _, entry := range entries { - if entry.EvidenceClass == candidate.class { - return candidate.state, candidate.class - } - } - } - return "missing_test_inventory", "" -} -func commandState(entries []testevidenceinventory.Entry) string { - for _, entry := range entries { - if entry.EvidenceClass == "semantic_falsifier" { - return "command_semantic_falsifier_present" - } - } - for _, entry := range entries { - if entry.EvidenceClass == "benchmark" || - entry.EvidenceClass == "contract_admission" || - entry.EvidenceClass == "governance_or_release" || - entry.EvidenceClass == "helper_or_testkit" || - entry.EvidenceClass == "property_or_fuzz" { - return "command_owner_nonsemantic_evidence" - } - } - for _, entry := range entries { - if entry.EvidenceClass == "proof_route_candidate" { - return "command_proof_route_candidate_only" - } - } - for _, entry := range entries { - if entry.EvidenceClass == "routing_smoke_nonclaim" { - return "command_route_only_nonclaim" - } - } - return "missing_command_semantic_falsifier" -} -func requirementBlocks(requirement requirementsourceadmission.Requirement, state string, scope string) bool { - if requirement.ClaimLevel != "blocking" || requirement.Lifecycle.State != "active" || scope == "selected_paths_advisory" { - return false - } - switch state { - case "covered_by_semantic_falsifier", "covered_by_property_or_fuzz", "covered_by_contract_admission", "not_applicable", "deferred_with_owner": +func requirementMappingBlocks(claimLevel string, lifecycleState string, state string, scope string) bool { + if claimLevel != "blocking" || lifecycleState != "active" || scope == "selected_paths_advisory" { return false - default: - return true } + return !requirementCoverageStateComplete(state) } -func requirementWarns(requirement requirementsourceadmission.Requirement, state string, scope string) bool { +func requirementMappingWarns(claimLevel string, state string, scope string) bool { if scope == "selected_paths_advisory" { - return state != "covered_by_semantic_falsifier" && state != "covered_by_property_or_fuzz" && state != "covered_by_contract_admission" + return !requirementCoverageStateComplete(state) } - return requirement.ClaimLevel != "blocking" && state != "covered_by_semantic_falsifier" && state != "covered_by_property_or_fuzz" && state != "covered_by_contract_admission" + return claimLevel != "blocking" && !requirementCoverageStateComplete(state) } func unknownInventoryRefs(entries []testevidenceinventory.Entry, requirements map[string]struct{}, invariants map[string]struct{}, commands map[string]struct{}, witnesses map[string]struct{}) []string { failures := []string{} @@ -493,7 +428,7 @@ func entriesReferencingCommand(entries []testevidenceinventory.Entry, commandID func guidanceSummary(state string, failures []string, warnings []string) map[string]any { nextAction := "Inspect caller-owned coverage classifications before making repository policy decisions." if state == "failed" { - nextAction = "Repair missing semantic falsifiers, unknown refs, or failed inventory before using this view as merge-adjacent guidance." + nextAction = "Repair missing declared test routes, unknown refs, or failed inventory before using this mapping view as guidance." } return map[string]any{ "failureCount": len(failures), diff --git a/internal/command/requirementcoverageview/classifications.go b/internal/command/requirementcoverageview/classifications.go index 372df09..4490f84 100644 --- a/internal/command/requirementcoverageview/classifications.go +++ b/internal/command/requirementcoverageview/classifications.go @@ -28,17 +28,16 @@ func diagnosticClassID(diagnostic string) string { return "declared_dead_zone" case strings.HasPrefix(diagnostic, "missing_proof_binding_route:") || strings.HasPrefix(diagnostic, "proof_binding_unknown_requirement:"): return "missing_requirement_binding" - case strings.HasPrefix(diagnostic, "missing_test_inventory:") || strings.HasPrefix(diagnostic, "missing_command_semantic_falsifier:"): - return "missing_semantic_test" + case strings.HasPrefix(diagnostic, "missing_test_inventory:") || strings.HasPrefix(diagnostic, "missing_command_declared_semantic_falsifier_route:"): + return "missing_declared_test_route" case strings.HasPrefix(diagnostic, "missing_owner_invariant_inventory:"): - return "missing_semantic_test" + return "missing_declared_test_route" case strings.HasPrefix(diagnostic, "nonsemantic_command_evidence:"): return "nonsemantic_command_evidence" case strings.HasPrefix(diagnostic, "proof_route_candidate_only:") || strings.HasPrefix(diagnostic, "command_proof_route_candidate_only:"): return "proof_route_candidate_only" case strings.HasPrefix(diagnostic, "route_only_nonclaim:") || - strings.HasPrefix(diagnostic, "command_route_only_nonclaim:") || - strings.HasPrefix(diagnostic, "covered_by_routing_smoke_nonclaim:"): + strings.HasPrefix(diagnostic, "command_route_only_nonclaim:"): return "routing_smoke_only" case strings.HasPrefix(diagnostic, "unknown_requirement_ref:") || strings.HasPrefix(diagnostic, "unknown_owner_invariant_ref:") || @@ -49,7 +48,7 @@ func diagnosticClassID(diagnostic string) string { return "owner_scope_violation" case strings.HasPrefix(diagnostic, "test_inventory_failed:"): return "failed_test_inventory" - case strings.HasPrefix(diagnostic, "covered_by_governance_invariant_nonproduct:"): + case strings.HasPrefix(diagnostic, "mapped_to_declared_governance_or_release_route_nonproduct:"): return "nonsemantic_governance_evidence" case strings.HasPrefix(diagnostic, "not_applicable:"): return "not_applicable_with_reason" diff --git a/internal/command/requirementcoverageview/envelope.go b/internal/command/requirementcoverageview/envelope.go index c56aa5f..af90930 100644 --- a/internal/command/requirementcoverageview/envelope.go +++ b/internal/command/requirementcoverageview/envelope.go @@ -38,11 +38,11 @@ func agentEnvelope(view map[string]any, renderer cliexec.Renderer) map[string]an { "commandIds": []any{}, "evidenceRefs": []any{"proofkit.requirement-coverage-view.summary"}, - "instruction": "Use the coverage view to identify requirements, commands, and test surfaces lacking semantic falsifiers.", + "instruction": "Use the coverage view to identify requirements, commands, and test surfaces lacking declared falsification routes.", "nonClaims": []any{"This action does not approve merge or create tests."}, "owner": "consumer_repository", "phase": "review", - "rationale": "Semantic coverage cannot be inferred from route bindings alone.", + "rationale": "Route declarations establish mapping only; execution-backed semantic evidence requires separate owners.", "stepId": "proofkit.coverage.review-gaps", }, }, @@ -63,14 +63,15 @@ func agentEnvelope(view map[string]any, renderer cliexec.Renderer) map[string]an ContextRefs: []map[string]any{ contextRef("proofkit.requirement-coverage-view.summary", "/guidanceSummary", "Coverage guidance summary."), contextRef("proofkit.requirement-coverage-view.failures", "/failures", "Fail-closed coverage diagnostics."), - contextRef("proofkit.requirement-coverage-view.owner-invariants", "/ownerInvariantCoverage", "Per-owner-invariant coverage states."), - contextRef("proofkit.requirement-coverage-view.requirements", "/requirementCoverage", "Per-requirement coverage states."), - contextRef("proofkit.requirement-coverage-view.commands", "/commandCoverage", "Per-command semantic falsifier states."), + contextRef("proofkit.requirement-coverage-view.owner-invariants", "/ownerInvariantCoverage", "Per-owner-invariant route-mapping states."), + contextRef("proofkit.requirement-coverage-view.requirements", "/requirementCoverage", "Per-requirement route-mapping states."), + contextRef("proofkit.requirement-coverage-view.commands", "/commandCoverage", "Per-command declared-route states."), }, EnvelopeID: "proofkit.requirement-coverage-view.agent-envelope", NonClaims: []string{ "Requirement coverage envelopes do not scan repositories.", "Requirement coverage envelopes do not execute tests or approve merge.", + "Requirement coverage envelopes do not prove selector resolution, oracle quality, freshness, trust, sensitivity, or assurance closure.", }, Omitted: []map[string]any{}, ReceiptRefs: []map[string]any{}, diff --git a/internal/command/requirementcoverageview/mapping_ledger.go b/internal/command/requirementcoverageview/mapping_ledger.go new file mode 100644 index 0000000..230469a --- /dev/null +++ b/internal/command/requirementcoverageview/mapping_ledger.go @@ -0,0 +1,223 @@ +package requirementcoverageview + +import "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + +type coverageStateDescriptor struct { + commandPriority int + commandState string + evidenceClass string + ownerInvariantAdmissible bool + ownerInvariantWarns bool + requirementAdmissible bool + requirementMappingComplete bool + requirementPriority int + requirementState string +} + +var coverageStateDescriptors = []coverageStateDescriptor{ + { + commandPriority: 0, + commandState: "command_declared_semantic_falsifier_route_present", + evidenceClass: testevidenceinventory.EvidenceClassDeclaredSemanticFalsifierRoute, + ownerInvariantAdmissible: true, + requirementAdmissible: true, + requirementMappingComplete: true, + requirementPriority: 0, + requirementState: "mapped_to_declared_semantic_falsifier_route", + }, + { + commandPriority: 1, + commandState: "command_owner_nonsemantic_evidence", + evidenceClass: testevidenceinventory.EvidenceClassDeclaredPropertyOrFuzzRoute, + ownerInvariantAdmissible: true, + requirementAdmissible: true, + requirementMappingComplete: true, + requirementPriority: 1, + requirementState: "mapped_to_declared_property_or_fuzz_route", + }, + { + commandPriority: 1, + commandState: "command_owner_nonsemantic_evidence", + evidenceClass: testevidenceinventory.EvidenceClassDeclaredContractAdmissionRoute, + ownerInvariantAdmissible: true, + requirementAdmissible: true, + requirementMappingComplete: true, + requirementPriority: 2, + requirementState: "mapped_to_declared_contract_admission_route", + }, + { + commandPriority: 1, + commandState: "command_owner_nonsemantic_evidence", + evidenceClass: "governance_or_release", + ownerInvariantAdmissible: true, + requirementAdmissible: true, + requirementPriority: 3, + requirementState: "mapped_to_declared_governance_or_release_route_nonproduct", + }, + { + commandPriority: 1, + commandState: "command_owner_nonsemantic_evidence", + evidenceClass: "benchmark", + ownerInvariantAdmissible: true, + ownerInvariantWarns: true, + requirementAdmissible: true, + requirementPriority: 4, + requirementState: "benchmark_advisory_missing_policy", + }, + { + commandPriority: 2, + commandState: "command_proof_route_candidate_only", + evidenceClass: testevidenceinventory.EvidenceClassProofRouteCandidate, + ownerInvariantAdmissible: true, + ownerInvariantWarns: true, + requirementAdmissible: true, + requirementPriority: 5, + requirementState: "proof_route_candidate_only", + }, + { + commandPriority: 3, + commandState: "command_route_only_nonclaim", + evidenceClass: "routing_smoke_nonclaim", + ownerInvariantAdmissible: true, + ownerInvariantWarns: true, + requirementAdmissible: true, + requirementPriority: 6, + requirementState: "route_only_nonclaim", + }, + { + commandPriority: 1, + commandState: "command_owner_nonsemantic_evidence", + evidenceClass: "helper_or_testkit", + ownerInvariantAdmissible: true, + ownerInvariantWarns: true, + requirementAdmissible: true, + requirementPriority: 7, + requirementState: "helper_or_testkit_nonclaim", + }, + { + evidenceClass: "", + requirementAdmissible: true, + requirementState: "deferred_with_owner", + }, + { + evidenceClass: "", + requirementAdmissible: true, + requirementState: "missing_proof_binding_route", + }, + { + evidenceClass: "", + ownerInvariantAdmissible: true, + ownerInvariantWarns: true, + requirementAdmissible: true, + requirementState: "missing_test_inventory", + }, + { + evidenceClass: "", + requirementAdmissible: true, + requirementState: "not_applicable", + }, +} + +const missingCommandCoverageState = "missing_command_declared_semantic_falsifier_route" + +func strongestMappingState(entries []testevidenceinventory.Entry) (string, string) { + bestPriority := len(coverageStateDescriptors) + 1 + bestState := "missing_test_inventory" + bestClass := "" + for _, entry := range entries { + for _, descriptor := range coverageStateDescriptors { + if descriptor.evidenceClass == "" || descriptor.evidenceClass != entry.EvidenceClass || descriptor.requirementPriority >= bestPriority { + continue + } + bestPriority = descriptor.requirementPriority + bestState = descriptor.requirementState + bestClass = descriptor.evidenceClass + } + } + return bestState, bestClass +} + +func commandState(entries []testevidenceinventory.Entry) string { + bestPriority := len(coverageStateDescriptors) + 1 + bestState := missingCommandCoverageState + for _, entry := range entries { + for _, descriptor := range coverageStateDescriptors { + if descriptor.commandState == "" || descriptor.evidenceClass != entry.EvidenceClass || descriptor.commandPriority >= bestPriority { + continue + } + bestPriority = descriptor.commandPriority + bestState = descriptor.commandState + } + } + return bestState +} + +func requirementCoverageStateDescriptor(state string) (coverageStateDescriptor, bool) { + for _, descriptor := range coverageStateDescriptors { + if descriptor.requirementAdmissible && descriptor.requirementState == state { + return descriptor, true + } + } + return coverageStateDescriptor{}, false +} + +func ownerInvariantCoverageStateDescriptor(state string) (coverageStateDescriptor, bool) { + for _, descriptor := range coverageStateDescriptors { + if descriptor.ownerInvariantAdmissible && descriptor.requirementState == state { + return descriptor, true + } + } + return coverageStateDescriptor{}, false +} + +func commandCoverageStateAllowed(state string) bool { + if state == missingCommandCoverageState { + return true + } + for _, descriptor := range coverageStateDescriptors { + if descriptor.commandState == state { + return true + } + } + return false +} + +func commandCoverageDiagnostics(commandID string, state string, completenessDeclaration string) ([]string, []string) { + failures := []string{} + warnings := []string{} + diagnostic := state + ":" + commandID + switch state { + case missingCommandCoverageState: + if completenessDeclaration == "selected_paths_advisory" { + warnings = append(warnings, diagnostic) + } else { + failures = append(failures, diagnostic) + } + case "command_owner_nonsemantic_evidence": + diagnostic = "nonsemantic_command_evidence:" + commandID + if completenessDeclaration == "selected_paths_advisory" { + warnings = append(warnings, diagnostic) + } else { + failures = append(failures, diagnostic) + } + case "command_route_only_nonclaim": + warnings = append(warnings, diagnostic) + case "command_proof_route_candidate_only": + if completenessDeclaration == "selected_paths_advisory" { + warnings = append(warnings, diagnostic) + } else { + failures = append(failures, diagnostic) + } + } + return failures, warnings +} + +func requirementCoverageStateComplete(state string) bool { + descriptor, ok := requirementCoverageStateDescriptor(state) + return ok && descriptor.requirementMappingComplete +} + +func ownerInvariantCoverageStateWarns(state string) bool { + descriptor, ok := ownerInvariantCoverageStateDescriptor(state) + return !ok || descriptor.ownerInvariantWarns +} diff --git a/internal/command/requirementcoverageview/mapping_ledger_test.go b/internal/command/requirementcoverageview/mapping_ledger_test.go new file mode 100644 index 0000000..db96560 --- /dev/null +++ b/internal/command/requirementcoverageview/mapping_ledger_test.go @@ -0,0 +1,36 @@ +package requirementcoverageview + +import ( + "slices" + "sort" + "testing" + + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" +) + +func TestCoverageStateLedgerExactlyCoversEvidenceVocabulary(t *testing.T) { + classes := make([]string, 0, len(coverageStateDescriptors)) + requirementStates := map[string]struct{}{} + requirementPriorities := map[int]struct{}{} + for _, descriptor := range coverageStateDescriptors { + if _, duplicate := requirementStates[descriptor.requirementState]; duplicate { + t.Fatalf("duplicate requirement coverage state %q", descriptor.requirementState) + } + requirementStates[descriptor.requirementState] = struct{}{} + if descriptor.evidenceClass == "" { + continue + } + classes = append(classes, descriptor.evidenceClass) + if _, duplicate := requirementPriorities[descriptor.requirementPriority]; duplicate { + t.Fatalf("duplicate evidence requirement priority %d", descriptor.requirementPriority) + } + requirementPriorities[descriptor.requirementPriority] = struct{}{} + if !descriptor.requirementAdmissible || !descriptor.ownerInvariantAdmissible || descriptor.commandState == "" { + t.Fatalf("evidence class %q lacks a complete coverage projection", descriptor.evidenceClass) + } + } + sort.Strings(classes) + if want := testevidenceinventory.EvidenceClasses(); !slices.Equal(classes, want) { + t.Fatalf("coverage evidence classes=%v, want inventory-owned vocabulary %v", classes, want) + } +} diff --git a/internal/command/requirementcoverageview/output_admission.go b/internal/command/requirementcoverageview/output_admission.go index 019386a..3e692f1 100644 --- a/internal/command/requirementcoverageview/output_admission.go +++ b/internal/command/requirementcoverageview/output_admission.go @@ -4,6 +4,7 @@ import ( "bytes" "encoding/json" "fmt" + "slices" "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" @@ -13,17 +14,30 @@ import ( var outputKeys = []string{ "authority", "bindingId", "commandCoverage", "commandCoverageCount", - "completenessDeclaration", "contractId", "coverageUniverseId", "deadZones", + "completenessDeclaration", "contractId", "coverageBasis", "coverageUniverseId", "deadZones", "failureClassifications", "failureCount", "failures", "guidanceSummary", "nonClaims", "ownerInvariantCoverage", "ownerInvariantCoverageCount", "ownerInvariantRegistryId", "proofMode", "requirementCoverage", "requirementCoverageCount", "schemaVersion", "sourceId", "state", - "testInventoryId", "viewInputId", "viewKind", "warningClassifications", + "testInventoryId", "unmappedTests", "viewInputId", "viewKind", "warningClassifications", "warningCount", "warnings", } +type coverageRowDescriptor struct { + rowsKey string + countKey string + idKey string +} + +var coverageRowDescriptors = []coverageRowDescriptor{ + {rowsKey: "requirementCoverage", countKey: "requirementCoverageCount", idKey: "requirementId"}, + {rowsKey: "ownerInvariantCoverage", countKey: "ownerInvariantCoverageCount", idKey: "ownerInvariantId"}, + {rowsKey: "commandCoverage", countKey: "commandCoverageCount", idKey: "commandId"}, +} + // AdmitOutput preserves the requirement-coverage owner's wire projection for -// parent commands without reinterpreting coverage semantics. +// parent commands and replays retained report, requirement, and owner-invariant +// derivations without claiming source authenticity. func AdmitOutput(raw any) (map[string]any, error) { record, ok := raw.(map[string]any) if !ok { @@ -32,8 +46,13 @@ func AdmitOutput(raw any) (map[string]any, error) { if err := admit.KnownKeys(record, outputKeys, "requirement coverage output"); err != nil { return nil, err } - if !admit.JSONNumberEquals(record["schemaVersion"], 1) && record["schemaVersion"] != 1 { - return nil, fmt.Errorf("requirement coverage output schemaVersion must be 1") + for _, key := range outputKeys { + if _, ok := record[key]; !ok { + return nil, fmt.Errorf("requirement coverage output is missing required field %s", key) + } + } + if !admit.JSONNumberEquals(record["schemaVersion"], 2) && record["schemaVersion"] != 2 { + return nil, fmt.Errorf("requirement coverage output schemaVersion must be 2") } if record["viewKind"] != "proofkit.requirement-coverage-view" || record["authority"] != "lookup_only" { return nil, fmt.Errorf("requirement coverage output identity is invalid") @@ -43,29 +62,28 @@ func AdmitOutput(raw any) (map[string]any, error) { return nil, err } } + nonClaims, err := admit.PreserveSortedTextArray(record["nonClaims"], "requirement coverage output nonClaims", false) + if err != nil { + return nil, err + } + for _, nonClaim := range defaultNonClaims { + if !slices.Contains(nonClaims, nonClaim) { + return nil, fmt.Errorf("requirement coverage output nonClaims must retain command-owned boundary %q", nonClaim) + } + } if _, err := admit.Enum(record["state"], map[string]struct{}{"failed": {}, "passed": {}}, "requirement coverage output state"); err != nil { return nil, err } - for _, row := range []struct { - rowsKey string - countKey string - idKey string - }{ - {rowsKey: "requirementCoverage", countKey: "requirementCoverageCount", idKey: "requirementId"}, - {rowsKey: "ownerInvariantCoverage", countKey: "ownerInvariantCoverageCount", idKey: "ownerInvariantId"}, - {rowsKey: "commandCoverage", countKey: "commandCoverageCount", idKey: "commandId"}, - } { + for _, row := range coverageRowDescriptors { if err := admitCoverageOutputRows(record, row.rowsKey, row.countKey, row.idKey); err != nil { return nil, err } } - for _, pair := range [][2]string{{"failures", "failureCount"}, {"warnings", "warningCount"}} { - if err := admitCountedArray(record, pair[0], pair[1]); err != nil { - return nil, err - } + if err := admitNestedRecords(record["unmappedTests"], projectedTestKeys, "requirement coverage output unmappedTests"); err != nil { + return nil, err } - if _, ok := record["deadZones"].([]any); !ok { - return nil, fmt.Errorf("requirement coverage output deadZones must be an array") + if err := validateCoverageOutputSemantics(record); err != nil { + return nil, err } findings, err := secretjson.Scan(record, "requirement_coverage_output") if err != nil || len(findings) > 0 { @@ -90,7 +108,7 @@ func SelectRequirements(output map[string]any, selected map[string]struct{}) map for _, raw := range output["requirementCoverage"].([]any) { row := raw.(map[string]any) if _, ok := selected[row["requirementId"].(string)]; ok { - rows = append(rows, row) + rows = append(rows, cloneCoverageJSONValue(row)) } } return map[string]any{ @@ -112,15 +130,21 @@ func admitCoverageOutputRows(record map[string]any, rowsKey, countKey, idKey str if !wireCountEquals(record[countKey], len(rows)) { return fmt.Errorf("requirement coverage output %s does not match %s", countKey, rowsKey) } - seen := map[string]struct{}{} + previousID := "" for index, raw := range rows { row, ok := raw.(map[string]any) if !ok { return fmt.Errorf("requirement coverage output %s[%d] must be an object", rowsKey, index) } - if err := admit.KnownKeys(row, coverageRowKeys(rowsKey), "requirement coverage output "+rowsKey); err != nil { + rowKeys := coverageRowKeys(rowsKey) + if err := admit.KnownKeys(row, rowKeys, "requirement coverage output "+rowsKey); err != nil { return err } + for _, key := range rowKeys { + if _, ok := row[key]; !ok { + return fmt.Errorf("requirement coverage output %s[%d] is missing required field %s", rowsKey, index, key) + } + } if err := admitCoverageNestedRows(row, rowsKey); err != nil { return err } @@ -128,29 +152,45 @@ func admitCoverageOutputRows(record map[string]any, rowsKey, countKey, idKey str if err != nil { return err } - if _, exists := seen[id]; exists { - return fmt.Errorf("requirement coverage output %s identities must be unique", rowsKey) + if previousID != "" && previousID >= id { + return fmt.Errorf("requirement coverage output %s must be sorted and unique by %s", rowsKey, idKey) } - seen[id] = struct{}{} + previousID = id } return nil } -func admitCoverageNestedRows(row map[string]any, rowsKey string) error { - if rowsKey == "commandCoverage" { - return nil +func cloneCoverageJSONValue(value any) any { + switch typed := value.(type) { + case map[string]any: + cloned := make(map[string]any, len(typed)) + for key, child := range typed { + cloned[key] = cloneCoverageJSONValue(child) + } + return cloned + case []any: + cloned := make([]any, len(typed)) + for index, child := range typed { + cloned[index] = cloneCoverageJSONValue(child) + } + return cloned + default: + return typed } - if err := admitNestedRecords(row["tests"], []string{"commandRefs", "evidenceClass", "expectedPublicOutcome", "negativeCaseId", "nonClaims", "oracleKind", "oracleSummary", "qualityFindings", "selector", "sourcePath", "testId", "witnessRefs", "wrongImplementationClassId"}, "requirement coverage output tests"); err != nil { +} + +func admitCoverageNestedRows(row map[string]any, rowsKey string) error { + if err := admitNestedRecords(row["tests"], projectedTestKeys, "requirement coverage output tests"); err != nil { return err } for _, raw := range row["tests"].([]any) { test := raw.(map[string]any) - if err := admitNestedRecords(test["qualityFindings"], []string{"class", "evidenceRefs", "findingId", "nonClaims", "ownerReviewState", "severity"}, "requirement coverage output quality findings"); err != nil { + if err := admitNestedRecords(test["qualityFindings"], projectedQualityFindingKeys, "requirement coverage output quality findings"); err != nil { return err } } if rowsKey == "requirementCoverage" { - return admitNestedRecords(row["scenarios"], []string{"commandIds", "environmentClasses", "scenarioId", "witnessId", "witnessKind", "witnessPath"}, "requirement coverage output scenarios") + return admitNestedRecords(row["scenarios"], []string{"commandIds", "environmentClasses", "scenarioId", "verifyCommands", "witnessId", "witnessKind", "witnessPath", "witnessSelectors"}, "requirement coverage output scenarios") } return nil } @@ -179,23 +219,12 @@ func coverageRowKeys(rowsKey string) []string { case "ownerInvariantCoverage": return []string{"coverageState", "evidenceClass", "nonClaims", "ownerId", "ownerInvariantId", "sourcePath", "summary", "testIds", "tests", "warnings"} case "commandCoverage": - return []string{"commandId", "coverageState", "failures", "testIds"} + return []string{"commandId", "coverageState", "failures", "testIds", "tests"} default: return nil } } -func admitCountedArray(record map[string]any, rowsKey, countKey string) error { - rows, ok := record[rowsKey].([]any) - if !ok { - return fmt.Errorf("requirement coverage output %s must be an array", rowsKey) - } - if !wireCountEquals(record[countKey], len(rows)) { - return fmt.Errorf("requirement coverage output %s does not match %s", countKey, rowsKey) - } - return nil -} - func wireCountEquals(raw any, expected int) bool { switch value := raw.(type) { case int: diff --git a/internal/command/requirementcoverageview/output_basis.go b/internal/command/requirementcoverageview/output_basis.go new file mode 100644 index 0000000..7521600 --- /dev/null +++ b/internal/command/requirementcoverageview/output_basis.go @@ -0,0 +1,168 @@ +package requirementcoverageview + +import ( + "fmt" + "slices" + "sort" + + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/kernel/digest" +) + +var coverageBasisKeys = []string{ + "fullRepositoryOutOfScopeSourceRequirements", "ownerIds", "testInventoryDigest", +} + +var sourceRequirementOwnerKeys = []string{"ownerId", "requirementId"} + +type admittedCoverageBasis struct { + ownerSet map[string]struct{} + ownerScopeFailures []string +} + +func buildCoverageBasis(input compositeInput, entries []testevidenceinventory.Entry) (map[string]any, error) { + ownerSet := mapSet(input.CoverageUniverse.OwnerIDs) + outOfScope := []any{} + if input.CoverageUniverse.CompletenessDeclaration == "full_repository" { + requirements := slices.Clone(input.Source.Requirements) + sort.Slice(requirements, func(left, right int) bool { + return requirements[left].RequirementID < requirements[right].RequirementID + }) + for _, requirement := range requirements { + if inOwnerScope(requirement.OwnerID, ownerSet) { + continue + } + outOfScope = append(outOfScope, map[string]any{ + "ownerId": requirement.OwnerID, "requirementId": requirement.RequirementID, + }) + } + } + var inventoryDigest any + if input.Inventory != nil { + value, err := testInventoryProjectionDigest(entries) + if err != nil { + return nil, err + } + inventoryDigest = value + } + return map[string]any{ + "fullRepositoryOutOfScopeSourceRequirements": outOfScope, + "ownerIds": admit.StringSliceToAny(input.CoverageUniverse.OwnerIDs), + "testInventoryDigest": inventoryDigest, + }, nil +} + +func admitCoverageBasis(raw any, inventoryMissing bool, completenessDeclaration string, entries []testevidenceinventory.Entry) (admittedCoverageBasis, error) { + record, ok := raw.(map[string]any) + if !ok { + return admittedCoverageBasis{}, fmt.Errorf("requirement coverage output coverageBasis must be an object") + } + if err := admit.KnownKeys(record, coverageBasisKeys, "requirement coverage output coverageBasis"); err != nil { + return admittedCoverageBasis{}, err + } + for _, key := range coverageBasisKeys { + if _, ok := record[key]; !ok { + return admittedCoverageBasis{}, fmt.Errorf("requirement coverage output coverageBasis is missing required field %s", key) + } + } + ownerIDs, err := admit.PreserveSortedTextArray(record["ownerIds"], "requirement coverage output coverageBasis ownerIds", false) + if err != nil { + return admittedCoverageBasis{}, err + } + for index, ownerID := range ownerIDs { + if _, err := admit.RuleID(ownerID, fmt.Sprintf("requirement coverage output coverageBasis ownerIds[%d]", index)); err != nil { + return admittedCoverageBasis{}, err + } + } + ownerSet := mapSet(ownerIDs) + outOfScope, err := admitOutOfScopeSourceRequirements(record["fullRepositoryOutOfScopeSourceRequirements"], ownerSet, completenessDeclaration) + if err != nil { + return admittedCoverageBasis{}, err + } + if inventoryMissing { + if record["testInventoryDigest"] != nil { + return admittedCoverageBasis{}, fmt.Errorf("requirement coverage output coverageBasis testInventoryDigest must be null without testInventoryId") + } + } else { + actual, err := admit.SHA256Ref(record["testInventoryDigest"], "requirement coverage output coverageBasis testInventoryDigest") + if err != nil { + return admittedCoverageBasis{}, err + } + if err := requireCanonicalWireString(record["testInventoryDigest"], actual, "requirement coverage output coverageBasis testInventoryDigest"); err != nil { + return admittedCoverageBasis{}, err + } + expected, err := testInventoryProjectionDigest(entries) + if err != nil { + return admittedCoverageBasis{}, err + } + if actual != expected { + return admittedCoverageBasis{}, fmt.Errorf("requirement coverage output coverageBasis testInventoryDigest does not match retained test projections") + } + } + failures := make([]string, 0, len(outOfScope)+len(entries)) + for _, requirement := range outOfScope { + failures = append(failures, "full_repository_source_requirement_outside_owner_scope:"+requirement.requirementID) + } + for _, entry := range entries { + if _, ok := ownerSet[entry.OwnerID]; !ok { + failures = append(failures, "inventory_entry_owner_outside_scope:"+entry.TestID+":"+entry.OwnerID) + } + } + return admittedCoverageBasis{ownerSet: ownerSet, ownerScopeFailures: sortedUnique(failures)}, nil +} + +type sourceRequirementOwner struct { + ownerID string + requirementID string +} + +func admitOutOfScopeSourceRequirements(raw any, ownerSet map[string]struct{}, completenessDeclaration string) ([]sourceRequirementOwner, error) { + values, ok := raw.([]any) + if !ok { + return nil, fmt.Errorf("requirement coverage output coverageBasis fullRepositoryOutOfScopeSourceRequirements must be an array") + } + if completenessDeclaration != "full_repository" && len(values) != 0 { + return nil, fmt.Errorf("requirement coverage output coverageBasis must not retain full-repository omissions outside full_repository scope") + } + result := make([]sourceRequirementOwner, 0, len(values)) + previousID := "" + for index, rawValue := range values { + record, ok := rawValue.(map[string]any) + if !ok { + return nil, fmt.Errorf("requirement coverage output coverageBasis fullRepositoryOutOfScopeSourceRequirements[%d] must be an object", index) + } + context := fmt.Sprintf("requirement coverage output coverageBasis fullRepositoryOutOfScopeSourceRequirements[%d]", index) + if err := admit.KnownKeys(record, sourceRequirementOwnerKeys, context); err != nil { + return nil, err + } + for _, key := range sourceRequirementOwnerKeys { + if _, ok := record[key]; !ok { + return nil, fmt.Errorf("%s is missing required field %s", context, key) + } + } + requirementID, err := admit.RuleID(record["requirementId"], context+" requirementId") + if err != nil { + return nil, err + } + if previousID != "" && previousID >= requirementID { + return nil, fmt.Errorf("requirement coverage output coverageBasis fullRepositoryOutOfScopeSourceRequirements must be sorted and unique by requirementId") + } + previousID = requirementID + ownerID, err := admit.RuleID(record["ownerId"], context+" ownerId") + if err != nil { + return nil, err + } + if _, inScope := ownerSet[ownerID]; inScope { + return nil, fmt.Errorf("%s ownerId must be outside coverageBasis ownerIds", context) + } + result = append(result, sourceRequirementOwner{ownerID: ownerID, requirementID: requirementID}) + } + return result, nil +} + +func testInventoryProjectionDigest(entries []testevidenceinventory.Entry) (string, error) { + ordered := slices.Clone(entries) + sort.Slice(ordered, func(left, right int) bool { return ordered[left].TestID < ordered[right].TestID }) + return digest.StableJSONSHA256Ref(testEntriesToAny(ordered)) +} diff --git a/internal/command/requirementcoverageview/output_closure_test.go b/internal/command/requirementcoverageview/output_closure_test.go new file mode 100644 index 0000000..b0ccab5 --- /dev/null +++ b/internal/command/requirementcoverageview/output_closure_test.go @@ -0,0 +1,641 @@ +package requirementcoverageview + +import ( + "encoding/json" + "strings" + "testing" +) + +func TestAdmitOutputRejectsRequirementProofProjectionDrift(t *testing.T) { + tests := []struct { + name string + mutate func(map[string]any) + want string + }{ + { + name: "scenario removal", + mutate: func(row map[string]any) { + row["scenarios"] = []any{} + row["scenarioCount"] = 0 + }, + want: "proofState and scenarios are inconsistent", + }, + { + name: "scenario count", + mutate: func(row map[string]any) { + row["scenarioCount"] = 0 + }, + want: "scenarioCount does not match scenarios", + }, + { + name: "command union", + mutate: func(row map[string]any) { + row["commandIds"] = []any{} + }, + want: "commandIds are not derived from scenarios", + }, + { + name: "environment union", + mutate: func(row map[string]any) { + row["environmentClasses"] = []any{} + }, + want: "environmentClasses are not derived from scenarios", + }, + { + name: "witness union", + mutate: func(row map[string]any) { + row["witnessRefs"] = []any{} + }, + want: "witnessRefs are not derived from scenarios", + }, + } + for _, item := range tests { + t.Run(item.name, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + item.mutate(record["requirementCoverage"].([]any)[0].(map[string]any)) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.want) { + t.Fatalf("AdmitOutput() error=%v, want %q", err, item.want) + } + }) + } +} + +func TestAdmitOutputRejectsIncompleteProjectedTestFields(t *testing.T) { + for _, field := range projectedTestKeys { + t.Run(field, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + mutateProjectedTestCopies(t, record, "test.coverage.semantic", func(test map[string]any) { + delete(test, field) + }) + if _, err := AdmitOutput(record); err == nil { + t.Fatalf("AdmitOutput() admitted projected test without %s", field) + } + }) + } +} + +func TestAdmitOutputRejectsMissingInverseParentProjection(t *testing.T) { + for _, rowsKey := range []string{"requirementCoverage", "ownerInvariantCoverage", "commandCoverage"} { + t.Run(rowsKey, func(t *testing.T) { + record := coverageRecordWithOwnerInvariant(t) + row := record[rowsKey].([]any)[0].(map[string]any) + row["tests"] = []any{} + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "missing from retained parent") { + t.Fatalf("AdmitOutput() error=%v, want inverse parent projection rejection", err) + } + }) + } +} + +func TestAdmitOutputRejectsCompactProjectionDrift(t *testing.T) { + buildCompact := func(t *testing.T) map[string]any { + t.Helper() + input := validCoverageInput(t).(map[string]any) + input["requirementProofBinding"] = nil + input["compactProofContract"] = validCompactCoverageContract() + input["localEnvironmentPolicy"] = map[string]any{ + "authority": "caller_provided", "localEnvironmentClasses": []any{"local-go"}, + } + inventoryEntry(input)["witnessRefs"] = []any{} + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + return view.(map[string]any) + } + for _, item := range []struct { + name string + mutate func(map[string]any) + want string + }{ + {name: "command ids", mutate: func(row map[string]any) { row["commandIds"] = []any{"proofkit.coverage.other"} }, want: "compact commandIds are not derived from projected tests"}, + {name: "witness selector", mutate: func(row map[string]any) { row["witnessSelectors"] = []any{"not-a-selector"} }, want: "selector identity"}, + {name: "verify command", mutate: func(row map[string]any) { row["verifyCommands"] = []any{"go test ./...; false"} }, want: "shell control tokens"}, + {name: "valid witness selector substitution", mutate: func(row map[string]any) { + row["witnessSelectors"] = []any{"internal/command/requirementcoverageview/requirementcoverageview_test.go::falsification.substituted"} + }, want: "compact witnessSelectors are not derived from scenarios"}, + {name: "valid verify command substitution", mutate: func(row map[string]any) { + row["verifyCommands"] = []any{"go test ./internal/command/requirementcontext"} + }, want: "compact verifyCommands are not derived from scenarios"}, + {name: "valid but unscoped scenario id", mutate: func(row map[string]any) { + row["scenarios"].([]any)[0].(map[string]any)["scenarioId"] = "arbitrary-scenario" + }, want: "surface_id::stable_anchor"}, + } { + t.Run(item.name, func(t *testing.T) { + record := buildCompact(t) + item.mutate(record["requirementCoverage"].([]any)[0].(map[string]any)) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.want) { + t.Fatalf("AdmitOutput() error=%v, want %q", err, item.want) + } + }) + } +} + +func TestAdmitOutputRequiresEveryDeclaredRootField(t *testing.T) { + for _, field := range outputKeys { + t.Run(field, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + delete(record, field) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "missing required field "+field) { + t.Fatalf("AdmitOutput() error=%v, want missing required field %s", err, field) + } + }) + } +} + +func TestAdmitOutputRequiresEveryCoverageRowField(t *testing.T) { + for _, descriptor := range coverageRowDescriptors { + for _, field := range coverageRowKeys(descriptor.rowsKey) { + t.Run(descriptor.rowsKey+"/"+field, func(t *testing.T) { + record := coverageRecordWithOwnerInvariant(t) + delete(record[descriptor.rowsKey].([]any)[0].(map[string]any), field) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "missing required field "+field) { + t.Fatalf("AdmitOutput() error=%v, want missing required field %s", err, field) + } + }) + } + } +} + +func TestAdmitOutputValidatesEveryCoverageRowMetadataField(t *testing.T) { + tests := []struct { + name string + rowsKey string + field string + value any + }{ + {name: "requirement claim level", rowsKey: "requirementCoverage", field: "claimLevel", value: json.Number("7")}, + {name: "requirement invariant", rowsKey: "requirementCoverage", field: "invariant", value: json.Number("7")}, + {name: "requirement lifecycle", rowsKey: "requirementCoverage", field: "lifecycleState", value: "unknown"}, + {name: "requirement non claims", rowsKey: "requirementCoverage", field: "nonClaims", value: "not-an-array"}, + {name: "requirement owner", rowsKey: "requirementCoverage", field: "ownerId", value: json.Number("7")}, + {name: "requirement path", rowsKey: "requirementCoverage", field: "specPath", value: "docs/../outside.json"}, + {name: "owner invariant non claims", rowsKey: "ownerInvariantCoverage", field: "nonClaims", value: "not-an-array"}, + {name: "owner invariant owner", rowsKey: "ownerInvariantCoverage", field: "ownerId", value: json.Number("7")}, + {name: "owner invariant path", rowsKey: "ownerInvariantCoverage", field: "sourcePath", value: "docs/../outside.json"}, + {name: "owner invariant summary", rowsKey: "ownerInvariantCoverage", field: "summary", value: json.Number("7")}, + } + for _, item := range tests { + t.Run(item.name, func(t *testing.T) { + record := coverageRecordWithOwnerInvariant(t) + record[item.rowsKey].([]any)[0].(map[string]any)[item.field] = item.value + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.field) { + t.Fatalf("AdmitOutput() error=%v, want %s metadata rejection", err, item.field) + } + }) + } +} + +func TestAdmitOutputRejectsNonCanonicalWireProjectionText(t *testing.T) { + tests := []struct { + name string + field string + mutate func(map[string]any) + }{ + { + name: "requirement invariant", + field: "invariant", + mutate: func(record map[string]any) { + row := record["requirementCoverage"].([]any)[0].(map[string]any) + row["invariant"] = " " + row["invariant"].(string) + " " + }, + }, + { + name: "test selector", + field: "selector", + mutate: func(record map[string]any) { + row := record["requirementCoverage"].([]any)[0].(map[string]any) + test := row["tests"].([]any)[0].(map[string]any) + test["selector"] = " " + test["selector"].(string) + " " + }, + }, + { + name: "test inventory digest", + field: "testInventoryDigest", + mutate: func(record map[string]any) { + basis := record["coverageBasis"].(map[string]any) + basis["testInventoryDigest"] = " " + basis["testInventoryDigest"].(string) + " " + }, + }, + } + for _, item := range tests { + t.Run(item.name, func(t *testing.T) { + record := coverageRecordWithOwnerInvariant(t) + item.mutate(record) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.field) { + t.Fatalf("AdmitOutput() error=%v, want non-canonical %s rejection", err, item.field) + } + }) + } +} + +func TestAdmitOutputRequiresCanonicalCoverageRowOrder(t *testing.T) { + secondIDs := map[string]string{ + "requirementCoverage": "REQ-PROOFKIT-COVERAGE-000", + "ownerInvariantCoverage": "invariant.coverage.alpha", + "commandCoverage": "proofkit.coverage.alpha", + } + for _, descriptor := range coverageRowDescriptors { + t.Run(descriptor.rowsKey, func(t *testing.T) { + record := coverageRecordWithOwnerInvariant(t) + rows := record[descriptor.rowsKey].([]any) + second := cloneCoverageJSONValue(rows[0]).(map[string]any) + second[descriptor.idKey] = secondIDs[descriptor.rowsKey] + record[descriptor.rowsKey] = append(rows, second) + record[descriptor.countKey] = 2 + if err := admitCoverageOutputRows(record, descriptor.rowsKey, descriptor.countKey, descriptor.idKey); err == nil || !strings.Contains(err.Error(), "sorted and unique") { + t.Fatalf("admitCoverageOutputRows() error=%v, want canonical-order rejection", err) + } + }) + } +} + +func TestAdmitOutputRetainsCommandOwnedNonClaims(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + record["nonClaims"] = []any{"Benign caller-owned non-claim."} + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "command-owned boundary") { + t.Fatalf("AdmitOutput() error=%v, want command-owned non-claim rejection", err) + } +} + +func TestAdmitOutputReplaysMissingInventoryWarning(t *testing.T) { + input := validCoverageInput(t).(map[string]any) + input["testEvidenceInventory"] = nil + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + setCoverageAggregateDiagnostics( + record, + stringArray(record["failures"]), + withoutDiagnostic(stringArray(record["warnings"]), "missing_test_inventory:input"), + ) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "missing_test_inventory:input warning") { + t.Fatalf("AdmitOutput() error=%v, want missing-inventory warning rejection", err) + } +} + +func TestAdmitOutputRejectsRemovedValidUnmappedInventoryEntry(t *testing.T) { + input := validCoverageInput(t) + entry := inventoryEntry(input) + entry["evidenceClass"] = "helper_or_testkit" + entry["commandRefs"] = []any{} + entry["ownerInvariantRefs"] = []any{} + entry["requirementRefs"] = []any{} + entry["witnessRefs"] = []any{} + delete(entry, "falsifier") + delete(entry, "oracle") + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + if len(record["unmappedTests"].([]any)) != 1 { + t.Fatalf("unmappedTests=%#v, want one valid helper entry", record["unmappedTests"]) + } + record["unmappedTests"] = []any{} + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "testInventoryDigest") { + t.Fatalf("AdmitOutput() error=%v, want inventory digest rejection", err) + } +} + +func TestAdmitOutputReplaysOwnerScopeFailures(t *testing.T) { + input := validCoverageInput(t) + inventoryEntry(input)["ownerId"] = "proofkit.other" + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + diagnostic := "inventory_entry_owner_outside_scope:test.coverage.semantic:proofkit.other" + setCoverageAggregateDiagnostics( + record, + withoutDiagnostic(stringArray(record["failures"]), diagnostic), + stringArray(record["warnings"]), + ) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "owner-scope failures") { + t.Fatalf("AdmitOutput() error=%v, want owner-scope diagnostic rejection", err) + } +} + +func TestAdmitOutputReplaysFullRepositorySourceOwnerScopeFailures(t *testing.T) { + input := validCoverageInput(t).(map[string]any) + universe := input["coverageUniverse"].(map[string]any) + universe["completenessDeclaration"] = "full_repository" + source := input["requirementSource"].(map[string]any) + requirements := source["requirements"].([]any) + outOfScope := cloneCoverageJSONValue(requirements[0]).(map[string]any) + outOfScope["requirementId"] = "REQ-PROOFKIT-COVERAGE-002" + outOfScope["ownerId"] = "proofkit.other" + outOfScope["updatePolicy"].(map[string]any)["reviewOwnerId"] = "proofkit.other" + source["requirements"] = append(requirements, outOfScope) + + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + diagnostic := "full_repository_source_requirement_outside_owner_scope:REQ-PROOFKIT-COVERAGE-002" + setCoverageAggregateDiagnostics( + record, + withoutDiagnostic(stringArray(record["failures"]), diagnostic), + stringArray(record["warnings"]), + ) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "owner-scope failures") { + t.Fatalf("AdmitOutput() error=%v, want full-repository owner-scope rejection", err) + } +} + +func TestAdmitOutputRequiresEveryCoverageBasisField(t *testing.T) { + for _, field := range coverageBasisKeys { + t.Run(field, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + delete(record["coverageBasis"].(map[string]any), field) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "missing required field "+field) { + t.Fatalf("AdmitOutput() error=%v, want missing coverageBasis field %s", err, field) + } + }) + } +} + +func TestAdmitOutputRequiresModeOwnedIdentity(t *testing.T) { + structured, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON(structured) error=%v", err) + } + compactInput := validCoverageInput(t).(map[string]any) + compactInput["requirementProofBinding"] = nil + compactInput["compactProofContract"] = validCompactCoverageContract() + compactInput["localEnvironmentPolicy"] = map[string]any{ + "authority": "caller_provided", "localEnvironmentClasses": []any{"local-go"}, + } + inventoryEntry(compactInput)["witnessRefs"] = []any{} + compact, _, err := BuildJSON(compactInput, Options{}) + if err != nil { + t.Fatalf("BuildJSON(compact) error=%v", err) + } + for _, item := range []struct { + name string + record map[string]any + mutate func(map[string]any) + want string + }{ + {name: "structured binding id", record: structured.(map[string]any), mutate: func(record map[string]any) { record["bindingId"] = "" }, want: "bindingId"}, + {name: "structured contract id", record: structured.(map[string]any), mutate: func(record map[string]any) { record["contractId"] = "proofkit.coverage.contract" }, want: "contractId must be empty"}, + {name: "compact binding id", record: compact.(map[string]any), mutate: func(record map[string]any) { record["bindingId"] = "proofkit.coverage.binding" }, want: "bindingId must be empty"}, + {name: "compact contract id", record: compact.(map[string]any), mutate: func(record map[string]any) { record["contractId"] = "" }, want: "contractId"}, + } { + t.Run(item.name, func(t *testing.T) { + encoded, err := json.Marshal(item.record) + if err != nil { + t.Fatalf("json.Marshal() error=%v", err) + } + copy := map[string]any{} + decoder := json.NewDecoder(strings.NewReader(string(encoded))) + decoder.UseNumber() + if err := decoder.Decode(©); err != nil { + t.Fatalf("json.Decode() error=%v", err) + } + item.mutate(copy) + if _, err := AdmitOutput(copy); err == nil || !strings.Contains(err.Error(), item.want) { + t.Fatalf("AdmitOutput() error=%v, want %q", err, item.want) + } + }) + } +} + +func TestAdmitOutputReplaysFailedInventoryQualitySemantics(t *testing.T) { + input := validCoverageInput(t) + inventoryEntry(input)["qualityFindings"] = []any{map[string]any{ + "class": "implementation_mirror", "evidenceRefs": []any{"test.coverage.semantic"}, + "findingId": "finding.coverage.implementation-mirror", + "nonClaims": []any{"Caller-owned finding does not prove native execution."}, + "ownerReviewState": "confirmed", "severity": "failure", + }} + view, exitCode, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + if exitCode == 0 { + t.Fatal("failure-severity quality finding must fail coverage output") + } + record := view.(map[string]any) + setCoverageAggregateDiagnostics( + record, + withoutDiagnostic(stringArray(record["failures"]), "test_inventory_failed:proofkit.coverage.inventory"), + stringArray(record["warnings"]), + ) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "test-inventory failures") { + t.Fatalf("AdmitOutput() error=%v, want retained inventory-failure rejection", err) + } +} + +func TestAdmitOutputRetainsFailedInventoryEntriesWithoutProjectedParents(t *testing.T) { + input := validCoverageInput(t) + entry := inventoryEntry(input) + entry["commandRefs"] = []any{} + entry["ownerInvariantRefs"] = []any{} + entry["requirementRefs"] = []any{} + entry["witnessRefs"] = []any{} + view, exitCode, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + if exitCode == 0 { + t.Fatal("unanchored inventory entry must fail coverage output") + } + record := view.(map[string]any) + unmapped := record["unmappedTests"].([]any) + if len(unmapped) != 1 || unmapped[0].(map[string]any)["testId"] != "test.coverage.semantic" { + t.Fatalf("unmappedTests=%#v, want exact failed inventory entry", unmapped) + } + if _, err := AdmitOutput(record); err != nil { + t.Fatalf("AdmitOutput(valid failed output) error=%v", err) + } + record["unmappedTests"] = []any{} + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "testInventoryDigest") { + t.Fatalf("AdmitOutput() error=%v, want removed unmapped-entry rejection", err) + } +} + +func coverageRecordWithOwnerInvariant(t *testing.T) map[string]any { + t.Helper() + input := validCoverageInput(t).(map[string]any) + input["ownerInvariantRegistry"] = map[string]any{ + "schemaVersion": json.Number("1"), "registryId": "proofkit.coverage.owner-invariants", + "invariants": []any{map[string]any{ + "ownerInvariantId": "invariant.coverage.semantic", "ownerId": "proofkit.coverage", + "sourcePath": "docs/specs/proofkit-coverage/requirements.v1.json", + "summary": "Coverage owner invariant fixture.", "nonClaims": []any{"Fixture does not claim execution."}, + }}, + "nonClaims": []any{"Registry fixture is caller-owned."}, + } + inventoryEntry(input)["ownerInvariantRefs"] = []any{"invariant.coverage.semantic"} + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + return view.(map[string]any) +} + +func TestBuildJSONTreatsNonWitnessProofStateAsMissingRoute(t *testing.T) { + input := validCoverageInput(t) + record := input.(map[string]any) + proof := record["requirementProofBinding"].(map[string]any) + proofRequirement := proof["requirements"].([]any)[0].(map[string]any) + proofRequirement["claimLevel"] = "advisory" + proofRequirement["proofState"] = "not_bound" + proof["bindings"] = []any{} + proof["witnessCommands"] = []any{} + sourceRequirement(input)["claimLevel"] = "advisory" + inventoryEntry(input)["witnessRefs"] = []any{} + + view, exitCode, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + if exitCode != 0 { + t.Fatalf("advisory non-witness route should remain a warning: %#v", view) + } + requirement := view.(map[string]any)["requirementCoverage"].([]any)[0].(map[string]any) + if requirement["proofState"] != "not_bound" || requirement["coverageState"] != "missing_proof_binding_route" { + t.Fatalf("non-witness proof state was not retained as a missing route: %#v", requirement) + } +} + +func TestAdmitOutputRejectsRemovedRetainedUnknownReferenceDiagnostics(t *testing.T) { + tests := []struct { + name string + mutate func(map[string]any) + want string + }{ + { + name: "requirement", + mutate: func(entry map[string]any) { + entry["requirementRefs"] = []any{"REQ-PROOFKIT-COVERAGE-999"} + }, + want: "unknown_requirement_ref:test.coverage.semantic:REQ-PROOFKIT-COVERAGE-999", + }, + { + name: "owner invariant", + mutate: func(entry map[string]any) { + entry["ownerInvariantRefs"] = []any{"invariant.coverage.unknown"} + }, + want: "unknown_owner_invariant_ref:test.coverage.semantic:invariant.coverage.unknown", + }, + { + name: "command", + mutate: func(entry map[string]any) { + entry["commandRefs"] = []any{"proofkit.coverage.unknown"} + }, + want: "unknown_command_or_witness_ref:test.coverage.semantic:proofkit.coverage.unknown", + }, + { + name: "witness", + mutate: func(entry map[string]any) { + entry["witnessRefs"] = []any{"proofkit.coverage.unknown-witness"} + }, + want: "unknown_command_or_witness_ref:test.coverage.semantic:proofkit.coverage.unknown-witness", + }, + } + for _, item := range tests { + t.Run(item.name, func(t *testing.T) { + input := validCoverageInput(t) + item.mutate(inventoryEntry(input)) + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + setCoverageAggregateDiagnostics( + record, + withoutDiagnostic(stringArray(record["failures"]), item.want), + stringArray(record["warnings"]), + ) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "unknown-reference diagnostics are inconsistent") { + t.Fatalf("AdmitOutput() error=%v, want retained unknown-reference rejection", err) + } + }) + } +} + +func TestAdmitOutputDerivesCommandStateFromCommandLocalTests(t *testing.T) { + input := validCoverageInput(t) + entry := inventoryEntry(input) + entry["evidenceClass"] = "routing_smoke_nonclaim" + entry["requirementRefs"] = []any{} + entry["witnessRefs"] = []any{} + entry["falsifier"] = nil + entry["oracle"] = nil + entry["nonClaims"] = []any{"Route-only fixture does not prove command behavior."} + + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error=%v", err) + } + record := view.(map[string]any) + command := record["commandCoverage"].([]any)[0].(map[string]any) + if len(command["tests"].([]any)) != 1 { + t.Fatalf("command-local test projection missing: %#v", command) + } + command["coverageState"] = "command_declared_semantic_falsifier_route_present" + command["failures"] = []any{} + setCoverageAggregateDiagnostics(record, stringArray(record["failures"]), withoutDiagnostic(stringArray(record["warnings"]), "command_route_only_nonclaim:proofkit.coverage.command")) + + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "coverageState is not derived from projected tests") { + t.Fatalf("AdmitOutput() error=%v, want command-local state rejection", err) + } +} + +func mutateProjectedTestCopies(t *testing.T, record map[string]any, testID string, mutate func(map[string]any)) { + t.Helper() + count := 0 + for _, rowsKey := range []string{"requirementCoverage", "ownerInvariantCoverage", "commandCoverage"} { + for _, rawRow := range record[rowsKey].([]any) { + for _, rawTest := range rawRow.(map[string]any)["tests"].([]any) { + test := rawTest.(map[string]any) + if test["testId"] == testID { + mutate(test) + count++ + } + } + } + } + if count == 0 { + t.Fatalf("projected test %s not found", testID) + } +} + +func refreshCoverageBasisInventoryDigest(t *testing.T, record map[string]any) { + t.Helper() + registry, err := admitCoverageProjectedTestRegistry(record) + if err != nil { + t.Fatalf("admitCoverageProjectedTestRegistry() error=%v", err) + } + value, err := testInventoryProjectionDigest(projectedRegistryEntries(registry)) + if err != nil { + t.Fatalf("testInventoryProjectionDigest() error=%v", err) + } + record["coverageBasis"].(map[string]any)["testInventoryDigest"] = value +} diff --git a/internal/command/requirementcoverageview/output_diagnostics.go b/internal/command/requirementcoverageview/output_diagnostics.go new file mode 100644 index 0000000..ebdb371 --- /dev/null +++ b/internal/command/requirementcoverageview/output_diagnostics.go @@ -0,0 +1,203 @@ +package requirementcoverageview + +import ( + "bytes" + "fmt" + "slices" + "strings" + + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" +) + +func expectedCoverageRowDiagnostics(row map[string]any, rowsKey string, completenessDeclaration string) ([]string, []string) { + state := stringValue(row["coverageState"]) + switch rowsKey { + case "requirementCoverage": + failures := expectedRequirementFailures(row, state, completenessDeclaration) + warnings := []string{} + if requirementMappingWarns(stringValue(row["claimLevel"]), state, completenessDeclaration) { + warnings = append(warnings, state+":"+stringValue(row["requirementId"])) + } + return failures, warnings + case "ownerInvariantCoverage": + if ownerInvariantCoverageStateWarns(state) { + return nil, []string{"missing_owner_invariant_inventory:" + stringValue(row["ownerInvariantId"])} + } + return nil, nil + case "commandCoverage": + return commandCoverageDiagnostics(stringValue(row["commandId"]), state, completenessDeclaration) + default: + return nil, nil + } +} + +func requireExactDiagnosticClass(actual []string, expected []string, context string, prefixes ...string) error { + filtered := make([]string, 0, len(actual)) + for _, diagnostic := range actual { + for _, prefix := range prefixes { + if strings.HasPrefix(diagnostic, prefix) { + filtered = append(filtered, diagnostic) + break + } + } + } + if !slices.Equal(filtered, expected) { + return fmt.Errorf("requirement coverage output %s are inconsistent with retained projection", context) + } + return nil +} + +func admitCoverageDeadZones(raw any) ([]map[string]any, error) { + values, ok := raw.([]any) + if !ok { + return nil, fmt.Errorf("requirement coverage output deadZones must be an array") + } + result := make([]map[string]any, 0, len(values)) + previousID := "" + for index, rawValue := range values { + record, ok := rawValue.(map[string]any) + if !ok { + return nil, fmt.Errorf("requirement coverage output deadZones[%d] must be an object", index) + } + context := fmt.Sprintf("requirement coverage output deadZones[%d]", index) + if err := admit.KnownKeys(record, []string{"deadZoneKind", "ownerId", "path", "surfaceId"}, context); err != nil { + return nil, err + } + kind, err := admit.Enum(record["deadZoneKind"], map[string]struct{}{ + "unbound_code_surface": {}, "unbound_spec_surface": {}, "unbound_test_surface": {}, + }, context+" deadZoneKind") + if err != nil { + return nil, err + } + ownerID, err := admit.RuleID(record["ownerId"], context+" ownerId") + if err != nil { + return nil, err + } + path, err := admit.SafeRepoRelativePath(stringValue(record["path"]), context+" path") + if err != nil { + return nil, err + } + surfaceID, err := admit.RuleID(record["surfaceId"], context+" surfaceId") + if err != nil { + return nil, err + } + if previousID != "" && previousID >= surfaceID { + return nil, fmt.Errorf("requirement coverage output deadZones must be sorted and unique by surfaceId") + } + previousID = surfaceID + result = append(result, map[string]any{"deadZoneKind": kind, "ownerId": ownerID, "path": path, "surfaceId": surfaceID}) + } + return result, nil +} + +func admitCoverageDiagnostics(record map[string]any, rowsKey string, countKey string) ([]string, error) { + values, err := admit.PreserveSortedTextArray(record[rowsKey], "requirement coverage output "+rowsKey, true) + if err != nil { + return nil, err + } + if !wireCountEquals(record[countKey], len(values)) { + return nil, fmt.Errorf("requirement coverage output %s does not match %s", countKey, rowsKey) + } + return values, nil +} + +func requireExactDerivedValue(actual any, expected any, context string) error { + actualBytes, err := stablejson.Marshal(actual) + if err != nil { + return fmt.Errorf("%s must be canonical JSON: %w", context, err) + } + expectedBytes, err := stablejson.Marshal(expected) + if err != nil { + return err + } + if !bytes.Equal(actualBytes, expectedBytes) { + return fmt.Errorf("%s is inconsistent with its owner inputs", context) + } + return nil +} + +func requireExactRowDiagnostics(actual []string, expected []string, context string) error { + filtered := make([]string, 0, len(actual)) + for _, diagnostic := range actual { + if isCoverageRowDiagnostic(diagnostic) { + filtered = append(filtered, diagnostic) + } + } + if !slices.Equal(filtered, expected) { + return fmt.Errorf("requirement coverage output %s contain diagnostics inconsistent with retained rows", context) + } + return nil +} + +func admitRetainedUnknownReferenceDiagnostics(record map[string]any, failures []string, registry map[string]projectedTestRecord) error { + knownRequirements := map[string]struct{}{} + knownOwnerInvariants := map[string]struct{}{} + knownCommands := map[string]struct{}{} + knownWitnesses := map[string]struct{}{} + for _, raw := range record["requirementCoverage"].([]any) { + row := raw.(map[string]any) + knownRequirements[stringValue(row["requirementId"])] = struct{}{} + for _, witness := range append(stringArray(row["witnessRefs"]), stringArray(row["witnessSelectors"])...) { + knownWitnesses[witness] = struct{}{} + } + } + for _, raw := range record["ownerInvariantCoverage"].([]any) { + knownOwnerInvariants[stringValue(raw.(map[string]any)["ownerInvariantId"])] = struct{}{} + } + for _, raw := range record["commandCoverage"].([]any) { + knownCommands[stringValue(raw.(map[string]any)["commandId"])] = struct{}{} + } + + entries := make([]testevidenceinventory.Entry, 0, len(registry)) + for _, projected := range registry { + entries = append(entries, projected.entry) + } + expected := sortedUnique(unknownInventoryRefs(entries, knownRequirements, knownOwnerInvariants, knownCommands, knownWitnesses)) + actual := []string{} + for _, diagnostic := range failures { + if isUnknownReferenceDiagnosticForRetainedTest(diagnostic, registry) { + actual = append(actual, diagnostic) + } + } + actual = sortedUnique(actual) + if !slices.Equal(actual, expected) { + return fmt.Errorf("requirement coverage output unknown-reference diagnostics are inconsistent with retained projected tests") + } + return nil +} + +func isUnknownReferenceDiagnosticForRetainedTest(diagnostic string, registry map[string]projectedTestRecord) bool { + for _, kind := range []string{"unknown_requirement_ref", "unknown_owner_invariant_ref", "unknown_command_or_witness_ref"} { + for testID := range registry { + if strings.HasPrefix(diagnostic, kind+":"+testID+":") { + return true + } + } + } + return false +} + +func isCoverageRowDiagnostic(diagnostic string) bool { + if diagnostic == "missing_test_inventory:input" { + return false + } + prefix, _, ok := strings.Cut(diagnostic, ":") + if !ok { + return false + } + if _, ok := requirementCoverageStateDescriptor(prefix); ok { + return true + } + switch prefix { + case "missing_owner_invariant_inventory", + missingCommandCoverageState, + "nonsemantic_command_evidence", + "command_route_only_nonclaim", + "command_proof_route_candidate_only": + return true + default: + return false + } +} diff --git a/internal/command/requirementcoverageview/output_metadata.go b/internal/command/requirementcoverageview/output_metadata.go new file mode 100644 index 0000000..9af8b9e --- /dev/null +++ b/internal/command/requirementcoverageview/output_metadata.go @@ -0,0 +1,118 @@ +package requirementcoverageview + +import ( + "fmt" + + "github.com/research-engineering/agentic-proofkit/internal/command/requirementsourceadmission" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" +) + +func admitCoverageRowMetadata(row map[string]any, rowsKey string, ownerSet map[string]struct{}, context string) error { + switch rowsKey { + case "requirementCoverage": + requirementID, err := requirementsourceadmission.AdmitRequirementID(row["requirementId"], context+" requirementId") + if err != nil { + return err + } + if err := requireCanonicalWireString(row["requirementId"], requirementID, context+" requirementId"); err != nil { + return err + } + claimLevel, err := requirementsourceadmission.AdmitClaimLevel(row["claimLevel"], context+" claimLevel") + if err != nil { + return err + } + if err := requireCanonicalWireString(row["claimLevel"], claimLevel, context+" claimLevel"); err != nil { + return err + } + lifecycleState, err := requirementsourceadmission.AdmitLifecycleState(row["lifecycleState"], context+" lifecycleState") + if err != nil { + return err + } + if err := requireCanonicalWireString(row["lifecycleState"], lifecycleState, context+" lifecycleState"); err != nil { + return err + } + invariant, err := requirementsourceadmission.AdmitInvariantText(row["invariant"], context+" invariant") + if err != nil { + return err + } + if err := requireCanonicalWireString(row["invariant"], invariant, context+" invariant"); err != nil { + return err + } + if err := admitCoverageOwner(row["ownerId"], ownerSet, context+" ownerId"); err != nil { + return err + } + if err := admitCoveragePath(row["specPath"], context+" specPath"); err != nil { + return err + } + _, err = admit.PreserveSortedTextArray(row["nonClaims"], context+" nonClaims", true) + return err + case "ownerInvariantCoverage": + if err := admitCoverageOwner(row["ownerId"], ownerSet, context+" ownerId"); err != nil { + return err + } + if err := admitCoveragePath(row["sourcePath"], context+" sourcePath"); err != nil { + return err + } + if err := admitCoverageText(row["summary"], context+" summary"); err != nil { + return err + } + _, err := admit.PreserveSortedTextArray(row["nonClaims"], context+" nonClaims", true) + return err + case "commandCoverage": + return nil + default: + return fmt.Errorf("%s has unsupported row kind", context) + } +} + +func admitCoverageOwner(raw any, ownerSet map[string]struct{}, context string) error { + ownerID, err := admit.RuleID(raw, context) + if err != nil { + return err + } + if _, ok := ownerSet[ownerID]; !ok { + return fmt.Errorf("%s must be within coverageBasis ownerIds", context) + } + return nil +} + +func admitCoverageText(raw any, context string) error { + value, ok := raw.(string) + if !ok { + return fmt.Errorf("%s must be text", context) + } + canonical, err := admit.NonEmptyText(value, context) + if err != nil { + return err + } + if canonical != value { + return fmt.Errorf("%s must be canonical text", context) + } + return nil +} + +func requireCanonicalWireString(raw any, canonical string, context string) error { + value, ok := raw.(string) + if !ok { + return fmt.Errorf("%s must be text", context) + } + if value != canonical { + return fmt.Errorf("%s must be canonical text", context) + } + return nil +} + +func admitCoveragePath(raw any, context string) error { + value, ok := raw.(string) + if !ok { + return fmt.Errorf("%s must be text", context) + } + canonical, err := admit.SafeRepoRelativePath(value, context) + if err != nil { + return err + } + if canonical != value { + return fmt.Errorf("%s must be canonical", context) + } + return nil +} diff --git a/internal/command/requirementcoverageview/output_proof_projection.go b/internal/command/requirementcoverageview/output_proof_projection.go new file mode 100644 index 0000000..85bcb6c --- /dev/null +++ b/internal/command/requirementcoverageview/output_proof_projection.go @@ -0,0 +1,228 @@ +package requirementcoverageview + +import ( + "fmt" + "slices" + + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/kernel/compactproofcontract" + "github.com/research-engineering/agentic-proofkit/internal/kernel/proofvocab" +) + +type admittedCoverageScenario struct { + commandIDs []string + environmentClasses []string + scenarioID string + verifyCommands []string + witnessID string + witnessSelectors []string +} + +func admitRequirementProofProjection(row map[string]any, proofMode string, entries []testevidenceinventory.Entry, context string) error { + proofState, ok := row["proofState"].(string) + if !ok { + return fmt.Errorf("%s proofState must be a string", context) + } + if proofState != "" { + if _, err := admit.Enum(proofState, proofvocab.RequirementProofStateSet(), context+" proofState"); err != nil { + return err + } + } + scenarios, err := admitCoverageScenarios(row["scenarios"], proofMode, context+" scenarios") + if err != nil { + return err + } + if !wireCountEquals(row["scenarioCount"], len(scenarios)) { + return fmt.Errorf("%s scenarioCount does not match scenarios", context) + } + if (proofState == "witness_backed") != (len(scenarios) > 0) { + return fmt.Errorf("%s proofState and scenarios are inconsistent", context) + } + if proofMode == "compact" && proofState != "" && proofState != "witness_backed" { + return fmt.Errorf("%s compact proofState must be witness_backed", context) + } + + commandIDs, err := admitProjectedRuleIDs(row["commandIds"], context+" commandIds") + if err != nil { + return err + } + environmentClasses, err := admitProjectedRuleIDs(row["environmentClasses"], context+" environmentClasses") + if err != nil { + return err + } + verifyCommands, err := admitProjectedCommands(row["verifyCommands"], context+" verifyCommands") + if err != nil { + return err + } + witnessRefs, err := admitProjectedRuleIDs(row["witnessRefs"], context+" witnessRefs") + if err != nil { + return err + } + witnessSelectors, err := admitProjectedWitnessSelectors(row["witnessSelectors"], context+" witnessSelectors") + if err != nil { + return err + } + + expectedEnvironments := []string{} + for _, scenario := range scenarios { + expectedEnvironments = append(expectedEnvironments, scenario.environmentClasses...) + } + if !slices.Equal(environmentClasses, sortedUnique(expectedEnvironments)) { + return fmt.Errorf("%s environmentClasses are not derived from scenarios", context) + } + if proofMode == "compact" { + if len(witnessRefs) != 0 { + return fmt.Errorf("%s compact witnessRefs must be empty", context) + } + expectedCommands := []string{} + if proofState == "witness_backed" { + expectedCommands = entryCommandRefs(entries) + } + if !slices.Equal(commandIDs, expectedCommands) { + return fmt.Errorf("%s compact commandIds are not derived from projected tests", context) + } + expectedVerifyCommands := []string{} + expectedWitnessSelectors := []string{} + for _, scenario := range scenarios { + expectedVerifyCommands = append(expectedVerifyCommands, scenario.verifyCommands...) + expectedWitnessSelectors = append(expectedWitnessSelectors, scenario.witnessSelectors...) + } + if !slices.Equal(verifyCommands, sortedUnique(expectedVerifyCommands)) { + return fmt.Errorf("%s compact verifyCommands are not derived from scenarios", context) + } + if !slices.Equal(witnessSelectors, sortedUnique(expectedWitnessSelectors)) { + return fmt.Errorf("%s compact witnessSelectors are not derived from scenarios", context) + } + return nil + } + + expectedCommands := []string{} + expectedWitnesses := []string{} + for _, scenario := range scenarios { + expectedCommands = append(expectedCommands, scenario.commandIDs...) + expectedWitnesses = append(expectedWitnesses, scenario.witnessID) + } + if !slices.Equal(commandIDs, sortedUnique(expectedCommands)) { + return fmt.Errorf("%s commandIds are not derived from scenarios", context) + } + if !slices.Equal(witnessRefs, sortedUnique(expectedWitnesses)) { + return fmt.Errorf("%s witnessRefs are not derived from scenarios", context) + } + if len(verifyCommands) != 0 || len(witnessSelectors) != 0 { + return fmt.Errorf("%s structured proof must not project compact verifyCommands or witnessSelectors", context) + } + return nil +} + +func admitCoverageScenarios(raw any, proofMode string, context string) ([]admittedCoverageScenario, error) { + values, ok := raw.([]any) + if !ok { + return nil, fmt.Errorf("%s must be an array", context) + } + result := make([]admittedCoverageScenario, 0, len(values)) + previousID := "" + for index, rawScenario := range values { + record, ok := rawScenario.(map[string]any) + if !ok { + return nil, fmt.Errorf("%s[%d] must be an object", context, index) + } + scenarioContext := fmt.Sprintf("%s[%d]", context, index) + commandIDs, err := admitProjectedRuleIDs(record["commandIds"], scenarioContext+" commandIds") + if err != nil { + return nil, err + } + environmentClasses, err := admitProjectedRuleIDs(record["environmentClasses"], scenarioContext+" environmentClasses") + if err != nil { + return nil, err + } + verifyCommands, err := admitProjectedCommands(record["verifyCommands"], scenarioContext+" verifyCommands") + if err != nil { + return nil, err + } + witnessSelectors, err := admitProjectedWitnessSelectors(record["witnessSelectors"], scenarioContext+" witnessSelectors") + if err != nil { + return nil, err + } + scenarioID, err := admitCoverageScenarioID(record["scenarioId"], proofMode, scenarioContext+" scenarioId") + if err != nil { + return nil, err + } + if previousID != "" && previousID >= scenarioID { + return nil, fmt.Errorf("%s must be sorted and unique by scenarioId", context) + } + previousID = scenarioID + + witnessID := stringValue(record["witnessId"]) + witnessKind := stringValue(record["witnessKind"]) + witnessPath := stringValue(record["witnessPath"]) + if proofMode == "compact" { + if len(commandIDs) != 0 || witnessID != "" || witnessKind != "" || witnessPath != "" { + return nil, fmt.Errorf("%s compact scenario must not synthesize command or witness identity", scenarioContext) + } + if len(verifyCommands) == 0 || len(witnessSelectors) == 0 { + return nil, fmt.Errorf("%s compact scenario must retain verify commands and witness selectors", scenarioContext) + } + } else { + if len(verifyCommands) != 0 || len(witnessSelectors) != 0 { + return nil, fmt.Errorf("%s structured scenario must not project compact routes", scenarioContext) + } + if _, err := admit.RuleID(witnessID, scenarioContext+" witnessId"); err != nil { + return nil, err + } + if _, err := admit.RuleID(witnessKind, scenarioContext+" witnessKind"); err != nil { + return nil, err + } + if _, err := admit.SafeRepoRelativePath(witnessPath, scenarioContext+" witnessPath"); err != nil { + return nil, err + } + } + result = append(result, admittedCoverageScenario{ + commandIDs: commandIDs, environmentClasses: environmentClasses, + scenarioID: scenarioID, verifyCommands: verifyCommands, witnessID: witnessID, + witnessSelectors: witnessSelectors, + }) + } + return result, nil +} + +func admitCoverageScenarioID(raw any, proofMode string, context string) (string, error) { + if proofMode == "structured" { + return admit.RuleID(raw, context) + } + value, err := admit.NonEmptyText(raw, context) + if err != nil { + return "", err + } + if raw != value { + return "", fmt.Errorf("%s must be canonical non-empty text", context) + } + scenarioID, _, err := compactproofcontract.AdmitScenarioID(value, context) + return scenarioID, err +} + +func admitProjectedCommands(raw any, context string) ([]string, error) { + values, err := admit.PreserveSortedTextArray(raw, context, true) + if err != nil { + return nil, err + } + for index, value := range values { + if _, err := admit.DisplayOnlyCommandText(value, fmt.Sprintf("%s[%d]", context, index)); err != nil { + return nil, err + } + } + return values, nil +} + +func admitProjectedWitnessSelectors(raw any, context string) ([]string, error) { + values, err := admit.PreserveSortedTextArray(raw, context, true) + if err != nil { + return nil, err + } + for index, value := range values { + if _, err := compactproofcontract.AdmitWitnessSelector(value, fmt.Sprintf("%s[%d]", context, index)); err != nil { + return nil, err + } + } + return values, nil +} diff --git a/internal/command/requirementcoverageview/output_semantics.go b/internal/command/requirementcoverageview/output_semantics.go new file mode 100644 index 0000000..07bec99 --- /dev/null +++ b/internal/command/requirementcoverageview/output_semantics.go @@ -0,0 +1,331 @@ +package requirementcoverageview + +import ( + "fmt" + "slices" + + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/kernel/proofvocab" +) + +func validateCoverageOutputSemantics(record map[string]any) error { + proofMode, err := admit.Enum(record["proofMode"], map[string]struct{}{"compact": {}, "structured": {}}, "requirement coverage output proofMode") + if err != nil { + return err + } + if err := admitCoverageOutputIdentity(record, proofMode); err != nil { + return err + } + completenessDeclaration, err := admit.Enum(record["completenessDeclaration"], map[string]struct{}{ + "full_repository": {}, "selected_owner_surfaces": {}, "selected_paths_advisory": {}, + }, "requirement coverage output completenessDeclaration") + if err != nil { + return err + } + failures, err := admitCoverageDiagnostics(record, "failures", "failureCount") + if err != nil { + return err + } + warnings, err := admitCoverageDiagnostics(record, "warnings", "warningCount") + if err != nil { + return err + } + expectedState := "passed" + if len(failures) > 0 { + expectedState = "failed" + } + if record["state"] != expectedState { + return fmt.Errorf("requirement coverage output state is inconsistent with failures") + } + if err := requireExactDerivedValue(record["failureClassifications"], mapsToAny(diagnosticClassifications(failures, "failure")), "failureClassifications"); err != nil { + return err + } + if err := requireExactDerivedValue(record["warningClassifications"], mapsToAny(diagnosticClassifications(warnings, "warning")), "warningClassifications"); err != nil { + return err + } + if err := requireExactDerivedValue(record["guidanceSummary"], guidanceSummary(expectedState, failures, warnings), "guidanceSummary"); err != nil { + return err + } + deadZones, err := admitCoverageDeadZones(record["deadZones"]) + if err != nil { + return err + } + deadZoneFailures := []string{} + deadZoneWarnings := []string{} + if completenessDeclaration == "selected_paths_advisory" { + deadZoneWarnings = deadZoneDiagnostics("dead_zone_advisory", deadZones) + } else { + deadZoneFailures = deadZoneDiagnostics("dead_zone", deadZones) + } + if err := requireExactDiagnosticClass(failures, deadZoneFailures, "dead-zone failures", "dead_zone:", "dead_zone_advisory:"); err != nil { + return err + } + if err := requireExactDiagnosticClass(warnings, deadZoneWarnings, "dead-zone warnings", "dead_zone:", "dead_zone_advisory:"); err != nil { + return err + } + inventoryMissing := record["testInventoryId"] == nil + if slices.Contains(warnings, "missing_test_inventory:input") != inventoryMissing { + return fmt.Errorf("requirement coverage output missing_test_inventory:input warning is inconsistent with testInventoryId") + } + projectedTests, err := admitCoverageProjectedTestRegistry(record) + if err != nil { + return err + } + projectedEntries := projectedRegistryEntries(projectedTests) + coverageBasis, err := admitCoverageBasis(record["coverageBasis"], inventoryMissing, completenessDeclaration, projectedEntries) + if err != nil { + return err + } + if err := requireExactDiagnosticClass( + failures, + coverageBasis.ownerScopeFailures, + "owner-scope failures", + "full_repository_source_requirement_outside_owner_scope:", + "inventory_entry_owner_outside_scope:", + ); err != nil { + return err + } + inventoryUnavailable := inventoryMissing + expectedInventoryFailures := []string{} + if inventoryUnavailable { + if len(projectedEntries) != 0 { + return fmt.Errorf("requirement coverage output without testInventoryId must not retain projected tests") + } + } else { + inventoryFailures, _, err := testevidenceinventory.ClassifyProjectedEntries(projectedEntries) + if err != nil { + return err + } + if len(inventoryFailures) > 0 { + inventoryUnavailable = true + expectedInventoryFailures = []string{"test_inventory_failed:" + record["testInventoryId"].(string)} + } + } + if err := requireExactDiagnosticClass(failures, expectedInventoryFailures, "test-inventory failures", "test_inventory_failed:"); err != nil { + return err + } + if err := admitProjectedParentClosure(record, projectedTests); err != nil { + return err + } + requiredFailures := []string{} + requiredWarnings := []string{} + for _, descriptor := range coverageRowDescriptors { + rowsKey := descriptor.rowsKey + rows, ok := record[rowsKey].([]any) + if !ok { + return fmt.Errorf("requirement coverage output %s must be an array", rowsKey) + } + for index, raw := range rows { + row, ok := raw.(map[string]any) + if !ok { + return fmt.Errorf("requirement coverage output %s[%d] must be an object", rowsKey, index) + } + if err := admitCoverageRowSemantics(row, rowsKey, index, proofMode, inventoryUnavailable, completenessDeclaration, coverageBasis.ownerSet); err != nil { + return err + } + rowFailures, rowWarnings := expectedCoverageRowDiagnostics(row, rowsKey, completenessDeclaration) + requiredFailures = append(requiredFailures, rowFailures...) + requiredWarnings = append(requiredWarnings, rowWarnings...) + } + } + if err := requireExactRowDiagnostics(failures, sortedUnique(requiredFailures), "failures"); err != nil { + return err + } + if err := requireExactRowDiagnostics(warnings, sortedUnique(requiredWarnings), "warnings"); err != nil { + return err + } + if err := admitRetainedUnknownReferenceDiagnostics(record, failures, projectedTests); err != nil { + return err + } + return nil +} + +func admitCoverageOutputIdentity(record map[string]any, proofMode string) error { + bindingID, bindingOK := record["bindingId"].(string) + contractID, contractOK := record["contractId"].(string) + if !bindingOK || !contractOK { + return fmt.Errorf("requirement coverage output bindingId and contractId must be strings") + } + if proofMode == "structured" { + if _, err := admit.RuleID(bindingID, "requirement coverage output bindingId"); err != nil { + return err + } + if contractID != "" { + return fmt.Errorf("requirement coverage structured output contractId must be empty") + } + } else { + if bindingID != "" { + return fmt.Errorf("requirement coverage compact output bindingId must be empty") + } + if _, err := admit.RuleID(contractID, "requirement coverage output contractId"); err != nil { + return err + } + } + registryID, ok := record["ownerInvariantRegistryId"].(string) + if !ok { + return fmt.Errorf("requirement coverage output ownerInvariantRegistryId must be a string") + } + if registryID != "" { + if _, err := admit.RuleID(registryID, "requirement coverage output ownerInvariantRegistryId"); err != nil { + return err + } + } + if record["testInventoryId"] != nil { + if _, err := admit.RuleID(record["testInventoryId"], "requirement coverage output testInventoryId"); err != nil { + return err + } + } + return nil +} + +func admitCoverageRowSemantics(row map[string]any, rowsKey string, index int, proofMode string, inventoryUnavailable bool, completenessDeclaration string, ownerSet map[string]struct{}) error { + context := fmt.Sprintf("requirement coverage output %s[%d]", rowsKey, index) + if err := admitCoverageRowMetadata(row, rowsKey, ownerSet, context); err != nil { + return err + } + entries, err := admitProjectedTestSemantics(row["tests"], context+" tests") + if err != nil { + return err + } + if err := admitExactProjectedTestIDs(row["testIds"], entries, context+" testIds"); err != nil { + return err + } + if err := requireProjectedParentRef(entries, row, rowsKey, context); err != nil { + return err + } + switch rowsKey { + case "requirementCoverage": + if err := admitCoverageStateClass(row, requirementCoverageStateDescriptor, context); err != nil { + return err + } + if err := admitRequirementProofProjection(row, proofMode, entries, context); err != nil { + return err + } + case "ownerInvariantCoverage": + if err := admitCoverageStateClass(row, ownerInvariantCoverageStateDescriptor, context); err != nil { + return err + } + case "commandCoverage": + state, ok := row["coverageState"].(string) + if !ok || !commandCoverageStateAllowed(state) { + return fmt.Errorf("%s coverageState must be one of the admitted command coverage states", context) + } + expectedState := commandState(entries) + if state != expectedState { + return fmt.Errorf("%s coverageState is not derived from projected tests", context) + } + expectedFailures, _ := commandCoverageDiagnostics(stringValue(row["commandId"]), state, completenessDeclaration) + if err := requireExactDerivedValue(row["failures"], admit.StringSliceToAny(expectedFailures), context+" failures"); err != nil { + return err + } + return nil + default: + return fmt.Errorf("%s has unsupported row kind", context) + } + state := stringValue(row["coverageState"]) + class := stringValue(row["evidenceClass"]) + switch rowsKey { + case "requirementCoverage": + expectedState, expectedClass, err := expectedRequirementCoverageState(row, entries, inventoryUnavailable, context) + if err != nil { + return err + } + if state != expectedState || class != expectedClass { + return fmt.Errorf("%s coverageState and evidenceClass are not derived from projected tests and lifecycle", context) + } + expectedFailures := expectedRequirementFailures(row, expectedState, completenessDeclaration) + if err := requireExactDerivedValue(row["failures"], admit.StringSliceToAny(expectedFailures), context+" failures"); err != nil { + return err + } + case "ownerInvariantCoverage": + expectedState, expectedClass := "missing_test_inventory", "" + if !inventoryUnavailable { + expectedState, expectedClass = strongestMappingState(entries) + } + if state != expectedState || class != expectedClass { + return fmt.Errorf("%s coverageState and evidenceClass are not derived from projected tests", context) + } + expectedWarnings := []string{} + if ownerInvariantCoverageStateWarns(expectedState) { + expectedWarnings = append(expectedWarnings, "missing_owner_invariant_inventory:"+stringValue(row["ownerInvariantId"])) + } + if err := requireExactDerivedValue(row["warnings"], admit.StringSliceToAny(expectedWarnings), context+" warnings"); err != nil { + return err + } + } + return nil +} + +func admitCoverageStateClass(row map[string]any, lookup func(string) (coverageStateDescriptor, bool), context string) error { + state, ok := row["coverageState"].(string) + if !ok { + return fmt.Errorf("%s coverageState must be a string", context) + } + descriptor, ok := lookup(state) + if !ok { + return fmt.Errorf("%s coverageState is unsupported", context) + } + class, ok := row["evidenceClass"].(string) + if !ok { + return fmt.Errorf("%s evidenceClass must be a string", context) + } + if class != descriptor.evidenceClass { + return fmt.Errorf("%s coverageState and evidenceClass are inconsistent", context) + } + if class != "" { + if _, err := testevidenceinventory.AdmitEvidenceClass(class, context+" evidenceClass"); err != nil { + return err + } + } + return nil +} + +func expectedRequirementCoverageState(row map[string]any, entries []testevidenceinventory.Entry, inventoryUnavailable bool, context string) (string, string, error) { + claimLevel, ok := row["claimLevel"].(string) + if !ok || (claimLevel != "advisory" && claimLevel != "blocking" && claimLevel != "deferred") { + return "", "", fmt.Errorf("%s claimLevel is invalid", context) + } + lifecycleState, ok := row["lifecycleState"].(string) + if !ok || (lifecycleState != "active" && lifecycleState != "deprecated" && lifecycleState != "removed" && lifecycleState != "superseded") { + return "", "", fmt.Errorf("%s lifecycleState is invalid", context) + } + proofState, ok := row["proofState"].(string) + if !ok { + return "", "", fmt.Errorf("%s proofState must be a string", context) + } + if proofState != "" { + if _, err := admit.Enum(proofState, proofvocab.RequirementProofStateSet(), context+" proofState"); err != nil { + return "", "", err + } + } + if claimLevel == "deferred" { + return "deferred_with_owner", "", nil + } + if lifecycleState == "removed" { + return "not_applicable", "", nil + } + if proofState != "witness_backed" { + return "missing_proof_binding_route", "", nil + } + if inventoryUnavailable { + return "missing_test_inventory", "", nil + } + state, class := strongestMappingState(entries) + return state, class, nil +} + +func expectedRequirementFailures(row map[string]any, state string, completenessDeclaration string) []string { + requirementID := stringValue(row["requirementId"]) + claimLevel := stringValue(row["claimLevel"]) + lifecycleState := stringValue(row["lifecycleState"]) + proofState := stringValue(row["proofState"]) + failures := []string{} + if proofState != "witness_backed" && claimLevel == "blocking" && lifecycleState == "active" { + failures = append(failures, "missing_proof_binding_route:"+requirementID) + } + if requirementMappingBlocks(claimLevel, lifecycleState, state, completenessDeclaration) { + failures = append(failures, state+":"+requirementID) + } + return sortedUnique(failures) +} diff --git a/internal/command/requirementcoverageview/output_test_projection.go b/internal/command/requirementcoverageview/output_test_projection.go new file mode 100644 index 0000000..f163711 --- /dev/null +++ b/internal/command/requirementcoverageview/output_test_projection.go @@ -0,0 +1,400 @@ +package requirementcoverageview + +import ( + "bytes" + "fmt" + "slices" + + "github.com/research-engineering/agentic-proofkit/internal/command/testevidenceinventory" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/kernel/stablejson" +) + +var projectedTestKeys = []string{ + "commandRefs", "dominanceGroup", "evidenceClass", "expectedPublicOutcome", "falsifierId", + "negativeCaseId", "nonClaims", "oracleId", "oracleKind", "oracleSummary", "ownerId", + "ownerInvariantRefs", "qualityFindings", "requirementRefs", "selector", "sourcePath", + "supersedes", "supersessionDeclarationRef", "testId", "witnessRefs", "wrongImplementationClassId", +} + +var projectedQualityFindingKeys = []string{ + "class", "evidenceRefs", "findingId", "nonClaims", "ownerReviewState", "severity", +} + +func admitProjectedTestSemantics(raw any, context string) ([]testevidenceinventory.Entry, error) { + tests, ok := raw.([]any) + if !ok { + return nil, fmt.Errorf("%s must be an array", context) + } + entries := make([]testevidenceinventory.Entry, 0, len(tests)) + previousID := "" + for testIndex, rawTest := range tests { + test, ok := rawTest.(map[string]any) + if !ok { + return nil, fmt.Errorf("%s[%d] must be an object", context, testIndex) + } + testContext := fmt.Sprintf("%s[%d]", context, testIndex) + if err := admit.KnownKeys(test, projectedTestKeys, testContext); err != nil { + return nil, err + } + testID, err := admit.RuleID(test["testId"], testContext+" testId") + if err != nil { + return nil, err + } + if previousID != "" && previousID >= testID { + return nil, fmt.Errorf("%s must be sorted and unique by testId", context) + } + previousID = testID + evidenceClass, err := testevidenceinventory.AdmitEvidenceClass(test["evidenceClass"], testContext+" evidenceClass") + if err != nil { + return nil, err + } + ownerID, err := admit.RuleID(test["ownerId"], testContext+" ownerId") + if err != nil { + return nil, err + } + commandRefs, err := admitProjectedRuleIDs(test["commandRefs"], testContext+" commandRefs") + if err != nil { + return nil, err + } + ownerInvariantRefs, err := admitProjectedRuleIDs(test["ownerInvariantRefs"], testContext+" ownerInvariantRefs") + if err != nil { + return nil, err + } + requirementRefs, err := admitProjectedRuleIDs(test["requirementRefs"], testContext+" requirementRefs") + if err != nil { + return nil, err + } + witnessRefs, err := admitProjectedRuleIDs(test["witnessRefs"], testContext+" witnessRefs") + if err != nil { + return nil, err + } + selector, err := admit.DisplayOnlyCommandText(test["selector"], testContext+" selector") + if err != nil { + return nil, err + } + if err := requireCanonicalWireString(test["selector"], selector, testContext+" selector"); err != nil { + return nil, err + } + sourceText, err := admit.NonEmptyText(test["sourcePath"], testContext+" sourcePath") + if err != nil { + return nil, err + } + sourcePath, err := admit.SafeRepoRelativePath(sourceText, testContext+" sourcePath") + if err != nil { + return nil, err + } + if sourcePath != sourceText { + return nil, fmt.Errorf("%s sourcePath must be canonical", testContext) + } + if err := admit.StructuredSelectorSourcePath(selector, sourcePath, testContext+" selector"); err != nil { + return nil, err + } + nonClaims, err := admit.PreserveSortedTextArray(test["nonClaims"], testContext+" nonClaims", true) + if err != nil { + return nil, err + } + expectedPublicOutcome, err := admitProjectedOptionalText(test["expectedPublicOutcome"], testContext+" expectedPublicOutcome") + if err != nil { + return nil, err + } + oracleSummary, err := admitProjectedOptionalText(test["oracleSummary"], testContext+" oracleSummary") + if err != nil { + return nil, err + } + oracleKind, err := admitProjectedOptionalRuleID(test["oracleKind"], testContext+" oracleKind") + if err != nil { + return nil, err + } + oracleID, err := admitProjectedOptionalRuleID(test["oracleId"], testContext+" oracleId") + if err != nil { + return nil, err + } + falsifierID, err := admitProjectedOptionalRuleID(test["falsifierId"], testContext+" falsifierId") + if err != nil { + return nil, err + } + dominanceGroup, err := admitProjectedOptionalRuleID(test["dominanceGroup"], testContext+" dominanceGroup") + if err != nil { + return nil, err + } + negativeCaseID, err := admitProjectedOptionalRuleID(test["negativeCaseId"], testContext+" negativeCaseId") + if err != nil { + return nil, err + } + wrongImplementationClassID, err := admitProjectedOptionalRuleID(test["wrongImplementationClassId"], testContext+" wrongImplementationClassId") + if err != nil { + return nil, err + } + supersedes, err := admitProjectedRuleIDs(test["supersedes"], testContext+" supersedes") + if err != nil { + return nil, err + } + supersessionDeclarationRef, err := admitProjectedOptionalRuleID(test["supersessionDeclarationRef"], testContext+" supersessionDeclarationRef") + if err != nil { + return nil, err + } + qualityFindings, err := admitProjectedQualityFindings(test["qualityFindings"], testContext+" qualityFindings") + if err != nil { + return nil, err + } + entry := testevidenceinventory.Entry{ + CommandRefs: commandRefs, EvidenceClass: evidenceClass, OwnerInvariantRefs: ownerInvariantRefs, + NonClaims: nonClaims, OwnerID: ownerID, QualityFindings: qualityFindings, RequirementRefs: requirementRefs, + Selector: selector, SourcePath: sourcePath, TestID: testID, WitnessRefs: witnessRefs, + } + if oracleID != "" || oracleKind != "" || oracleSummary != "" || expectedPublicOutcome != "" { + if oracleID == "" || oracleKind == "" { + return nil, fmt.Errorf("%s projected oracle must retain oracleId and oracleKind", testContext) + } + entry.Oracle = &testevidenceinventory.Oracle{ + AssertionSummary: oracleSummary, ExpectedPublicOutcome: expectedPublicOutcome, + OracleID: oracleID, OracleKind: oracleKind, + } + } + if falsifierID != "" || dominanceGroup != "" || negativeCaseID != "" || wrongImplementationClassID != "" || len(supersedes) > 0 || supersessionDeclarationRef != "" { + if falsifierID == "" || dominanceGroup == "" || negativeCaseID == "" || wrongImplementationClassID == "" { + return nil, fmt.Errorf("%s projected falsifier must retain all identity fields", testContext) + } + entry.Falsifier = &testevidenceinventory.Falsifier{ + DominanceGroup: dominanceGroup, FalsifierID: falsifierID, NegativeCaseID: negativeCaseID, + Supersedes: supersedes, SupersessionDeclarationRef: supersessionDeclarationRef, + WrongImplementationClassID: wrongImplementationClassID, + } + } + entries = append(entries, entry) + } + return entries, nil +} + +func admitProjectedOptionalText(raw any, context string) (string, error) { + value, ok := raw.(string) + if !ok { + return "", fmt.Errorf("%s must be text", context) + } + if value == "" { + return "", nil + } + canonical, err := admit.NonEmptyText(value, context) + if err != nil { + return "", err + } + if canonical != value { + return "", fmt.Errorf("%s must be canonical text", context) + } + return value, nil +} + +func admitProjectedOptionalRuleID(raw any, context string) (string, error) { + value, ok := raw.(string) + if !ok { + return "", fmt.Errorf("%s must be text", context) + } + if value == "" { + return "", nil + } + return admit.RuleID(value, context) +} + +func admitProjectedQualityFindings(raw any, context string) ([]testevidenceinventory.QualityFinding, error) { + values, ok := raw.([]any) + if !ok { + return nil, fmt.Errorf("%s must be an array", context) + } + result := make([]testevidenceinventory.QualityFinding, 0, len(values)) + previousID := "" + for index, rawFinding := range values { + finding, ok := rawFinding.(map[string]any) + if !ok { + return nil, fmt.Errorf("%s[%d] must be an object", context, index) + } + findingContext := fmt.Sprintf("%s[%d]", context, index) + if err := admit.KnownKeys(finding, projectedQualityFindingKeys, findingContext); err != nil { + return nil, err + } + findingID, err := admit.RuleID(finding["findingId"], findingContext+" findingId") + if err != nil { + return nil, err + } + if previousID != "" && previousID >= findingID { + return nil, fmt.Errorf("%s must be sorted and unique by findingId", context) + } + previousID = findingID + class, err := testevidenceinventory.AdmitQualityFindingClass(finding["class"], findingContext+" class") + if err != nil { + return nil, err + } + severity, err := testevidenceinventory.AdmitQualityFindingSeverity(finding["severity"], findingContext+" severity") + if err != nil { + return nil, err + } + reviewState, err := testevidenceinventory.AdmitQualityFindingReviewState(finding["ownerReviewState"], findingContext+" ownerReviewState") + if err != nil { + return nil, err + } + evidenceRefs, err := admitProjectedRuleIDs(finding["evidenceRefs"], findingContext+" evidenceRefs") + if err != nil { + return nil, err + } + if len(evidenceRefs) == 0 { + return nil, fmt.Errorf("%s evidenceRefs must not be empty", findingContext) + } + nonClaims, err := admit.PreserveSortedTextArray(finding["nonClaims"], findingContext+" nonClaims", false) + if err != nil { + return nil, err + } + result = append(result, testevidenceinventory.QualityFinding{ + Class: class, EvidenceRefs: evidenceRefs, FindingID: findingID, + NonClaims: nonClaims, OwnerReviewState: reviewState, Severity: severity, + }) + } + return result, nil +} + +func admitProjectedRuleIDs(raw any, context string) ([]string, error) { + values, err := admit.PreserveSortedTextArray(raw, context, true) + if err != nil { + return nil, err + } + for index, value := range values { + if _, err := admit.RuleID(value, fmt.Sprintf("%s[%d]", context, index)); err != nil { + return nil, err + } + } + return values, nil +} + +func requireProjectedParentRef(entries []testevidenceinventory.Entry, row map[string]any, rowsKey string, context string) error { + var parentID string + var selectRefs func(testevidenceinventory.Entry) []string + switch rowsKey { + case "requirementCoverage": + parentID = stringValue(row["requirementId"]) + selectRefs = func(entry testevidenceinventory.Entry) []string { return entry.RequirementRefs } + case "ownerInvariantCoverage": + parentID = stringValue(row["ownerInvariantId"]) + selectRefs = func(entry testevidenceinventory.Entry) []string { return entry.OwnerInvariantRefs } + case "commandCoverage": + parentID = stringValue(row["commandId"]) + selectRefs = func(entry testevidenceinventory.Entry) []string { return entry.CommandRefs } + default: + return nil + } + for _, entry := range entries { + if !slices.Contains(selectRefs(entry), parentID) { + return fmt.Errorf("%s test %s does not reference its projected parent", context, entry.TestID) + } + } + return nil +} + +func admitExactProjectedTestIDs(raw any, entries []testevidenceinventory.Entry, context string) error { + actual, err := admit.PreserveSortedTextArray(raw, context, true) + if err != nil { + return err + } + expected := entryIDs(entries) + if !slices.Equal(actual, expected) { + return fmt.Errorf("%s must equal projected test identities", context) + } + return nil +} + +type projectedTestRecord struct { + entry testevidenceinventory.Entry + projection []byte +} + +func admitCoverageProjectedTestRegistry(record map[string]any) (map[string]projectedTestRecord, error) { + registry := map[string]projectedTestRecord{} + for _, descriptor := range coverageRowDescriptors { + rowsKey := descriptor.rowsKey + for rowIndex, rawRow := range record[rowsKey].([]any) { + row := rawRow.(map[string]any) + context := fmt.Sprintf("requirement coverage output %s[%d] tests", rowsKey, rowIndex) + entries, err := admitProjectedTestSemantics(row["tests"], context) + if err != nil { + return nil, err + } + tests := row["tests"].([]any) + for testIndex, entry := range entries { + projection, err := stablejson.Marshal(tests[testIndex]) + if err != nil { + return nil, fmt.Errorf("%s[%d] must be canonical JSON: %w", context, testIndex, err) + } + if previous, ok := registry[entry.TestID]; ok { + if !bytes.Equal(previous.projection, projection) { + return nil, fmt.Errorf("requirement coverage output test %s has inconsistent projections", entry.TestID) + } + continue + } + registry[entry.TestID] = projectedTestRecord{entry: entry, projection: projection} + } + } + } + unmappedEntries, err := admitProjectedTestSemantics(record["unmappedTests"], "requirement coverage output unmappedTests") + if err != nil { + return nil, err + } + for index, entry := range unmappedEntries { + if _, exists := registry[entry.TestID]; exists { + return nil, fmt.Errorf("requirement coverage output unmapped test %s is already projected under a retained parent", entry.TestID) + } + projection, err := stablejson.Marshal(record["unmappedTests"].([]any)[index]) + if err != nil { + return nil, fmt.Errorf("requirement coverage output unmappedTests[%d] must be canonical JSON: %w", index, err) + } + registry[entry.TestID] = projectedTestRecord{entry: entry, projection: projection} + } + return registry, nil +} + +func projectedRegistryEntries(registry map[string]projectedTestRecord) []testevidenceinventory.Entry { + ids := make([]string, 0, len(registry)) + for testID := range registry { + ids = append(ids, testID) + } + slices.Sort(ids) + entries := make([]testevidenceinventory.Entry, 0, len(ids)) + for _, testID := range ids { + entries = append(entries, registry[testID].entry) + } + return entries +} + +func admitProjectedParentClosure(record map[string]any, registry map[string]projectedTestRecord) error { + rowTests := map[string]map[string]map[string]struct{}{} + for _, descriptor := range coverageRowDescriptors { + byParent := map[string]map[string]struct{}{} + for _, rawRow := range record[descriptor.rowsKey].([]any) { + row := rawRow.(map[string]any) + testIDs := map[string]struct{}{} + for _, rawTest := range row["tests"].([]any) { + testIDs[rawTest.(map[string]any)["testId"].(string)] = struct{}{} + } + byParent[row[descriptor.idKey].(string)] = testIDs + } + rowTests[descriptor.rowsKey] = byParent + } + for testID, projected := range registry { + for _, relation := range []struct { + rowsKey string + refs []string + }{ + {rowsKey: "requirementCoverage", refs: projected.entry.RequirementRefs}, + {rowsKey: "ownerInvariantCoverage", refs: projected.entry.OwnerInvariantRefs}, + {rowsKey: "commandCoverage", refs: projected.entry.CommandRefs}, + } { + for _, parentID := range relation.refs { + testIDs, parentRetained := rowTests[relation.rowsKey][parentID] + if !parentRetained { + continue + } + if _, projectedUnderParent := testIDs[testID]; !projectedUnderParent { + return fmt.Errorf("requirement coverage output test %s is missing from retained parent %s", testID, parentID) + } + } + } + } + return nil +} diff --git a/internal/command/requirementcoverageview/projection.go b/internal/command/requirementcoverageview/projection.go index 6468596..7109d4b 100644 --- a/internal/command/requirementcoverageview/projection.go +++ b/internal/command/requirementcoverageview/projection.go @@ -19,9 +19,11 @@ func scenariosToAny(scenarios []scenario) []any { "commandIds": admit.StringSliceToAny(item.CommandIDs), "environmentClasses": admit.StringSliceToAny(item.EnvironmentClasses), "scenarioId": item.ScenarioID, + "verifyCommands": admit.StringSliceToAny(item.VerifyCommands), "witnessId": item.WitnessID, "witnessKind": item.WitnessKind, "witnessPath": item.WitnessPath, + "witnessSelectors": admit.StringSliceToAny(item.WitnessSelectors), }) } return result @@ -32,28 +34,46 @@ func testEntriesToAny(entries []testevidenceinventory.Entry) []any { expectedPublicOutcome := "" oracleSummary := "" oracleKind := "" + oracleID := "" if entry.Oracle != nil { expectedPublicOutcome = entry.Oracle.ExpectedPublicOutcome oracleSummary = entry.Oracle.AssertionSummary oracleKind = entry.Oracle.OracleKind + oracleID = entry.Oracle.OracleID } + dominanceGroup := "" + falsifierID := "" wrongImplementation := "" negativeCaseID := "" + supersedes := []string{} + supersessionDeclarationRef := "" if entry.Falsifier != nil { + dominanceGroup = entry.Falsifier.DominanceGroup + falsifierID = entry.Falsifier.FalsifierID wrongImplementation = entry.Falsifier.WrongImplementationClassID negativeCaseID = entry.Falsifier.NegativeCaseID + supersedes = entry.Falsifier.Supersedes + supersessionDeclarationRef = entry.Falsifier.SupersessionDeclarationRef } result = append(result, map[string]any{ "commandRefs": admit.StringSliceToAny(entry.CommandRefs), + "dominanceGroup": dominanceGroup, "evidenceClass": entry.EvidenceClass, "expectedPublicOutcome": expectedPublicOutcome, + "falsifierId": falsifierID, "negativeCaseId": negativeCaseID, "nonClaims": admit.StringSliceToAny(entry.NonClaims), + "oracleId": oracleID, "oracleKind": oracleKind, "oracleSummary": oracleSummary, + "ownerId": entry.OwnerID, + "ownerInvariantRefs": admit.StringSliceToAny(entry.OwnerInvariantRefs), "qualityFindings": qualityFindingsToAny(entry.QualityFindings), + "requirementRefs": admit.StringSliceToAny(entry.RequirementRefs), "selector": entry.Selector, "sourcePath": entry.SourcePath, + "supersedes": admit.StringSliceToAny(supersedes), + "supersessionDeclarationRef": supersessionDeclarationRef, "testId": entry.TestID, "witnessRefs": admit.StringSliceToAny(entry.WitnessRefs), "wrongImplementationClassId": wrongImplementation, @@ -85,6 +105,24 @@ func entryCommandRefs(entries []testevidenceinventory.Entry) []string { return sortedUnique(values) } +func unprojectedTestEntries(entries []testevidenceinventory.Entry, rowGroups ...[]map[string]any) []testevidenceinventory.Entry { + projected := map[string]struct{}{} + for _, rows := range rowGroups { + for _, row := range rows { + for _, testID := range stringArray(row["testIds"]) { + projected[testID] = struct{}{} + } + } + } + result := make([]testevidenceinventory.Entry, 0) + for _, entry := range entries { + if _, ok := projected[entry.TestID]; !ok { + result = append(result, entry) + } + } + return result +} + func inventoryID(inventory *testevidenceinventory.Result) any { if inventory == nil { return nil diff --git a/internal/command/requirementcoverageview/requirementcoverageview_test.go b/internal/command/requirementcoverageview/requirementcoverageview_test.go index 55e57a8..a334893 100644 --- a/internal/command/requirementcoverageview/requirementcoverageview_test.go +++ b/internal/command/requirementcoverageview/requirementcoverageview_test.go @@ -6,10 +6,11 @@ import ( "testing" "github.com/research-engineering/agentic-proofkit/internal/kernel/admission" + "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" ) -func TestBuildJSONBuildsSemanticRequirementAndCommandCoverage(t *testing.T) { +func TestBuildJSONBuildsDeclaredRequirementAndCommandRouteMapping(t *testing.T) { view, exitCode, err := BuildJSON(validCoverageInput(t), Options{}) if err != nil { t.Fatalf("BuildJSON() error = %v", err) @@ -22,18 +23,53 @@ func TestBuildJSONBuildsSemanticRequirementAndCommandCoverage(t *testing.T) { t.Fatalf("unexpected view: %#v", record) } requirement := record["requirementCoverage"].([]any)[0].(map[string]any) - if requirement["coverageState"] != "covered_by_semantic_falsifier" { + if requirement["coverageState"] != "mapped_to_declared_semantic_falsifier_route" { t.Fatalf("coverageState=%v", requirement["coverageState"]) } if requirement["tests"].([]any)[0].(map[string]any)["oracleSummary"] == "" { t.Fatalf("semantic coverage should expose test oracle detail: %#v", requirement) } command := record["commandCoverage"].([]any)[0].(map[string]any) - if command["coverageState"] != "command_semantic_falsifier_present" { + if command["coverageState"] != "command_declared_semantic_falsifier_route_present" { t.Fatalf("command coverageState=%v", command["coverageState"]) } } +func TestBuildJSONMissingSelectorRemainsMappingOnly(t *testing.T) { + input := validCoverageInput(t) + entry := inventoryEntry(input) + entry["sourcePath"] = "internal/command/does-not-exist/missing_test.go" + entry["selector"] = "go test ./internal/command/does-not-exist -run TestMissing" + testSurface := input.(map[string]any)["coverageUniverse"].(map[string]any)["testSurfaces"].([]any)[0].(map[string]any) + testSurface["path"] = "internal/command/does-not-exist/missing_test.go" + + view, exitCode, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + if exitCode != 0 { + t.Fatalf("BuildJSON() exitCode=%d view=%#v, want successful mapping report", exitCode, view) + } + record := view.(map[string]any) + requirement := record["requirementCoverage"].([]any)[0].(map[string]any) + if got := requirement["coverageState"]; got != "mapped_to_declared_semantic_falsifier_route" { + t.Fatalf("coverageState=%v, want mapping-only state", got) + } + command := record["commandCoverage"].([]any)[0].(map[string]any) + if got := command["coverageState"]; got != "command_declared_semantic_falsifier_route_present" { + t.Fatalf("command coverageState=%v, want declared-route state", got) + } + for _, row := range []map[string]any{requirement, command} { + state := row["coverageState"].(string) + if strings.Contains(state, "covered_by") || strings.Contains(state, "verified") || strings.Contains(state, "closed") { + t.Fatalf("caller declaration acquired assurance state %q", state) + } + } + if !stringArrayContains(record["nonClaims"], "Mapped declaration routes do not prove selector resolution, oracle quality, execution, freshness, trust, sensitivity, or assurance closure.") { + t.Fatalf("mapping-only boundary non-claim missing: %#v", record["nonClaims"]) + } +} + func TestAdmitOutputRejectsUnknownNestedFieldsAndBreaksCallerAliases(t *testing.T) { view, _, err := BuildJSON(validCoverageInput(t), Options{}) if err != nil { @@ -56,6 +92,397 @@ func TestAdmitOutputRejectsUnknownNestedFieldsAndBreaksCallerAliases(t *testing. } } +func TestAdmitOutputRejectsAssuranceVocabularyAndInconsistentStateClasses(t *testing.T) { + cases := []struct { + name string + mutate func(map[string]any) + want string + }{ + { + name: "legacy covered state", + mutate: func(record map[string]any) { + record["requirementCoverage"].([]any)[0].(map[string]any)["coverageState"] = "covered_by_semantic_falsifier" + }, + want: "coverageState is unsupported", + }, + { + name: "legacy evidence class", + mutate: func(record map[string]any) { + test := record["requirementCoverage"].([]any)[0].(map[string]any)["tests"].([]any)[0].(map[string]any) + test["evidenceClass"] = "semantic_falsifier" + }, + want: "evidenceClass must be one of", + }, + { + name: "verified owner review state", + mutate: func(record map[string]any) { + test := record["requirementCoverage"].([]any)[0].(map[string]any)["tests"].([]any)[0].(map[string]any) + test["qualityFindings"] = []any{map[string]any{ + "class": "missing_edge", "evidenceRefs": []any{"test.coverage.semantic"}, + "findingId": "finding.coverage.forged", "nonClaims": []any{}, + "ownerReviewState": "verified", "severity": "warning", + }} + }, + want: "ownerReviewState must be one of", + }, + { + name: "verified requirement proof state", + mutate: func(record map[string]any) { + record["requirementCoverage"].([]any)[0].(map[string]any)["proofState"] = "verified" + }, + want: "proofState must be one of", + }, + { + name: "inconsistent mapping pair", + mutate: func(record map[string]any) { + record["requirementCoverage"].([]any)[0].(map[string]any)["evidenceClass"] = "declared_property_or_fuzz_route" + }, + want: "coverageState and evidenceClass are inconsistent", + }, + { + name: "closed command state", + mutate: func(record map[string]any) { + record["commandCoverage"].([]any)[0].(map[string]any)["coverageState"] = "closed" + }, + want: "coverageState must be one of", + }, + } + for _, item := range cases { + t.Run(item.name, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + item.mutate(record) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.want) { + t.Fatalf("AdmitOutput() error=%v, want %q", err, item.want) + } + }) + } +} + +func TestAdmitOutputRejectsForgedDerivedCoverageAlgebra(t *testing.T) { + cases := []struct { + name string + mutate func(map[string]any) + want string + }{ + { + name: "passed state with failures", + mutate: func(record map[string]any) { + record["failures"] = []any{"proof_route_candidate_only:REQ-PROOFKIT-COVERAGE-001"} + record["failureCount"] = 1 + }, + want: "state is inconsistent with failures", + }, + { + name: "forged classification", + mutate: func(record map[string]any) { + record["failureClassifications"] = []any{map[string]any{ + "classificationId": "verified", + "diagnostic": "forged", + "severity": "failure", + }} + }, + want: "failureClassifications is inconsistent", + }, + { + name: "forged guidance", + mutate: func(record map[string]any) { + record["guidanceSummary"].(map[string]any)["state"] = "failed" + }, + want: "guidanceSummary is inconsistent", + }, + { + name: "parent state disagrees with projected child class", + mutate: func(record map[string]any) { + mutateProjectedTestCopies(t, record, "test.coverage.semantic", func(test map[string]any) { + test["evidenceClass"] = "proof_route_candidate" + }) + refreshCoverageBasisInventoryDigest(t, record) + }, + want: "not derived from projected tests and lifecycle", + }, + { + name: "parent test ids disagree with projection", + mutate: func(record map[string]any) { + record["requirementCoverage"].([]any)[0].(map[string]any)["testIds"] = []any{} + }, + want: "must equal projected test identities", + }, + } + for _, item := range cases { + t.Run(item.name, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + item.mutate(record) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.want) { + t.Fatalf("AdmitOutput() error=%v, want %q", err, item.want) + } + }) + } +} + +func TestAdmitOutputRejectsAggregateRowDiagnosticDrift(t *testing.T) { + for _, item := range []struct { + name string + diagnostic string + }{ + {name: "requirement failure", diagnostic: "proof_route_candidate_only:REQ-PROOFKIT-COVERAGE-001"}, + {name: "command failure", diagnostic: "command_proof_route_candidate_only:proofkit.coverage.command"}, + } { + t.Run(item.name, func(t *testing.T) { + input := validCoverageInput(t) + entry := inventoryEntry(input) + entry["evidenceClass"] = "proof_route_candidate" + entry["falsifier"] = nil + entry["oracle"] = nil + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + setCoverageAggregateDiagnostics(record, withoutDiagnostic(stringArray(record["failures"]), item.diagnostic), stringArray(record["warnings"])) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "inconsistent with retained rows") { + t.Fatalf("AdmitOutput() error=%v, want aggregate row diagnostic rejection", err) + } + }) + } + + t.Run("owner warning", func(t *testing.T) { + input := validCoverageInput(t).(map[string]any) + input["ownerInvariantRegistry"] = map[string]any{ + "schemaVersion": json.Number("1"), + "registryId": "proofkit.coverage.owner-invariants", + "invariants": []any{map[string]any{ + "ownerInvariantId": "invariant.coverage.unmapped", + "ownerId": "proofkit.coverage", + "sourcePath": "docs/specs/proofkit-coverage/requirements.v1.json", + "summary": "Unmapped owner invariant fixture.", + "nonClaims": []any{"Fixture does not claim native execution."}, + }}, + "nonClaims": []any{"Registry fixture is caller-owned."}, + } + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + diagnostic := "missing_owner_invariant_inventory:invariant.coverage.unmapped" + setCoverageAggregateDiagnostics(record, stringArray(record["failures"]), withoutDiagnostic(stringArray(record["warnings"]), diagnostic)) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "inconsistent with retained rows") { + t.Fatalf("AdmitOutput() error=%v, want aggregate owner warning rejection", err) + } + }) + + t.Run("surplus requirement diagnostic", func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + failures := append(stringArray(record["failures"]), "missing_test_inventory:REQ-PROOFKIT-COVERAGE-FORGED") + setCoverageAggregateDiagnostics(record, sortedUnique(failures), stringArray(record["warnings"])) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "inconsistent with retained rows") { + t.Fatalf("AdmitOutput() error=%v, want surplus row diagnostic rejection", err) + } + }) +} + +func TestAdmitOutputRejectsMalformedCommandCoverageRows(t *testing.T) { + cases := []struct { + name string + mutate func(map[string]any) + want string + }{ + {name: "missing failures", mutate: func(row map[string]any) { delete(row, "failures") }, want: "missing required field failures"}, + {name: "missing test ids", mutate: func(row map[string]any) { delete(row, "testIds") }, want: "missing required field testIds"}, + {name: "wrong test ids type", mutate: func(row map[string]any) { row["testIds"] = "test.coverage.semantic" }, want: "testIds must be an array"}, + {name: "empty test ids with present route", mutate: func(row map[string]any) { row["testIds"] = []any{} }, want: "must equal projected test identities"}, + {name: "duplicate test ids", mutate: func(row map[string]any) { row["testIds"] = []any{"test.coverage.semantic", "test.coverage.semantic"} }, want: "must be sorted and unique"}, + {name: "unsorted test ids", mutate: func(row map[string]any) { row["testIds"] = []any{"test.coverage.z", "test.coverage.a"} }, want: "must be sorted and unique"}, + {name: "invalid test id", mutate: func(row map[string]any) { row["testIds"] = []any{"not a rule id"} }, want: "must equal projected test identities"}, + {name: "forged failures", mutate: func(row map[string]any) { row["failures"] = []any{"forged"} }, want: "failures is inconsistent"}, + } + for _, item := range cases { + t.Run(item.name, func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + item.mutate(record["commandCoverage"].([]any)[0].(map[string]any)) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), item.want) { + t.Fatalf("AdmitOutput() error=%v, want %q", err, item.want) + } + }) + } +} + +func TestAdmitOutputRejectsMissingCommandStateWithTestIdentity(t *testing.T) { + input := validCoverageInput(t) + universe := input.(map[string]any)["coverageUniverse"].(map[string]any) + universe["commandRefs"] = []any{"proofkit.coverage.command", "proofkit.coverage.uncovered"} + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + for _, rawRow := range record["commandCoverage"].([]any) { + row := rawRow.(map[string]any) + if row["commandId"] == "proofkit.coverage.uncovered" { + row["testIds"] = []any{"test.coverage.forged"} + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "must equal projected test identities") { + t.Fatalf("AdmitOutput() error=%v, want command test-projection cardinality rejection", err) + } + return + } + } + t.Fatal("missing command row not found") +} + +func TestAdmitOutputRejectsCommandStateWeakerThanRetainedTestProjection(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + command := record["commandCoverage"].([]any)[0].(map[string]any) + command["coverageState"] = "command_proof_route_candidate_only" + command["failures"] = []any{"command_proof_route_candidate_only:proofkit.coverage.command"} + setCoverageAggregateDiagnostics(record, []string{"command_proof_route_candidate_only:proofkit.coverage.command"}, stringArray(record["warnings"])) + + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "not derived from projected tests") { + t.Fatalf("AdmitOutput() error=%v, want correlated command-state rejection", err) + } +} + +func TestAdmitOutputRejectsProjectedTestParentRefDrift(t *testing.T) { + t.Run("requirement parent", func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + mutateProjectedTestCopies(t, record, "test.coverage.semantic", func(test map[string]any) { + test["requirementRefs"] = []any{"REQ-PROOFKIT-COVERAGE-OTHER"} + }) + refreshCoverageBasisInventoryDigest(t, record) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "does not reference its projected parent") { + t.Fatalf("AdmitOutput() error=%v, want requirement parent-ref rejection", err) + } + }) + + t.Run("owner invariant parent", func(t *testing.T) { + input := validCoverageInput(t).(map[string]any) + input["ownerInvariantRegistry"] = map[string]any{ + "schemaVersion": json.Number("1"), "registryId": "proofkit.coverage.owner-invariants", + "invariants": []any{map[string]any{ + "ownerInvariantId": "invariant.coverage.semantic", "ownerId": "proofkit.coverage", + "sourcePath": "docs/specs/proofkit-coverage/requirements.v1.json", + "summary": "Coverage owner invariant fixture.", "nonClaims": []any{"Fixture does not claim execution."}, + }}, + "nonClaims": []any{"Registry fixture is caller-owned."}, + } + entry := inventoryEntry(input) + entry["ownerInvariantRefs"] = []any{"invariant.coverage.semantic"} + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + mutateProjectedTestCopies(t, record, "test.coverage.semantic", func(test map[string]any) { + test["ownerInvariantRefs"] = []any{"invariant.coverage.other"} + }) + refreshCoverageBasisInventoryDigest(t, record) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "does not reference its projected parent") { + t.Fatalf("AdmitOutput() error=%v, want owner-invariant parent-ref rejection", err) + } + }) +} + +func TestAdmitOutputRejectsInconsistentRepeatedTestProjection(t *testing.T) { + input := validCoverageInput(t).(map[string]any) + input["ownerInvariantRegistry"] = map[string]any{ + "schemaVersion": json.Number("1"), "registryId": "proofkit.coverage.owner-invariants", + "invariants": []any{map[string]any{ + "ownerInvariantId": "invariant.coverage.semantic", "ownerId": "proofkit.coverage", + "sourcePath": "docs/specs/proofkit-coverage/requirements.v1.json", + "summary": "Coverage owner invariant fixture.", "nonClaims": []any{"Fixture does not claim execution."}, + }}, + "nonClaims": []any{"Registry fixture is caller-owned."}, + } + entry := inventoryEntry(input) + entry["ownerInvariantRefs"] = []any{"invariant.coverage.semantic"} + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + requirementTest := record["requirementCoverage"].([]any)[0].(map[string]any)["tests"].([]any)[0].(map[string]any) + ownerTest := record["ownerInvariantCoverage"].([]any)[0].(map[string]any)["tests"].([]any)[0].(map[string]any) + requirementTest["ownerInvariantRefs"] = []any{} + ownerTest["requirementRefs"] = []any{} + + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "inconsistent projections") { + t.Fatalf("AdmitOutput() error=%v, want repeated test projection rejection", err) + } +} + +func TestSelectRequirementsDoesNotAliasAdmittedCoverage(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + admitted, err := AdmitOutput(view) + if err != nil { + t.Fatalf("AdmitOutput() error = %v", err) + } + fragment := SelectRequirements(admitted, map[string]struct{}{"REQ-PROOFKIT-COVERAGE-001": {}}) + fragmentRow := fragment["requirementCoverage"].([]any)[0].(map[string]any) + fragmentRow["coverageState"] = "mutated" + fragmentTest := fragmentRow["tests"].([]any)[0].(map[string]any) + fragmentTest["commandRefs"].([]any)[0] = "proofkit.mutated" + + admittedRow := admitted["requirementCoverage"].([]any)[0].(map[string]any) + if admittedRow["coverageState"] == "mutated" { + t.Fatal("SelectRequirements retained a row-map alias") + } + admittedTest := admittedRow["tests"].([]any)[0].(map[string]any) + if admittedTest["commandRefs"].([]any)[0] == "proofkit.mutated" { + t.Fatal("SelectRequirements retained a nested-array alias") + } +} + +func setCoverageAggregateDiagnostics(record map[string]any, failures []string, warnings []string) { + state := "passed" + if len(failures) > 0 { + state = "failed" + } + record["failures"] = admit.StringSliceToAny(failures) + record["failureCount"] = len(failures) + record["failureClassifications"] = mapsToAny(diagnosticClassifications(failures, "failure")) + record["warnings"] = admit.StringSliceToAny(warnings) + record["warningCount"] = len(warnings) + record["warningClassifications"] = mapsToAny(diagnosticClassifications(warnings, "warning")) + record["state"] = state + record["guidanceSummary"] = guidanceSummary(state, failures, warnings) +} + +func withoutDiagnostic(values []string, removed string) []string { + result := make([]string, 0, len(values)) + for _, value := range values { + if value != removed { + result = append(result, value) + } + } + return result +} + func TestBuildJSONDoesNotPromoteProofRouteCandidateToSemanticCoverage(t *testing.T) { input := validCoverageInput(t) entry := inventoryEntry(input) @@ -78,6 +505,13 @@ func TestBuildJSONDoesNotPromoteProofRouteCandidateToSemanticCoverage(t *testing if command["coverageState"] != "command_proof_route_candidate_only" { t.Fatalf("proof-route command state=%v, want explicit candidate-only state", command["coverageState"]) } + requireExactClassifications(t, record, "failures", "failureClassifications", "failure", map[string]string{ + "command_proof_route_candidate_only:proofkit.coverage.command": "proof_route_candidate_only", + "proof_route_candidate_only:REQ-PROOFKIT-COVERAGE-001": "proof_route_candidate_only", + }) + if _, err := AdmitOutput(record); err != nil { + t.Fatalf("AdmitOutput(candidate-only report) error = %v", err) + } } func TestBuildJSONUsesNormalizedOnlyInventoryProjection(t *testing.T) { @@ -227,7 +661,7 @@ func TestBuildJSONCompactProjectionAggregatesScenariosAndRequirementLocalCommand "selector": "go test ./internal/command/requirementcoverageview -run TestUnrelated", "sourcePath": "internal/command/requirementcoverageview/requirementcoverageview_test.go", "ownerId": "proofkit.coverage", - "evidenceClass": "semantic_falsifier", + "evidenceClass": "declared_semantic_falsifier_route", "requirementRefs": []any{"REQ-PROOFKIT-COVERAGE-002"}, "ownerInvariantRefs": []any{}, "commandRefs": []any{"proofkit.coverage.unrelated"}, @@ -282,7 +716,7 @@ func TestBuildJSONCompactProjectionAggregatesScenariosAndRequirementLocalCommand } } -func TestBuildJSONCompactProjectionRejectsConflictingRequirementProofStates(t *testing.T) { +func TestBuildJSONCompactProjectionRejectsNonWitnessBackedBindingState(t *testing.T) { input := validCoverageInput(t) record := input.(map[string]any) second := compactCoverageBinding("proofkit.coverage::scenario.conflicting") @@ -298,8 +732,8 @@ func TestBuildJSONCompactProjectionRejectsConflictingRequirementProofStates(t *t } _, _, err := BuildJSON(input, Options{}) - if err == nil || !strings.Contains(err.Error(), "conflicting proofContractState") { - t.Fatalf("expected conflicting compact proof state error, got %v", err) + if err == nil || !strings.Contains(err.Error(), "must be witness_backed") { + t.Fatalf("expected witness-bearing compact state rejection, got %v", err) } } @@ -355,7 +789,7 @@ func TestBuildJSONRejectsRouteOnlyCoverageForBlockingRequirement(t *testing.T) { t.Fatalf("route-only command state not preserved: %#v", command) } requireExactClassifications(t, record, "failures", "failureClassifications", "failure", map[string]string{ - "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_semantic_test", + "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_declared_test_route", }) requireExactClassifications(t, record, "warnings", "warningClassifications", "warning", map[string]string{ "command_route_only_nonclaim:proofkit.coverage.command": "routing_smoke_only", @@ -395,7 +829,7 @@ func TestBuildJSONRejectsInventorySelectorSourcePathDrift(t *testing.T) { } } -func TestBuildJSONFailsMissingCommandSemanticFalsifierFromUniverse(t *testing.T) { +func TestBuildJSONFailsMissingCommandDeclaredSemanticFalsifierRouteFromUniverse(t *testing.T) { input := validCoverageInput(t) universe := input.(map[string]any)["coverageUniverse"].(map[string]any) universe["commandRefs"] = []any{"proofkit.coverage.command", "proofkit.coverage.uncovered"} @@ -408,11 +842,11 @@ func TestBuildJSONFailsMissingCommandSemanticFalsifierFromUniverse(t *testing.T) t.Fatalf("missing command semantic falsifier passed: %#v", view) } failures := strings.Join(stringArray(view.(map[string]any)["failures"]), "\n") - if !strings.Contains(failures, "missing_command_semantic_falsifier:proofkit.coverage.uncovered") { + if !strings.Contains(failures, "missing_command_declared_semantic_falsifier_route:proofkit.coverage.uncovered") { t.Fatalf("missing command failure not reported: %s", failures) } requireExactClassifications(t, view.(map[string]any), "failures", "failureClassifications", "failure", map[string]string{ - "missing_command_semantic_falsifier:proofkit.coverage.uncovered": "missing_semantic_test", + "missing_command_declared_semantic_falsifier_route:proofkit.coverage.uncovered": "missing_declared_test_route", }) } @@ -450,8 +884,8 @@ func TestBuildJSONProjectsOwnerInvariantCoverage(t *testing.T) { } invariant := coverage[0].(map[string]any) if invariant["ownerInvariantId"] != "invariant.coverage.semantic" || - invariant["coverageState"] != "covered_by_semantic_falsifier" || - invariant["evidenceClass"] != "semantic_falsifier" { + invariant["coverageState"] != "mapped_to_declared_semantic_falsifier_route" || + invariant["evidenceClass"] != "declared_semantic_falsifier_route" { t.Fatalf("unexpected owner invariant coverage: %#v", invariant) } tests := invariant["tests"].([]any) @@ -519,6 +953,59 @@ func TestBuildJSONKeepsDeadZonesAdvisoryForSelectedPaths(t *testing.T) { }) } +func TestAdmitOutputRejectsDeadZoneProjectionDrift(t *testing.T) { + for _, declaration := range []string{"selected_owner_surfaces", "selected_paths_advisory"} { + t.Run(declaration+" missing aggregate diagnostic", func(t *testing.T) { + input := validCoverageInput(t) + addUnboundCodeSurface(input, declaration) + view, _, err := BuildJSON(input, Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + failures := stringArray(record["failures"]) + warnings := stringArray(record["warnings"]) + if declaration == "selected_paths_advisory" { + warnings = withoutDiagnostic(warnings, "dead_zone_advisory:unbound_code_surface:proofkit.unbound.code") + } else { + failures = withoutDiagnostic(failures, "dead_zone:unbound_code_surface:proofkit.unbound.code") + } + setCoverageAggregateDiagnostics(record, failures, warnings) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "retained projection") { + t.Fatalf("AdmitOutput() error=%v, want retained dead-zone diagnostic rejection", err) + } + }) + } + + t.Run("surplus aggregate diagnostic", func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + failures := append(stringArray(record["failures"]), "dead_zone:unbound_code_surface:proofkit.forged") + setCoverageAggregateDiagnostics(record, sortedUnique(failures), stringArray(record["warnings"])) + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "retained projection") { + t.Fatalf("AdmitOutput() error=%v, want surplus dead-zone rejection", err) + } + }) + + t.Run("malformed row", func(t *testing.T) { + view, _, err := BuildJSON(validCoverageInput(t), Options{}) + if err != nil { + t.Fatalf("BuildJSON() error = %v", err) + } + record := view.(map[string]any) + record["deadZones"] = []any{map[string]any{ + "deadZoneKind": "unbound_code_surface", "ownerId": "proofkit.coverage", + "path": "../escape", "surfaceId": "proofkit.forged", + }} + if _, err := AdmitOutput(record); err == nil || !strings.Contains(err.Error(), "must not escape the repository root") { + t.Fatalf("AdmitOutput() error=%v, want malformed dead-zone rejection", err) + } + }) +} + func TestBuildJSONClassifiesFailedTestInventory(t *testing.T) { input := validCoverageInput(t) oracle := inventoryEntry(input)["oracle"].(map[string]any) @@ -532,7 +1019,7 @@ func TestBuildJSONClassifiesFailedTestInventory(t *testing.T) { t.Fatalf("failed inventory coverage view passed: %#v", view) } requireExactClassifications(t, view.(map[string]any), "failures", "failureClassifications", "failure", map[string]string{ - "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_semantic_test", + "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_declared_test_route", "test_inventory_failed:proofkit.coverage.inventory": "failed_test_inventory", }) } @@ -551,7 +1038,7 @@ func TestBuildJSONRejectsUnknownInventoryRefs(t *testing.T) { }, want: "unknown_requirement_ref:test.coverage.semantic:REQ-PROOFKIT-COVERAGE-999", expected: map[string]string{ - "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_semantic_test", + "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_declared_test_route", "unknown_requirement_ref:test.coverage.semantic:REQ-PROOFKIT-COVERAGE-999": "unknown_reference", }, }, @@ -572,7 +1059,7 @@ func TestBuildJSONRejectsUnknownInventoryRefs(t *testing.T) { }, want: "unknown_command_or_witness_ref:test.coverage.semantic:proofkit.coverage.unknown", expected: map[string]string{ - "missing_command_semantic_falsifier:proofkit.coverage.command": "missing_semantic_test", + "missing_command_declared_semantic_falsifier_route:proofkit.coverage.command": "missing_declared_test_route", "unknown_command_or_witness_ref:test.coverage.semantic:proofkit.coverage.unknown": "unknown_reference", }, }, @@ -622,7 +1109,7 @@ func TestBuildJSONClassifiesUnknownRefsByDiagnosticKindNotPayloadText(t *testing t.Fatalf("unknown requirement ref passed: %#v", view) } requireExactClassifications(t, view.(map[string]any), "failures", "failureClassifications", "failure", map[string]string{ - "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_semantic_test", + "missing_test_inventory:REQ-PROOFKIT-COVERAGE-001": "missing_declared_test_route", "unknown_requirement_ref:test.routing_smoke_nonclaim:REQ-PROOFKIT-COVERAGE-999": "unknown_reference", }) } @@ -799,14 +1286,17 @@ func TestBuildJSONRejectsInventoryEntryOutsideOwnerScope(t *testing.T) { } func TestBuildJSONRejectsGovernanceEvidenceForBlockingSemanticRequirement(t *testing.T) { - input := validCoverageInput(t) - entry := inventoryEntry(input) - entry["evidenceClass"] = "governance_or_release" - entry["falsifier"] = nil - entry["oracle"] = nil - entry["nonClaims"] = []any{"Governance evidence is not semantic product coverage."} + newInput := func() any { + input := validCoverageInput(t) + entry := inventoryEntry(input) + entry["evidenceClass"] = "governance_or_release" + entry["falsifier"] = nil + entry["oracle"] = nil + entry["nonClaims"] = []any{"Governance evidence is not semantic product coverage."} + return input + } - view, exitCode, err := BuildJSON(input, Options{}) + view, exitCode, err := BuildJSON(newInput(), Options{}) if err != nil { t.Fatalf("BuildJSON() error = %v", err) } @@ -814,7 +1304,7 @@ func TestBuildJSONRejectsGovernanceEvidenceForBlockingSemanticRequirement(t *tes t.Fatalf("governance evidence satisfied blocking semantic coverage: %#v", view) } requirement := view.(map[string]any)["requirementCoverage"].([]any)[0].(map[string]any) - if requirement["coverageState"] != "covered_by_governance_invariant_nonproduct" { + if requirement["coverageState"] != "mapped_to_declared_governance_or_release_route_nonproduct" { t.Fatalf("unexpected governance coverage state: %#v", requirement) } command := view.(map[string]any)["commandCoverage"].([]any)[0].(map[string]any) @@ -822,17 +1312,42 @@ func TestBuildJSONRejectsGovernanceEvidenceForBlockingSemanticRequirement(t *tes t.Fatalf("governance evidence should not satisfy semantic command closure: %#v", command) } failures := strings.Join(stringArray(view.(map[string]any)["failures"]), "\n") - if !strings.Contains(failures, "covered_by_governance_invariant_nonproduct:REQ-PROOFKIT-COVERAGE-001") { + if !strings.Contains(failures, "mapped_to_declared_governance_or_release_route_nonproduct:REQ-PROOFKIT-COVERAGE-001") { t.Fatalf("governance non-semantic failure missing: %s", failures) } requireExactClassifications(t, view.(map[string]any), "failures", "failureClassifications", "failure", map[string]string{ - "covered_by_governance_invariant_nonproduct:REQ-PROOFKIT-COVERAGE-001": "nonsemantic_governance_evidence", - "nonsemantic_command_evidence:proofkit.coverage.command": "nonsemantic_command_evidence", + "mapped_to_declared_governance_or_release_route_nonproduct:REQ-PROOFKIT-COVERAGE-001": "nonsemantic_governance_evidence", + "nonsemantic_command_evidence:proofkit.coverage.command": "nonsemantic_command_evidence", }) + + markdown, _, err := BuildMarkdown(newInput()) + if err != nil { + t.Fatalf("BuildMarkdown() error = %v", err) + } + html, _, err := BuildHTML(newInput()) + if err != nil { + t.Fatalf("BuildHTML() error = %v", err) + } + envelope, _, err := BuildJSON(newInput(), Options{AgentEnvelope: true}) + if err != nil { + t.Fatalf("BuildJSON(agent envelope) error = %v", err) + } + encodedView, _ := json.Marshal(view) + encodedEnvelope, _ := json.Marshal(envelope) + for name, output := range map[string]string{ + "agent envelope": string(encodedEnvelope), + "HTML": html, + "JSON": string(encodedView), + "Markdown": markdown, + } { + if strings.Contains(output, "covered_by") { + t.Fatalf("%s laundered declaration mapping into coverage vocabulary: %s", name, output) + } + } } func TestBuildJSONClassifiesNonsemanticCommandEvidenceClasses(t *testing.T) { - for _, evidenceClass := range []string{"benchmark", "contract_admission", "helper_or_testkit", "property_or_fuzz"} { + for _, evidenceClass := range []string{"benchmark", "declared_contract_admission_route", "helper_or_testkit", "declared_property_or_fuzz_route"} { t.Run(evidenceClass, func(t *testing.T) { input := validCoverageInput(t) entry := inventoryEntry(input) @@ -863,7 +1378,7 @@ func TestBuildJSONWarnsForNonsemanticCommandEvidenceInAdvisoryScope(t *testing.T input := validCoverageInput(t) input.(map[string]any)["coverageUniverse"].(map[string]any)["completenessDeclaration"] = "selected_paths_advisory" entry := inventoryEntry(input) - entry["evidenceClass"] = "contract_admission" + entry["evidenceClass"] = "declared_contract_admission_route" entry["nonClaims"] = []any{"Contract admission evidence fixture does not prove command behavior."} view, exitCode, err := BuildJSON(input, Options{}) @@ -916,7 +1431,7 @@ func TestBuildJSONWarnsWhenOwnerInvariantHasNoInventory(t *testing.T) { if got := stringArray(invariant["warnings"]); len(got) != 1 || got[0] != "missing_owner_invariant_inventory:invariant.coverage.uncovered" { t.Fatalf("row-local owner invariant warnings=%v", got) } - requireClassificationIncludes(t, view.(map[string]any), "warnings", "warningClassifications", "warning", "missing_owner_invariant_inventory:invariant.coverage.uncovered", "missing_semantic_test") + requireClassificationIncludes(t, view.(map[string]any), "warnings", "warningClassifications", "warning", "missing_owner_invariant_inventory:invariant.coverage.uncovered", "missing_declared_test_route") } func TestBuildJSONRequiresExactlyOneProofInput(t *testing.T) { @@ -972,11 +1487,10 @@ func TestDiagnosticClassIDVocabulary(t *testing.T) { {"dead_zone_advisory:unbound_code_surface:proofkit.unbound.code", "declared_dead_zone"}, {"missing_proof_binding_route:REQ-PROOFKIT-COVERAGE-001", "missing_requirement_binding"}, {"proof_binding_unknown_requirement:REQ-PROOFKIT-COVERAGE-999", "missing_requirement_binding"}, - {"missing_test_inventory:REQ-PROOFKIT-COVERAGE-001", "missing_semantic_test"}, - {"missing_command_semantic_falsifier:proofkit.coverage.command", "missing_semantic_test"}, + {"missing_test_inventory:REQ-PROOFKIT-COVERAGE-001", "missing_declared_test_route"}, + {"missing_command_declared_semantic_falsifier_route:proofkit.coverage.command", "missing_declared_test_route"}, {"route_only_nonclaim:REQ-PROOFKIT-COVERAGE-001", "routing_smoke_only"}, {"command_route_only_nonclaim:proofkit.coverage.command", "routing_smoke_only"}, - {"covered_by_routing_smoke_nonclaim:REQ-PROOFKIT-COVERAGE-001", "routing_smoke_only"}, {"unknown_requirement_ref:test.routing_smoke_nonclaim:REQ-PROOFKIT-COVERAGE-999", "unknown_reference"}, {"unknown_requirement_ref:test.coverage:REQ-UNKNOWN", "unknown_reference"}, {"unknown_owner_invariant_ref:test.coverage:invariant.unknown", "unknown_reference"}, @@ -984,8 +1498,8 @@ func TestDiagnosticClassIDVocabulary(t *testing.T) { {"full_repository_source_requirement_outside_owner_scope:REQ-PROOFKIT-COVERAGE-999", "owner_scope_violation"}, {"inventory_entry_owner_outside_scope:test.coverage:proofkit.other", "owner_scope_violation"}, {"test_inventory_failed:repo.tests", "failed_test_inventory"}, - {"covered_by_governance_invariant_nonproduct:REQ-PROOFKIT-COVERAGE-001", "nonsemantic_governance_evidence"}, - {"missing_owner_invariant_inventory:invariant.coverage.semantic", "missing_semantic_test"}, + {"mapped_to_declared_governance_or_release_route_nonproduct:REQ-PROOFKIT-COVERAGE-001", "nonsemantic_governance_evidence"}, + {"missing_owner_invariant_inventory:invariant.coverage.semantic", "missing_declared_test_route"}, {"nonsemantic_command_evidence:proofkit.coverage.command", "nonsemantic_command_evidence"}, {"not_applicable:REQ-PROOFKIT-COVERAGE-001", "not_applicable_with_reason"}, {"future_unmapped_diagnostic:example", "unclassified_gap"}, @@ -1159,7 +1673,7 @@ func validCoverageInput(t *testing.T) any { "selector": "go test ./internal/command/requirementcoverageview -run TestCoverage", "sourcePath": "internal/command/requirementcoverageview/requirementcoverageview_test.go", "ownerId": "proofkit.coverage", - "evidenceClass": "semantic_falsifier", + "evidenceClass": "declared_semantic_falsifier_route", "requirementRefs": ["REQ-PROOFKIT-COVERAGE-001"], "ownerInvariantRefs": [], "commandRefs": ["proofkit.coverage.command"], diff --git a/internal/command/requirementcoverageview/types.go b/internal/command/requirementcoverageview/types.go index d1ade21..58521c3 100644 --- a/internal/command/requirementcoverageview/types.go +++ b/internal/command/requirementcoverageview/types.go @@ -10,7 +10,9 @@ var defaultNonClaims = []string{ "Requirement coverage views are rendered lookup products only.", "Requirement coverage views do not scan repositories, execute native tests, authenticate receipts, decide proof freshness, or approve merge.", "Requirement coverage views classify only caller-owned structured inputs.", - "Route-only smoke evidence cannot satisfy semantic requirement coverage.", + "Mapped declaration routes do not prove selector resolution, oracle quality, execution, freshness, trust, sensitivity, or assurance closure.", + "Coverage basis commitments prove internal projection consistency only; they do not authenticate source inputs or producer identity.", + "Route-only smoke entries do not satisfy declared semantic-falsifier route mapping.", } type Options struct { @@ -80,7 +82,9 @@ type scenario struct { CommandIDs []string EnvironmentClasses []string ScenarioID string + VerifyCommands []string WitnessID string WitnessKind string WitnessPath string + WitnessSelectors []string } diff --git a/internal/command/requirementsourceadmission/projection.go b/internal/command/requirementsourceadmission/projection.go index 13a9a35..cb32d4a 100644 --- a/internal/command/requirementsourceadmission/projection.go +++ b/internal/command/requirementsourceadmission/projection.go @@ -8,6 +8,27 @@ type ComparisonField struct { Value any } +// AdmitRequirement exposes the requirement owner's canonical leaf admission. +func AdmitRequirement(raw any) (Requirement, error) { + return admitRequirement(raw) +} + +func AdmitRequirementID(raw any, context string) (string, error) { + return requirementID(raw, context) +} + +func AdmitClaimLevel(raw any, context string) (string, error) { + return enum(raw, claimLevelSet, claimLevels, context) +} + +func AdmitLifecycleState(raw any, context string) (string, error) { + return enum(raw, lifecycleStateSet, lifecycleStates, context) +} + +func AdmitInvariantText(raw any, context string) (string, error) { + return invariantText(raw, context) +} + func SourceValue(source Source) map[string]any { requirements := make([]any, 0, len(source.Requirements)) for _, requirement := range source.Requirements { diff --git a/internal/command/stackpreset/preset_ids_generated.go b/internal/command/stackpreset/preset_ids_generated.go index 38af489..95ac412 100644 --- a/internal/command/stackpreset/preset_ids_generated.go +++ b/internal/command/stackpreset/preset_ids_generated.go @@ -1,6 +1,6 @@ // Code generated by internal/tools/commandcontractgen; DO NOT EDIT. package stackpreset -const presetContractSourceSHA256 = "a6d6faaa1da035747847d7e2f9232857aa9e342413a1f7c2408ce2ca4cbe832b" +const presetContractSourceSHA256 = "3d2fab7d4b8da24fb1c105ce8814b531926c50dee9a594eacad953f8af897132" var presetIDs = []string{"agentic_runtime_repo", "generated_docs_contract_repo", "python_service", "python_typescript_service", "typescript_monorepo", "typescript_workspace"} diff --git a/internal/command/testevidenceinventory/discovery_draft.go b/internal/command/testevidenceinventory/discovery_draft.go index 7aca101..27e8d6d 100644 --- a/internal/command/testevidenceinventory/discovery_draft.go +++ b/internal/command/testevidenceinventory/discovery_draft.go @@ -94,14 +94,14 @@ func BuildDiscoveryDraft(raw any) (report.Record, int, error) { ReportID: input.DraftID, State: "passed", Summary: map[string]any{ - "agentActionCount": len(actions), - "candidateInventoryEntryCount": len(input.DiscoveredTests), - "discoveredTestCount": len(input.DiscoveredTests), - "fragileSelectorWarningCount": countPrefix(warnings, "selector_fragility:"), - "missingAnchorWarningCount": countPrefix(warnings, "missing_semantic_anchor:"), - "runnerKind": input.Runner.RunnerKind, - "weakOracleWarningCount": countPrefix(warnings, "weak_or_empty_oracle:"), - "warningCount": len(warnings), + "agentActionCount": len(actions), + "candidateInventoryEntryCount": len(input.DiscoveredTests), + "discoveredTestCount": len(input.DiscoveredTests), + "fragileSelectorWarningCount": countPrefix(warnings, "selector_fragility:"), + "missingAnchorWarningCount": countPrefix(warnings, "missing_declared_route_anchor:"), + "runnerKind": input.Runner.RunnerKind, + "missingDeclaredAssertionSignalWarningCount": countPrefix(warnings, "missing_declared_assertion_signal:"), + "warningCount": len(warnings), }, Diagnostics: []report.Diagnostic{ {Key: "agentActionPlan", Value: mapsToAny(actions)}, @@ -343,10 +343,10 @@ func discoveryWarnings(input discoveryDraftInput) []string { warnings := []string{} for _, test := range input.DiscoveredTests { if len(test.CandidateRequirementRefs) == 0 && len(test.OwnerInvariantRefs) == 0 { - warnings = append(warnings, "missing_semantic_anchor:"+test.TestID) + warnings = append(warnings, "missing_declared_route_anchor:"+test.TestID) } - if weakOracleSignals(test.OracleSignals) { - warnings = append(warnings, "weak_or_empty_oracle:"+test.TestID) + if !hasDeclaredAssertionSignal(test.OracleSignals) { + warnings = append(warnings, "missing_declared_assertion_signal:"+test.TestID) } if fragileSelectorSignals(test.SelectorSignals) { warnings = append(warnings, "selector_fragility:"+test.TestID) @@ -385,12 +385,12 @@ func discoveryActionMessage(actionType string) string { switch actionType { case "candidate_only": return "Materialize owner-reviewed strict test-evidence-inventory before using this test as coverage evidence." - case "missing_semantic_anchor": - return "Attach the test to a stable requirement or owner invariant before claiming semantic coverage." + case "missing_declared_route_anchor": + return "Attach the test to a stable requirement or owner invariant before materializing declared route mapping." case "selector_fragility": return "Review selector stability before materializing this test as durable evidence." - case "weak_or_empty_oracle": - return "Add a falsifiable oracle or demote the test to route-only evidence." + case "missing_declared_assertion_signal": + return "Declare at least one assertion-capable signal or keep the candidate as route-only evidence." default: return "Review discovered test before materializing strict inventory." } @@ -401,7 +401,7 @@ func discoveryCandidateInventory(input discoveryDraftInput) map[string]any { scopeNonClaims := discoveryScopeNonClaims(input) for _, test := range input.DiscoveredTests { entryNonClaims := []string{ - "Discovery draft rows are candidate-only and cannot close semantic coverage.", + "Discovery draft rows are candidate-only and cannot close execution or assurance coverage.", } entryNonClaims = append(entryNonClaims, scopeNonClaims...) entryNonClaims = append(entryNonClaims, test.NonClaims...) @@ -444,10 +444,10 @@ func discoveryQualityFindings(test discoveredTest) []any { discoveryQualityFinding(test.TestID, "candidate_only", "wrong_boundary", "warning"), } if len(test.CandidateRequirementRefs) == 0 && len(test.OwnerInvariantRefs) == 0 { - findings = append(findings, discoveryQualityFinding(test.TestID, "missing-semantic-anchor", "missing_edge", "warning")) + findings = append(findings, discoveryQualityFinding(test.TestID, "missing-declared-route-anchor", "missing_edge", "warning")) } - if weakOracleSignals(test.OracleSignals) { - findings = append(findings, discoveryQualityFinding(test.TestID, "weak-oracle", "empty_oracle", "warning")) + if !hasDeclaredAssertionSignal(test.OracleSignals) { + findings = append(findings, discoveryQualityFinding(test.TestID, "missing-declared-assertion-signal", "missing_edge", "warning")) } if fragileSelectorSignals(test.SelectorSignals) { findings = append(findings, discoveryQualityFinding(test.TestID, "selector-fragility", "wrong_boundary", "warning")) @@ -466,21 +466,18 @@ func discoveryQualityFinding(testID string, suffix string, class string, severit } } -func weakOracleSignals(signals []string) bool { - if len(signals) == 0 { - return true - } - strong := map[string]struct{}{ +func hasDeclaredAssertionSignal(signals []string) bool { + assertionSignals := map[string]struct{}{ "assertion_present": {}, "expected_exception": {}, "status_or_exit_assertion": {}, } for _, signal := range signals { - if _, ok := strong[signal]; ok { - return false + if _, ok := assertionSignals[signal]; ok { + return true } } - return true + return false } func fragileSelectorSignals(signals []string) bool { diff --git a/internal/command/testevidenceinventory/guidance.go b/internal/command/testevidenceinventory/guidance.go index c02c69e..e1c4224 100644 --- a/internal/command/testevidenceinventory/guidance.go +++ b/internal/command/testevidenceinventory/guidance.go @@ -47,19 +47,19 @@ func instructionForDiagnostic(diagnostic string, classificationID string) string case "declared_duplicate_falsifier": return "Keep one active falsifier per equivalence class, or use same-equivalence supersession to retire the older falsifier explicitly." case "invalid_falsifier_supersession": - return "Point supersedes only at an existing same-equivalence falsifier and cite a dominance proof ref, or remove the supersession claim." + return "Point supersedes only at an existing same-equivalence falsifier and cite a caller-owned supersession declaration ref, or remove the supersession declaration." case "missing_executable_command_ref": - return "Add the executable commandRef that runs this semantic falsifier, or reclassify the entry as nonsemantic evidence." - case "missing_semantic_anchor": + return "Add the caller-declared executable commandRef for this route, or reclassify the entry as nonsemantic evidence." + case "missing_declared_route_anchor": return "Bind this test to at least one requirementRef or ownerInvariantRef, or reclassify it as helper or route-only evidence." case "routing_smoke_only": - return "Treat this entry as wiring-only evidence; add a separate semantic_falsifier entry if a requirement or command must be satisfied." + return "Treat this entry as wiring-only evidence; add a declared semantic-falsifier route when mapping is required, then bind separately admitted execution and policy evidence before assurance closure." case "proof_route_candidate": - return "Review the projected proof route, bind it to an owner-authored executable oracle, and only then materialize a semantic_falsifier entry." - case "weak_or_empty_oracle": - return "Declare a falsifier and a non-empty assertion oracle that distinguishes the intended failure from the implementation under test." + return "Review the projected proof route and bind it to an owner-authored executable oracle before materializing a declared semantic-falsifier route; the declaration still does not prove execution." + case "incomplete_declared_oracle_metadata": + return "Declare complete falsifier and oracle metadata that describes the intended distinction; this still does not prove execution or oracle quality." case "wrong_evidence_boundary": - return "Remove semantic anchors from route-only smoke evidence, or replace it with a semantic_falsifier entry that carries a real oracle." + return "Remove requirement or invariant anchors from route-only smoke evidence, or replace it with a declared semantic-falsifier route whose oracle metadata remains explicitly caller-owned." default: return "Inspect this admitted inventory diagnostic and repair the caller-owned test inventory before using it as coverage guidance." } diff --git a/internal/command/testevidenceinventory/testevidenceinventory.go b/internal/command/testevidenceinventory/testevidenceinventory.go index 302eff5..feaeae5 100644 --- a/internal/command/testevidenceinventory/testevidenceinventory.go +++ b/internal/command/testevidenceinventory/testevidenceinventory.go @@ -16,15 +16,22 @@ const directAuthority = "caller_owned_inventory" const sourceSetAuthority = "caller_owned_inventory_source_set" const wrappedInventorySchema = "proofkit.requirement-test-inventory.v1" +const ( + EvidenceClassDeclaredContractAdmissionRoute = "declared_contract_admission_route" + EvidenceClassDeclaredPropertyOrFuzzRoute = "declared_property_or_fuzz_route" + EvidenceClassDeclaredSemanticFalsifierRoute = "declared_semantic_falsifier_route" + EvidenceClassProofRouteCandidate = "proof_route_candidate" +) + var evidenceClassSet = map[string]struct{}{ - "benchmark": {}, - "contract_admission": {}, - "governance_or_release": {}, - "helper_or_testkit": {}, - "property_or_fuzz": {}, - "proof_route_candidate": {}, - "routing_smoke_nonclaim": {}, - "semantic_falsifier": {}, + "benchmark": {}, + EvidenceClassDeclaredContractAdmissionRoute: {}, + "governance_or_release": {}, + "helper_or_testkit": {}, + EvidenceClassDeclaredPropertyOrFuzzRoute: {}, + EvidenceClassProofRouteCandidate: {}, + "routing_smoke_nonclaim": {}, + EvidenceClassDeclaredSemanticFalsifierRoute: {}, } var qualityFindingClassSet = map[string]struct{}{ @@ -53,8 +60,41 @@ var qualityFindingReviewStateSet = map[string]struct{}{ "confirmed": {}, } +// AdmitEvidenceClass keeps child projections on the inventory-owned vocabulary. +func AdmitEvidenceClass(raw any, context string) (string, error) { + return admit.Enum(raw, evidenceClassSet, context) +} + +// EvidenceClasses returns the closed inventory-owned vocabulary for consumers +// that must prove exhaustive projection coverage. +func EvidenceClasses() []string { + values := make([]string, 0, len(evidenceClassSet)) + for value := range evidenceClassSet { + values = append(values, value) + } + sort.Strings(values) + return values +} + +// AdmitQualityFindingClass keeps projected finding classes on the inventory owner. +func AdmitQualityFindingClass(raw any, context string) (string, error) { + return admit.Enum(raw, qualityFindingClassSet, context) +} + +// AdmitQualityFindingSeverity keeps projected finding severity on the inventory owner. +func AdmitQualityFindingSeverity(raw any, context string) (string, error) { + return admit.Enum(raw, qualityFindingSeveritySet, context) +} + +// AdmitQualityFindingReviewState keeps projected owner-review state on the inventory owner. +func AdmitQualityFindingReviewState(raw any, context string) (string, error) { + return admit.Enum(raw, qualityFindingReviewStateSet, context) +} + var defaultNonClaims = []string{ "Test evidence inventory reports do not execute native tests.", + "Test evidence inventory reports do not resolve selectors or review caller-authored oracle quality.", + "Test evidence inventory reports do not verify that a caller-declared falsifier supersession dominates the superseded falsifier.", "Test evidence inventory reports do not prove repository inventory completeness.", "Test evidence inventory reports do not authenticate runner output or receipt producers.", "Test evidence inventory reports do not approve merge, release, rollout, or repository policy.", @@ -117,7 +157,7 @@ type Falsifier struct { FalsifierID string NegativeCaseID string Supersedes []string - SupersessionProofRef string + SupersessionDeclarationRef string WrongImplementationClassID string } @@ -199,19 +239,19 @@ func Evaluate(raw any) (Result, error) { ReportID: inventory.InventoryID, State: state, Summary: map[string]any{ - "agentActionCount": len(actionPlan), - "entryCount": len(inventory.Entries), - "failureCount": len(failures), - "inputPathCount": len(inventory.InputPaths), - "qualityFindingFailureCount": countPrefix(failures, "quality_finding:"), - "qualityFindingWarningCount": countPrefix(warnings, "quality_finding:"), - "routeOnlyNonClaimCount": countEvidenceClass(inventory.Entries, "routing_smoke_nonclaim"), - "proofRouteCandidateCount": countEvidenceClass(inventory.Entries, "proof_route_candidate"), - "semanticFalsifierCount": countEvidenceClass(inventory.Entries, "semantic_falsifier"), - "sourceCount": inventory.SourceCount, - "weakOracleFailureCount": countContains(failures, "weak_or_empty_oracle"), - "wrongBoundaryFailureCount": countContains(failures, "wrong_evidence_boundary"), - "warningCount": len(warnings), + "agentActionCount": len(actionPlan), + "entryCount": len(inventory.Entries), + "failureCount": len(failures), + "inputPathCount": len(inventory.InputPaths), + "qualityFindingFailureCount": countPrefix(failures, "quality_finding:"), + "qualityFindingWarningCount": countPrefix(warnings, "quality_finding:"), + "routeOnlyNonClaimCount": countEvidenceClass(inventory.Entries, "routing_smoke_nonclaim"), + "proofRouteCandidateCount": countEvidenceClass(inventory.Entries, "proof_route_candidate"), + "declaredSemanticFalsifierRouteCount": countEvidenceClass(inventory.Entries, EvidenceClassDeclaredSemanticFalsifierRoute), + "sourceCount": inventory.SourceCount, + "incompleteDeclaredOracleMetadataFailureCount": countContains(failures, "incomplete_declared_oracle_metadata"), + "wrongBoundaryFailureCount": countContains(failures, "wrong_evidence_boundary"), + "warningCount": len(warnings), }, Diagnostics: []report.Diagnostic{ {Key: "agentActionPlan", Value: mapsToAny(actionPlan)}, @@ -320,8 +360,8 @@ func falsifierToAny(falsifier *Falsifier) any { "supersedes": admit.StringSliceToAny(falsifier.Supersedes), "wrongImplementationClassId": falsifier.WrongImplementationClassID, } - if falsifier.SupersessionProofRef != "" { - record["supersessionProofRef"] = falsifier.SupersessionProofRef + if falsifier.SupersessionDeclarationRef != "" { + record["supersessionDeclarationRef"] = falsifier.SupersessionDeclarationRef } return record } @@ -462,7 +502,7 @@ func admitEntry(raw any) (Entry, error) { if err != nil { return Entry{}, err } - evidenceClass, err := admit.Enum(record["evidenceClass"], evidenceClassSet, fmt.Sprintf("test evidence inventory %s evidenceClass", testID)) + evidenceClass, err := AdmitEvidenceClass(record["evidenceClass"], fmt.Sprintf("test evidence inventory %s evidenceClass", testID)) if err != nil { return Entry{}, err } @@ -514,7 +554,7 @@ func admitFalsifier(raw any, testID string) (*Falsifier, error) { if !ok { return nil, fmt.Errorf("test evidence inventory %s falsifier must be an object or null", testID) } - if err := admit.KnownKeys(record, []string{"dominanceGroup", "falsifierId", "negativeCaseId", "supersedes", "supersessionProofRef", "wrongImplementationClassId"}, "test evidence inventory falsifier"); err != nil { + if err := admit.KnownKeys(record, []string{"dominanceGroup", "falsifierId", "negativeCaseId", "supersedes", "supersessionDeclarationRef", "wrongImplementationClassId"}, "test evidence inventory falsifier"); err != nil { return nil, err } falsifierID, err := admit.RuleID(record["falsifierId"], fmt.Sprintf("test evidence inventory %s falsifierId", testID)) @@ -537,16 +577,16 @@ func admitFalsifier(raw any, testID string) (*Falsifier, error) { if err != nil { return nil, err } - supersessionProofRef := "" - if record["supersessionProofRef"] != nil { - supersessionProofRef, err = admit.RuleID(record["supersessionProofRef"], fmt.Sprintf("test evidence inventory %s supersessionProofRef", testID)) + supersessionDeclarationRef := "" + if record["supersessionDeclarationRef"] != nil { + supersessionDeclarationRef, err = admit.RuleID(record["supersessionDeclarationRef"], fmt.Sprintf("test evidence inventory %s supersessionDeclarationRef", testID)) if err != nil { return nil, err } } return &Falsifier{ DominanceGroup: dominanceGroup, FalsifierID: falsifierID, - NegativeCaseID: negativeCaseID, Supersedes: supersedes, SupersessionProofRef: supersessionProofRef, + NegativeCaseID: negativeCaseID, Supersedes: supersedes, SupersessionDeclarationRef: supersessionDeclarationRef, WrongImplementationClassID: wrongImplementationClassID, }, nil } @@ -622,15 +662,15 @@ func admitQualityFinding(raw any, testID string, index int) (QualityFinding, err if err != nil { return QualityFinding{}, err } - class, err := admit.Enum(record["class"], qualityFindingClassSet, context+" class") + class, err := AdmitQualityFindingClass(record["class"], context+" class") if err != nil { return QualityFinding{}, err } - severity, err := admit.Enum(record["severity"], qualityFindingSeveritySet, context+" severity") + severity, err := AdmitQualityFindingSeverity(record["severity"], context+" severity") if err != nil { return QualityFinding{}, err } - reviewState, err := admit.Enum(record["ownerReviewState"], qualityFindingReviewStateSet, context+" ownerReviewState") + reviewState, err := AdmitQualityFindingReviewState(record["ownerReviewState"], context+" ownerReviewState") if err != nil { return QualityFinding{}, err } @@ -653,14 +693,14 @@ func classify(inventory Inventory) ([]string, []string) { warnings := []string{} falsifiers := inventoryFalsifiers(inventory.Entries) for _, entry := range inventory.Entries { - if requiresSemanticAnchor(entry.EvidenceClass) && len(entry.RequirementRefs) == 0 && len(entry.OwnerInvariantRefs) == 0 { - failures = append(failures, fmt.Sprintf("missing_semantic_anchor:%s", entry.TestID)) + if requiresDeclaredRouteAnchor(entry.EvidenceClass) && len(entry.RequirementRefs) == 0 && len(entry.OwnerInvariantRefs) == 0 { + failures = append(failures, fmt.Sprintf("missing_declared_route_anchor:%s", entry.TestID)) } if requiresExecutableCommandRefs(entry.EvidenceClass) && len(entry.CommandRefs) == 0 { failures = append(failures, fmt.Sprintf("missing_executable_command_ref:%s", entry.TestID)) } - if requiresStrongOracle(entry.EvidenceClass) && !hasStrongOracle(entry) { - failures = append(failures, fmt.Sprintf("weak_or_empty_oracle:%s", entry.TestID)) + if requiresDeclaredOracleMetadata(entry.EvidenceClass) && !hasDeclaredOracleMetadata(entry) { + failures = append(failures, fmt.Sprintf("incomplete_declared_oracle_metadata:%s", entry.TestID)) } if entry.EvidenceClass == "routing_smoke_nonclaim" { if len(entry.RequirementRefs) > 0 || len(entry.OwnerInvariantRefs) > 0 { @@ -686,12 +726,12 @@ func classify(inventory Inventory) ([]string, []string) { } type falsifierLedgerEntry struct { - FalsifierID string - Key string - ProofRefs []string - Supersedes []string - ProofRef string - TestID string + DeclarationRef string + DeclarationRefs []string + FalsifierID string + Key string + Supersedes []string + TestID string } func inventoryFalsifiers(entries []Entry) []falsifierLedgerEntry { @@ -701,12 +741,12 @@ func inventoryFalsifiers(entries []Entry) []falsifierLedgerEntry { continue } out = append(out, falsifierLedgerEntry{ - FalsifierID: entry.Falsifier.FalsifierID, - Key: falsifierEquivalenceKey(entry), - ProofRefs: entry.OwnerInvariantRefs, - ProofRef: entry.Falsifier.SupersessionProofRef, - Supersedes: entry.Falsifier.Supersedes, - TestID: entry.TestID, + DeclarationRef: entry.Falsifier.SupersessionDeclarationRef, + DeclarationRefs: entry.OwnerInvariantRefs, + FalsifierID: entry.Falsifier.FalsifierID, + Key: falsifierEquivalenceKey(entry), + Supersedes: entry.Falsifier.Supersedes, + TestID: entry.TestID, }) } return out @@ -728,13 +768,13 @@ func classifyFalsifierSupersession(entries []falsifierLedgerEntry) []string { supersededByKey := map[string]map[string]struct{}{} for _, entry := range entries { for _, supersededID := range entry.Supersedes { - if entry.ProofRef == "" { - failures = append(failures, fmt.Sprintf("invalid_falsifier_supersession:%s:missing_dominance_proof:%s", entry.TestID, supersededID)) + if entry.DeclarationRef == "" { + failures = append(failures, fmt.Sprintf("invalid_falsifier_supersession:%s:missing_supersession_declaration:%s", entry.TestID, supersededID)) invalidFalsifierIDs[entry.FalsifierID] = struct{}{} continue } - if !containsString(entry.ProofRefs, entry.ProofRef) { - failures = append(failures, fmt.Sprintf("invalid_falsifier_supersession:%s:unowned_dominance_proof:%s", entry.TestID, entry.ProofRef)) + if !containsString(entry.DeclarationRefs, entry.DeclarationRef) { + failures = append(failures, fmt.Sprintf("invalid_falsifier_supersession:%s:unowned_supersession_declaration:%s", entry.TestID, entry.DeclarationRef)) invalidFalsifierIDs[entry.FalsifierID] = struct{}{} continue } @@ -778,18 +818,18 @@ func classifyFalsifierSupersession(entries []falsifierLedgerEntry) []string { return failures } -func requiresSemanticAnchor(evidenceClass string) bool { +func requiresDeclaredRouteAnchor(evidenceClass string) bool { switch evidenceClass { - case "semantic_falsifier", "proof_route_candidate", "contract_admission", "property_or_fuzz", "governance_or_release", "benchmark": + case EvidenceClassDeclaredSemanticFalsifierRoute, EvidenceClassProofRouteCandidate, EvidenceClassDeclaredContractAdmissionRoute, EvidenceClassDeclaredPropertyOrFuzzRoute, "governance_or_release", "benchmark": return true default: return false } } -func requiresStrongOracle(evidenceClass string) bool { +func requiresDeclaredOracleMetadata(evidenceClass string) bool { switch evidenceClass { - case "semantic_falsifier", "contract_admission", "property_or_fuzz": + case EvidenceClassDeclaredSemanticFalsifierRoute, EvidenceClassDeclaredContractAdmissionRoute, EvidenceClassDeclaredPropertyOrFuzzRoute: return true default: return false @@ -797,26 +837,43 @@ func requiresStrongOracle(evidenceClass string) bool { } func requiresExecutableCommandRefs(evidenceClass string) bool { - return evidenceClass == "semantic_falsifier" || evidenceClass == "proof_route_candidate" + return evidenceClass == EvidenceClassDeclaredSemanticFalsifierRoute || evidenceClass == EvidenceClassProofRouteCandidate } -func hasStrongOracle(entry Entry) bool { +func hasDeclaredOracleMetadata(entry Entry) bool { return entry.Falsifier != nil && + entry.Falsifier.NegativeCaseID != "" && + entry.Falsifier.WrongImplementationClassID != "" && entry.Oracle != nil && + entry.Oracle.OracleKind != "" && strings.TrimSpace(entry.Oracle.AssertionSummary) != "" && strings.TrimSpace(entry.Oracle.ExpectedPublicOutcome) != "" } +// ClassifyProjectedEntries replays inventory-owned cross-entry and quality +// semantics for a structurally complete child projection. It does not claim +// source authenticity, selector execution, or repository completeness. +func ClassifyProjectedEntries(entries []Entry) ([]string, []string, error) { + if err := assertUnique(entryIDs(entries), "projected test evidence inventory testIds"); err != nil { + return nil, nil, err + } + if err := assertUnique(falsifierIDs(entries), "projected test evidence inventory falsifierIds"); err != nil { + return nil, nil, err + } + failures, warnings := classify(Inventory{Entries: entries}) + return failures, warnings, nil +} + func ruleResults(failures []string, warnings []string) []report.RuleResult { return []report.RuleResult{ rule("test_inventory.input_admitted", "passed", "Test inventory input used strict known-key admission."), - ruleStatus("test_inventory.semantic_entries_have_anchors", !hasPrefix(failures, "missing_semantic_anchor"), "Semantic test inventory entries must cite requirement refs or stable owner invariant refs."), - ruleStatus("test_inventory.route_bearing_entries_have_commands", !hasPrefix(failures, "missing_executable_command_ref"), "Semantic falsifier and proof-route candidate entries must cite executable command refs."), - ruleStatus("test_inventory.strong_oracles", !hasPrefix(failures, "weak_or_empty_oracle"), "Semantic falsifier entries must declare a falsifier, assertion oracle, and expected public outcome."), - ruleStatus("test_inventory.no_duplicate_falsifiers", !hasPrefix(failures, "declared_duplicate_falsifier") && !hasPrefix(failures, "invalid_falsifier_supersession"), "Duplicate falsifier equivalence keys require explicit same-equivalence supersession with dominance proof."), - ruleStatus("test_inventory.route_only_boundaries", !hasPrefix(failures, "wrong_evidence_boundary"), "Route-only smoke evidence must remain a non-claim and cannot cite semantic requirement anchors."), + ruleStatus("test_inventory.declared_routes_have_anchors", !hasPrefix(failures, "missing_declared_route_anchor"), "Declared test routes must cite requirement refs or stable owner invariant refs."), + ruleStatus("test_inventory.route_bearing_entries_have_commands", !hasPrefix(failures, "missing_executable_command_ref"), "Declared semantic-falsifier routes and proof-route candidates must cite executable command refs."), + ruleStatus("test_inventory.declared_oracle_metadata", !hasPrefix(failures, "incomplete_declared_oracle_metadata"), "Declared semantic, contract-admission, and property/fuzz routes must include caller-authored falsifier and oracle metadata without claiming native oracle review."), + ruleStatus("test_inventory.declared_falsifier_supersession_consistency", !hasPrefix(failures, "declared_duplicate_falsifier") && !hasPrefix(failures, "invalid_falsifier_supersession"), "Duplicate falsifier equivalence keys require one caller-declared same-equivalence supersession ref included in ownerInvariantRefs; this does not prove dominance."), + ruleStatus("test_inventory.route_only_boundaries", !hasPrefix(failures, "wrong_evidence_boundary"), "Route-only smoke evidence must remain a non-claim and cannot cite requirement or owner-invariant anchors."), rule("test_inventory.route_only_warnings_are_advisory", "passed", "Route-only entries remain admitted advisory non-claim warnings and do not fail the report."), - rule("test_inventory.proof_route_candidates_are_advisory", "passed", "Proof-route candidates remain advisory until a consumer owner supplies executable semantic evidence."), + rule("test_inventory.proof_route_candidates_are_advisory", "passed", "Proof-route candidates remain advisory until separately admitted execution and policy evidence satisfies a consumer-owned assurance rule."), } } @@ -856,8 +913,8 @@ func diagnosticClassID(diagnostic string) string { return "declared_duplicate_falsifier" case strings.HasPrefix(diagnostic, "invalid_falsifier_supersession:"): return "invalid_falsifier_supersession" - case strings.HasPrefix(diagnostic, "missing_semantic_anchor:"): - return "missing_semantic_anchor" + case strings.HasPrefix(diagnostic, "missing_declared_route_anchor:"): + return "missing_declared_route_anchor" case strings.HasPrefix(diagnostic, "missing_executable_command_ref:"): return "missing_executable_command_ref" case strings.HasPrefix(diagnostic, "quality_finding:"): @@ -871,8 +928,8 @@ func diagnosticClassID(diagnostic string) string { return "routing_smoke_only" case strings.HasPrefix(diagnostic, "selector_fragility:"): return "selector_fragility" - case strings.HasPrefix(diagnostic, "weak_or_empty_oracle:"): - return "weak_or_empty_oracle" + case strings.HasPrefix(diagnostic, "incomplete_declared_oracle_metadata:"): + return "incomplete_declared_oracle_metadata" case strings.HasPrefix(diagnostic, "wrong_evidence_boundary:"): return "wrong_evidence_boundary" } diff --git a/internal/command/testevidenceinventory/testevidenceinventory_test.go b/internal/command/testevidenceinventory/testevidenceinventory_test.go index 4917f4c..536a2bb 100644 --- a/internal/command/testevidenceinventory/testevidenceinventory_test.go +++ b/internal/command/testevidenceinventory/testevidenceinventory_test.go @@ -12,7 +12,7 @@ import ( "github.com/research-engineering/agentic-proofkit/internal/testsupport/commandcoverage" ) -func TestBuildAdmitsSemanticFalsifierInventory(t *testing.T) { +func TestBuildAdmitsDeclaredSemanticFalsifierRouteInventory(t *testing.T) { record, exitCode, err := Build(validInventory(t)) if err != nil { t.Fatalf("Build() error = %v", err) @@ -23,8 +23,8 @@ func TestBuildAdmitsSemanticFalsifierInventory(t *testing.T) { if record.ReportKind != ReportKind || record.ReportID != "proofkit.test.inventory" { t.Fatalf("unexpected report identity: %#v", record.JSONValue()) } - if record.Summary["semanticFalsifierCount"] != 1 { - t.Fatalf("semanticFalsifierCount=%v", record.Summary["semanticFalsifierCount"]) + if record.Summary["declaredSemanticFalsifierRouteCount"] != 1 { + t.Fatalf("declaredSemanticFalsifierRouteCount=%v", record.Summary["declaredSemanticFalsifierRouteCount"]) } if record.Summary["agentActionCount"] != 0 { t.Fatalf("clean inventory emitted agent actions: %#v", record.JSONValue()) @@ -51,7 +51,7 @@ func TestBuildRejectsUnanchoredProofRouteCandidate(t *testing.T) { t.Fatalf("Build() exit=%d state=%s, want failed candidate admission", exitCode, record.State) } encoded, _ := json.Marshal(record.JSONValue()) - if !strings.Contains(string(encoded), "missing_semantic_anchor") || !strings.Contains(string(encoded), "missing_executable_command_ref") { + if !strings.Contains(string(encoded), "missing_declared_route_anchor") || !strings.Contains(string(encoded), "missing_executable_command_ref") { t.Fatalf("candidate failures=%s", encoded) } } @@ -119,6 +119,28 @@ func TestBuildDiscoveryDraftEmitsCandidateOnlyInventory(t *testing.T) { } } +func TestBuildDiscoveryDraftNamesMissingDeclaredRouteAnchor(t *testing.T) { + input := validDiscoveryDraft() + test := firstDiscoveryTest(input) + test["candidateRequirementRefs"] = []any{} + test["ownerInvariantRefs"] = []any{} + record, _, err := BuildDiscoveryDraft(input) + if err != nil { + t.Fatalf("BuildDiscoveryDraft() error = %v", err) + } + candidate := diagnosticValue(record.JSONValue(), "candidateInventory").(map[string]any) + entry := candidate["entries"].([]any)[0].(map[string]any) + findings := entry["qualityFindings"].([]any) + findingIDs := []string{} + for _, rawFinding := range findings { + findingIDs = append(findingIDs, rawFinding.(map[string]any)["findingId"].(string)) + } + joined := strings.Join(findingIDs, "\n") + if !strings.Contains(joined, ".missing-declared-route-anchor") || strings.Contains(joined, ".missing-semantic-anchor") { + t.Fatalf("candidate findings retain obsolete semantic-proof vocabulary: %s", joined) + } +} + func TestBuildDiscoveryDraftDefaultsOptionalNonClaims(t *testing.T) { input := validDiscoveryDraft() delete(input, "nonClaims") @@ -139,7 +161,7 @@ func TestBuildDiscoveryDraftDefaultsOptionalNonClaims(t *testing.T) { } } -func TestBuildDiscoveryDraftClassifiesWeakOracleSignals(t *testing.T) { +func TestBuildDiscoveryDraftClassifiesMissingDeclaredAssertionSignals(t *testing.T) { input := validDiscoveryDraft() strong := firstDiscoveryTest(input) weak := cloneMap(strong) @@ -158,26 +180,26 @@ func TestBuildDiscoveryDraftClassifiesWeakOracleSignals(t *testing.T) { t.Fatalf("BuildDiscoveryDraft() state=%s exit=%d report=%#v", record.State, exitCode, record.JSONValue()) } value := record.JSONValue() - if value["summary"].(map[string]any)["weakOracleWarningCount"] != 1 { - t.Fatalf("weakOracleWarningCount=%#v want 1", value["summary"].(map[string]any)["weakOracleWarningCount"]) + if value["summary"].(map[string]any)["missingDeclaredAssertionSignalWarningCount"] != 1 { + t.Fatalf("missingDeclaredAssertionSignalWarningCount=%#v want 1", value["summary"].(map[string]any)["missingDeclaredAssertionSignalWarningCount"]) } warnings := strings.Join(stringDiagnostics(value, "warnings"), "\n") - if !strings.Contains(warnings, "weak_or_empty_oracle:test.discovery.weak_oracle") { - t.Fatalf("warnings missing weak oracle diagnostic: %s", warnings) + if !strings.Contains(warnings, "missing_declared_assertion_signal:test.discovery.weak_oracle") { + t.Fatalf("warnings missing declared assertion-signal diagnostic: %s", warnings) } - if strings.Contains(warnings, "weak_or_empty_oracle:test.discovery.semantic") { - t.Fatalf("strong oracle row emitted weak oracle diagnostic: %s", warnings) + if strings.Contains(warnings, "missing_declared_assertion_signal:test.discovery.semantic") { + t.Fatalf("assertion-bearing row emitted missing signal diagnostic: %s", warnings) } actions := diagnosticValue(value, "agentActionPlan").([]any) foundAction := false for _, raw := range actions { action := raw.(map[string]any) - if action["testId"] == "test.discovery.weak_oracle" && action["type"] == "weak_or_empty_oracle" { + if action["testId"] == "test.discovery.weak_oracle" && action["type"] == "missing_declared_assertion_signal" { foundAction = true } } if !foundAction { - t.Fatalf("agentActionPlan missing weak_or_empty_oracle action: %#v", actions) + t.Fatalf("agentActionPlan missing missing_declared_assertion_signal action: %#v", actions) } candidate := diagnosticValue(value, "candidateInventory").(map[string]any) for _, rawEntry := range candidate["entries"].([]any) { @@ -188,11 +210,11 @@ func TestBuildDiscoveryDraftClassifiesWeakOracleSignals(t *testing.T) { findings := entry["qualityFindings"].([]any) for _, rawFinding := range findings { finding := rawFinding.(map[string]any) - if finding["class"] == "empty_oracle" { + if finding["class"] == "missing_edge" && finding["findingId"] == "finding.test.discovery.weak_oracle.missing-declared-assertion-signal" { return } } - t.Fatalf("weak oracle candidate entry missing empty_oracle quality finding: %#v", entry) + t.Fatalf("candidate entry missing declared assertion-signal finding: %#v", entry) } t.Fatalf("candidate inventory missing weak oracle row: %#v", candidate) } @@ -260,8 +282,8 @@ func TestBuildDiscoveryDraftRejectsUnsafeAndContradictoryFacts(t *testing.T) { } } -func TestBuildRejectsWeakOracleAndDuplicateFalsifier(t *testing.T) { - commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.081636031828961563284031836832627984765244046916564856218652240189648812663526") +func TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier(t *testing.T) { + commandcoverage.SemanticRoute(t, "proofkit.command_coverage.source_oracle.v1.107074977993094420943019248218359465183764901197584019504412197592065422528956") input := validInventory(t) entries := input.(map[string]any)["entries"].([]any) first := cloneMap(entries[0].(map[string]any)) @@ -295,17 +317,17 @@ func TestBuildRejectsWeakOracleAndDuplicateFalsifier(t *testing.T) { t.Fatalf("Build() accepted weak inventory: %#v", record.JSONValue()) } failures := strings.Join(stringDiagnostics(record.JSONValue(), "failures"), "\n") - for _, want := range []string{"declared_duplicate_falsifier:", "weak_or_empty_oracle:test.inventory.weak"} { + for _, want := range []string{"declared_duplicate_falsifier:", "incomplete_declared_oracle_metadata:test.inventory.weak"} { if !strings.Contains(failures, want) { t.Fatalf("failures missing %q: %s", want, failures) } } requireDiagnosticClassifications(t, record.JSONValue(), "failureClassifications", "failure", map[string]string{ "declared_duplicate_falsifier:test.inventory.duplicate:test.inventory.semantic": "declared_duplicate_falsifier", - "weak_or_empty_oracle:test.inventory.weak": "weak_or_empty_oracle", + "incomplete_declared_oracle_metadata:test.inventory.weak": "incomplete_declared_oracle_metadata", }) requireAgentAction(t, record.JSONValue(), "declared_duplicate_falsifier", "failure") - requireAgentAction(t, record.JSONValue(), "weak_or_empty_oracle", "failure") + requireAgentAction(t, record.JSONValue(), "incomplete_declared_oracle_metadata", "failure") } func TestBuildRejectsDuplicateFalsifierIDAcrossDifferentEquivalenceKeys(t *testing.T) { @@ -335,7 +357,7 @@ func TestBuildRejectsArbitraryFalsifierSupersedes(t *testing.T) { replacement["testId"] = "test.inventory.invalid_supersedes" replacement["falsifier"].(map[string]any)["falsifierId"] = "falsifier.inventory.invalid_supersedes" replacement["falsifier"].(map[string]any)["supersedes"] = []any{"falsifier.inventory.unrelated"} - replacement["falsifier"].(map[string]any)["supersessionProofRef"] = "proof.inventory.invalid_supersedes" + replacement["falsifier"].(map[string]any)["supersessionDeclarationRef"] = "proof.inventory.invalid_supersedes" replacement["ownerInvariantRefs"] = []any{"proof.inventory.invalid_supersedes"} input.(map[string]any)["entries"] = append(entries, replacement) @@ -355,7 +377,7 @@ func TestBuildRejectsArbitraryFalsifierSupersedes(t *testing.T) { }) } -func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithoutDominanceProof(t *testing.T) { +func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithoutDeclaration(t *testing.T) { input := validInventory(t) entries := input.(map[string]any)["entries"].([]any) replacement := cloneMap(entries[0].(map[string]any)) @@ -370,15 +392,15 @@ func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithoutDominanceProof(t t.Fatalf("Build() error = %v", err) } if exitCode == 0 || record.State != "failed" { - t.Fatalf("Build() accepted supersession without dominance proof: %#v", record.JSONValue()) + t.Fatalf("Build() accepted supersession without declaration ref: %#v", record.JSONValue()) } failures := strings.Join(stringDiagnostics(record.JSONValue(), "failures"), "\n") - if !strings.Contains(failures, "invalid_falsifier_supersession:test.inventory.superseding:missing_dominance_proof:falsifier.inventory.semantic") { - t.Fatalf("failures missing dominance proof diagnostic: %s", failures) + if !strings.Contains(failures, "invalid_falsifier_supersession:test.inventory.superseding:missing_supersession_declaration:falsifier.inventory.semantic") { + t.Fatalf("failures missing supersession declaration diagnostic: %s", failures) } } -func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithUnownedDominanceProof(t *testing.T) { +func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithUnownedDeclaration(t *testing.T) { input := validInventory(t) entries := input.(map[string]any)["entries"].([]any) replacement := cloneMap(entries[0].(map[string]any)) @@ -386,7 +408,7 @@ func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithUnownedDominancePro replacement["testId"] = "test.inventory.unowned_superseding" replacement["falsifier"].(map[string]any)["falsifierId"] = "falsifier.inventory.unowned_superseding" replacement["falsifier"].(map[string]any)["supersedes"] = []any{"falsifier.inventory.semantic"} - replacement["falsifier"].(map[string]any)["supersessionProofRef"] = "proof.inventory.unowned_superseding" + replacement["falsifier"].(map[string]any)["supersessionDeclarationRef"] = "proof.inventory.unowned_superseding" input.(map[string]any)["entries"] = append(entries, replacement) record, exitCode, err := Build(input) @@ -394,15 +416,15 @@ func TestBuildRejectsSameEquivalenceFalsifierSupersessionWithUnownedDominancePro t.Fatalf("Build() error = %v", err) } if exitCode == 0 || record.State != "failed" { - t.Fatalf("Build() accepted supersession with unowned dominance proof: %#v", record.JSONValue()) + t.Fatalf("Build() accepted supersession with unowned declaration ref: %#v", record.JSONValue()) } failures := strings.Join(stringDiagnostics(record.JSONValue(), "failures"), "\n") - if !strings.Contains(failures, "invalid_falsifier_supersession:test.inventory.unowned_superseding:unowned_dominance_proof:proof.inventory.unowned_superseding") { - t.Fatalf("failures missing unowned dominance proof diagnostic: %s", failures) + if !strings.Contains(failures, "invalid_falsifier_supersession:test.inventory.unowned_superseding:unowned_supersession_declaration:proof.inventory.unowned_superseding") { + t.Fatalf("failures missing unowned supersession declaration diagnostic: %s", failures) } } -func TestBuildAdmitsSameEquivalenceFalsifierSupersessionWithDominanceProof(t *testing.T) { +func TestBuildAdmitsSameEquivalenceFalsifierSupersessionWithOwnedDeclaration(t *testing.T) { input := validInventory(t) entries := input.(map[string]any)["entries"].([]any) replacement := cloneMap(entries[0].(map[string]any)) @@ -410,7 +432,7 @@ func TestBuildAdmitsSameEquivalenceFalsifierSupersessionWithDominanceProof(t *te replacement["testId"] = "test.inventory.superseding" replacement["falsifier"].(map[string]any)["falsifierId"] = "falsifier.inventory.superseding" replacement["falsifier"].(map[string]any)["supersedes"] = []any{"falsifier.inventory.semantic"} - replacement["falsifier"].(map[string]any)["supersessionProofRef"] = "proof.inventory.superseding" + replacement["falsifier"].(map[string]any)["supersessionDeclarationRef"] = "proof.inventory.superseding" replacement["ownerInvariantRefs"] = []any{"proof.inventory.superseding"} input.(map[string]any)["entries"] = append(entries, replacement) @@ -419,7 +441,7 @@ func TestBuildAdmitsSameEquivalenceFalsifierSupersessionWithDominanceProof(t *te t.Fatalf("Build() error = %v", err) } if exitCode != 0 || record.State != "passed" { - t.Fatalf("Build() rejected proven same-equivalence supersession: %#v", record.JSONValue()) + t.Fatalf("Build() rejected declared same-equivalence supersession: %#v", record.JSONValue()) } } @@ -431,7 +453,7 @@ func TestBuildAdmitsSupersessionIndependentOfTestIDSortOrder(t *testing.T) { replacement["testId"] = "test.inventory.aaa_superseding" replacement["falsifier"].(map[string]any)["falsifierId"] = "falsifier.inventory.aaa_superseding" replacement["falsifier"].(map[string]any)["supersedes"] = []any{"falsifier.inventory.semantic"} - replacement["falsifier"].(map[string]any)["supersessionProofRef"] = "proof.inventory.aaa_superseding" + replacement["falsifier"].(map[string]any)["supersessionDeclarationRef"] = "proof.inventory.aaa_superseding" replacement["ownerInvariantRefs"] = []any{"proof.inventory.aaa_superseding"} input.(map[string]any)["entries"] = append(entries, replacement) @@ -444,7 +466,7 @@ func TestBuildAdmitsSupersessionIndependentOfTestIDSortOrder(t *testing.T) { } } -func TestBuildRejectsSemanticFalsifierWithoutCommandRefs(t *testing.T) { +func TestBuildRejectsDeclaredSemanticFalsifierRouteWithoutCommandRefs(t *testing.T) { input := validInventory(t) entry := input.(map[string]any)["entries"].([]any)[0].(map[string]any) entry["commandRefs"] = []any{} @@ -465,7 +487,7 @@ func TestBuildRejectsSemanticFalsifierWithoutCommandRefs(t *testing.T) { }) } -func TestBuildRejectsSemanticFalsifierWithoutExpectedPublicOutcome(t *testing.T) { +func TestBuildRejectsDeclaredSemanticFalsifierRouteWithoutExpectedPublicOutcome(t *testing.T) { input := validInventory(t) entry := input.(map[string]any)["entries"].([]any)[0].(map[string]any) entry["oracle"].(map[string]any)["expectedPublicOutcome"] = "" @@ -478,7 +500,7 @@ func TestBuildRejectsSemanticFalsifierWithoutExpectedPublicOutcome(t *testing.T) t.Fatalf("Build() accepted semantic falsifier without expected public outcome: %#v", record.JSONValue()) } failures := strings.Join(stringDiagnostics(record.JSONValue(), "failures"), "\n") - if !strings.Contains(failures, "weak_or_empty_oracle:test.inventory.semantic") { + if !strings.Contains(failures, "incomplete_declared_oracle_metadata:test.inventory.semantic") { t.Fatalf("failures missing weak oracle diagnostic: %s", failures) } } @@ -623,7 +645,7 @@ func TestBuildRejectsUnknownQualityFindingClass(t *testing.T) { } } -func TestBuildRejectsSemanticEntryWithoutStableAnchor(t *testing.T) { +func TestBuildRejectsDeclaredRouteWithoutStableAnchor(t *testing.T) { input := validInventory(t) entry := input.(map[string]any)["entries"].([]any)[0].(map[string]any) entry["requirementRefs"] = []any{} @@ -636,7 +658,7 @@ func TestBuildRejectsSemanticEntryWithoutStableAnchor(t *testing.T) { if exitCode == 0 || record.State != "failed" { t.Fatalf("Build() accepted anchorless semantic entry: %#v", record.JSONValue()) } - if failures := strings.Join(stringDiagnostics(record.JSONValue(), "failures"), "\n"); !strings.Contains(failures, "missing_semantic_anchor:test.inventory.semantic") { + if failures := strings.Join(stringDiagnostics(record.JSONValue(), "failures"), "\n"); !strings.Contains(failures, "missing_declared_route_anchor:test.inventory.semantic") { t.Fatalf("missing semantic anchor failure not reported: %s", failures) } } @@ -747,8 +769,8 @@ func TestBuildAdmitsWrappedSourceSetInventory(t *testing.T) { if record.Summary["sourceCount"] != 1 || record.Summary["inputPathCount"] != 1 { t.Fatalf("unexpected source summary: %#v", record.Summary) } - if record.Summary["semanticFalsifierCount"] != 1 { - t.Fatalf("semanticFalsifierCount=%v", record.Summary["semanticFalsifierCount"]) + if record.Summary["declaredSemanticFalsifierRouteCount"] != 1 { + t.Fatalf("declaredSemanticFalsifierRouteCount=%v", record.Summary["declaredSemanticFalsifierRouteCount"]) } } @@ -997,7 +1019,7 @@ func validInventory(t *testing.T) any { "selector": "go test ./internal/command/testevidenceinventory -run TestSemantic", "sourcePath": "internal/command/testevidenceinventory/testevidenceinventory_test.go", "ownerId": "proofkit.test", - "evidenceClass": "semantic_falsifier", + "evidenceClass": "declared_semantic_falsifier_route", "requirementRefs": ["REQ-PROOFKIT-TEST-001"], "ownerInvariantRefs": [], "commandRefs": ["proofkit.test.command"], diff --git a/internal/kernel/compactproofcontract/compactproofcontract.go b/internal/kernel/compactproofcontract/compactproofcontract.go index 8e5b617..f38419a 100644 --- a/internal/kernel/compactproofcontract/compactproofcontract.go +++ b/internal/kernel/compactproofcontract/compactproofcontract.go @@ -7,6 +7,7 @@ import ( "strings" "github.com/research-engineering/agentic-proofkit/internal/kernel/admit" + "github.com/research-engineering/agentic-proofkit/internal/kernel/proofvocab" ) const ( @@ -315,10 +316,13 @@ func admitBinding(row []any, bindingColumns map[string]int, witnessColumns map[s if err != nil { return Binding{}, err } - proofContractState, err := admit.RuleID(column(row, bindingColumns, "proof_contract_state"), context+" proof_contract_state") + proofContractState, err := admit.Enum(column(row, bindingColumns, "proof_contract_state"), proofvocab.RequirementProofStateSet(), context+" proof_contract_state") if err != nil { return Binding{}, err } + if proofContractState != "witness_backed" { + return Binding{}, fmt.Errorf("%s proof_contract_state must be witness_backed because a compact binding row contains positive and falsification witnesses", context) + } blockingStatus, err := admit.RuleID(column(row, bindingColumns, "blocking_status"), context+" blocking_status") if err != nil { return Binding{}, err @@ -355,7 +359,7 @@ func admitWitness(raw any, context string, columns map[string]int) (Witness, err if err != nil { return Witness{}, err } - selector, err = admitWitnessSelector(selector, context+" selector") + selector, err = AdmitWitnessSelector(selector, context+" selector") if err != nil { return Witness{}, err } @@ -652,25 +656,37 @@ func resolverNonClaims(nonClaims []string) []any { } func admitScopedScenarioID(value string, surfaceID string, context string) (string, error) { - parts := strings.Split(value, "::") - if len(parts) != 2 || parts[0] == "" || parts[1] == "" { - return "", fmt.Errorf("%s must use surface_id::stable_anchor scenario identity", context) - } - scenarioSurfaceID, err := admit.RuleID(parts[0], context+" surface_id") + scenarioID, scenarioSurfaceID, err := AdmitScenarioID(value, context) if err != nil { return "", err } if scenarioSurfaceID != surfaceID { return "", fmt.Errorf("%s must be scoped under surface_id %s", context, surfaceID) } + return scenarioID, nil +} + +// AdmitScenarioID preserves the compact-contract owner for the encoded +// surface_id::stable_anchor scenario identity used by child projections. +func AdmitScenarioID(value string, context string) (string, string, error) { + parts := strings.Split(value, "::") + if len(parts) != 2 || parts[0] == "" || parts[1] == "" { + return "", "", fmt.Errorf("%s must use surface_id::stable_anchor scenario identity", context) + } + scenarioSurfaceID, err := admit.RuleID(parts[0], context+" surface_id") + if err != nil { + return "", "", err + } anchor, err := admit.RuleID(parts[1], context+" anchor") if err != nil { - return "", err + return "", "", err } - return scenarioSurfaceID + "::" + anchor, nil + return scenarioSurfaceID + "::" + anchor, scenarioSurfaceID, nil } -func admitWitnessSelector(value string, context string) (string, error) { +// AdmitWitnessSelector preserves the compact-contract owner for selector +// identity when child projections re-admit compact witness routes. +func AdmitWitnessSelector(value string, context string) (string, error) { parts := strings.Split(value, "::") if len(parts) != 2 || parts[0] == "" || parts[1] == "" { return "", fmt.Errorf("%s must use repo/path::stable_anchor selector identity", context) diff --git a/internal/kernel/compactproofcontract/compactproofcontract_test.go b/internal/kernel/compactproofcontract/compactproofcontract_test.go index 2eedec8..fd904d8 100644 --- a/internal/kernel/compactproofcontract/compactproofcontract_test.go +++ b/internal/kernel/compactproofcontract/compactproofcontract_test.go @@ -56,6 +56,43 @@ func TestAdmitRejectsUnknownCompactColumn(t *testing.T) { } } +func TestAdmitRejectsUnknownRequirementProofState(t *testing.T) { + input := validCompactContract() + input["bindings"].([]any)[0].([]any)[5] = "verified" + + _, err := Admit(input) + if err == nil || !strings.Contains(err.Error(), "proof_contract_state must be one of") { + t.Fatalf("Admit() error=%v, want closed proof-state vocabulary rejection", err) + } +} + +func TestAdmitRejectsNonWitnessBackedBindingState(t *testing.T) { + for _, state := range []string{"explicitly_deferred", "not_bound"} { + t.Run(state, func(t *testing.T) { + input := validCompactContract() + input["bindings"].([]any)[0].([]any)[5] = state + + _, err := Admit(input) + if err == nil || !strings.Contains(err.Error(), "must be witness_backed") { + t.Fatalf("Admit() error=%v, want witness-bearing binding state rejection", err) + } + }) + } +} + +func TestAdmitScenarioIDOwnsScopedIdentityGrammar(t *testing.T) { + scenarioID, surfaceID, err := AdmitScenarioID("proofkit.coverage::scenario", "scenarioId") + if err != nil { + t.Fatalf("AdmitScenarioID() error=%v", err) + } + if scenarioID != "proofkit.coverage::scenario" || surfaceID != "proofkit.coverage" { + t.Fatalf("AdmitScenarioID()=(%q,%q), want exact scoped identity", scenarioID, surfaceID) + } + if _, _, err := AdmitScenarioID("unscoped-scenario", "scenarioId"); err == nil { + t.Fatal("AdmitScenarioID() admitted an unscoped identity") + } +} + func TestAdmittedContractIsIndependentFromRawMutation(t *testing.T) { input := validCompactContract() contract, err := Admit(input) diff --git a/internal/kernel/proofvocab/proofvocab.go b/internal/kernel/proofvocab/proofvocab.go index c68e088..b29e76a 100644 --- a/internal/kernel/proofvocab/proofvocab.go +++ b/internal/kernel/proofvocab/proofvocab.go @@ -6,6 +6,12 @@ var mergeSatisfactionClasses = []string{"advisory", "merge_satisfying"} var obligationClasses = []string{"advisory", "blocking", "deferred"} +var requirementProofStates = []string{ + "explicitly_deferred", + "not_bound", + "witness_backed", +} + var obligationDecisionStates = []string{ "invalid_producer", "invalid_receipt", @@ -64,6 +70,14 @@ func ObligationClassSet() map[string]struct{} { return set(obligationClasses) } +func RequirementProofStates() []string { + return clone(requirementProofStates) +} + +func RequirementProofStateSet() map[string]struct{} { + return set(requirementProofStates) +} + func ObligationDecisionStates() []string { return clone(obligationDecisionStates) } diff --git a/internal/kernel/proofvocab/proofvocab_test.go b/internal/kernel/proofvocab/proofvocab_test.go index dec307a..391f6e2 100644 --- a/internal/kernel/proofvocab/proofvocab_test.go +++ b/internal/kernel/proofvocab/proofvocab_test.go @@ -62,6 +62,16 @@ func TestVocabularyAccessorsReturnCopies(t *testing.T) { if _, ok := ObligationClassSet()["blocking"]; !ok { t.Fatalf("ObligationClassSet returned mutable owner set") } + proofStates := RequirementProofStates() + proofStates[0] = "mutated" + if RequirementProofStates()[0] == "mutated" { + t.Fatalf("RequirementProofStates returned mutable owner slice") + } + proofStateSet := RequirementProofStateSet() + delete(proofStateSet, "witness_backed") + if _, ok := RequirementProofStateSet()["witness_backed"]; !ok { + t.Fatalf("RequirementProofStateSet returned mutable owner set") + } decisionStates := ObligationDecisionStates() decisionStates[0] = "mutated" if ObligationDecisionStates()[0] == "mutated" { diff --git a/internal/tools/coveragemetrics/main.go b/internal/tools/coveragemetrics/main.go index 950c0f7..baa1dd4 100644 --- a/internal/tools/coveragemetrics/main.go +++ b/internal/tools/coveragemetrics/main.go @@ -120,30 +120,31 @@ type witnessPlanMetrics struct { } type cliContractMetrics struct { - CommandCount int `json:"commandCount"` + CommandCount int `json:"commandCount"` + Commands []string `json:"commands"` } type commandRouteMetrics struct { - AdmittedInventoryEntryCount int `json:"admittedInventoryEntryCount"` - CommandCount int `json:"commandCount"` - CommandWithoutProofRouteCandidateCount int `json:"commandWithoutProofRouteCandidateCount"` - CommandsWithoutProofRouteCandidate []string `json:"commandsWithoutProofRouteCandidate"` - ContractOnlyCommandCount int `json:"contractOnlyCommandCount"` - ContractOnlyCommands []string `json:"contractOnlyCommands"` - CommandWithoutSemanticFalsifierRouteCount int `json:"commandWithoutSemanticFalsifierRouteCount"` - CommandsWithoutSemanticFalsifierRoute []string `json:"commandsWithoutSemanticFalsifierRoute"` - RouteCount int `json:"routeCount"` - RouteOnlyCommandCount int `json:"routeOnlyCommandCount"` - RouteOnlyCommands []string `json:"routeOnlyCommands"` - RouteSmokeCount int `json:"routeSmokeCount"` - ProofRouteCandidateInventoryEntryCount int `json:"proofRouteCandidateInventoryEntryCount"` - ProofRouteCandidateRouteCount int `json:"proofRouteCandidateRouteCount"` - SemanticInventoryEntryCount int `json:"semanticInventoryEntryCount"` - SemanticRouteCount int `json:"semanticRouteCount"` - UnknownProofRouteCandidateRefs []string `json:"unknownProofRouteCandidateRefs"` - UnknownProofRouteCandidateRefCount int `json:"unknownProofRouteCandidateRefCount"` - UnknownSemanticCommandRefs []string `json:"unknownSemanticCommandRefs"` - UnknownSemanticCommandRefCount int `json:"unknownSemanticCommandRefCount"` + AdmittedInventoryEntryCount int `json:"admittedInventoryEntryCount"` + CommandCount int `json:"commandCount"` + Commands []string `json:"commands"` + CommandWithoutProofRouteCandidateCount int `json:"commandWithoutProofRouteCandidateCount"` + CommandsWithoutProofRouteCandidate []string `json:"commandsWithoutProofRouteCandidate"` + ContractOnlyCommandCount int `json:"contractOnlyCommandCount"` + ContractOnlyCommands []string `json:"contractOnlyCommands"` + CommandWithoutDeclaredSemanticFalsifierRouteCount int `json:"commandWithoutDeclaredSemanticFalsifierRouteCount"` + CommandsWithoutDeclaredSemanticFalsifierRoute []string `json:"commandsWithoutDeclaredSemanticFalsifierRoute"` + RouteCount int `json:"routeCount"` + RouteOnlyCommandCount int `json:"routeOnlyCommandCount"` + RouteOnlyCommands []string `json:"routeOnlyCommands"` + RouteSmokeCount int `json:"routeSmokeCount"` + ProofRouteCandidateInventoryEntryCount int `json:"proofRouteCandidateInventoryEntryCount"` + ProofRouteCandidateRouteCount int `json:"proofRouteCandidateRouteCount"` + DeclaredSemanticFalsifierRouteEntryCount int `json:"declaredSemanticFalsifierRouteEntryCount"` + UnknownProofRouteCandidateRefs []string `json:"unknownProofRouteCandidateRefs"` + UnknownProofRouteCandidateRefCount int `json:"unknownProofRouteCandidateRefCount"` + UnknownDeclaredSemanticRouteCommandRefs []string `json:"unknownDeclaredSemanticRouteCommandRefs"` + UnknownDeclaredSemanticRouteCommandRefCount int `json:"unknownDeclaredSemanticRouteCommandRefCount"` } type deadZoneMetrics struct { @@ -216,6 +217,7 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { {"REQ-PROOFKIT-PACKAGE-002", "proofkit.package-boundary.generated-command-caller-preservation"}: {"TestBootstrapPreservesCallerDisplayCommandInGuidancePayload"}, {"REQ-PROOFKIT-PACKAGE-002", "proofkit.package-boundary.cli-output-root-witnesses"}: { "TestAdoptionContractEnvelopeCLIABI", + "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", "TestSelfCheckOutputUsesExactRootShape", "TestStandaloneMultiVariantCommandsUseExactRootShapes", }, @@ -237,6 +239,7 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-abi-golden"}: { "TestAdoptionContractEnvelopeCLIABI", "TestRequiredInputCommandsRouteStructuralErrorsByMode", + "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", "TestRequirementBrowserOneShotCLIOutputVariants", "TestSelfCheckOutputUsesExactRootShape", "TestStandaloneMultiVariantCommandsUseExactRootShapes", @@ -255,6 +258,9 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-output-witness-contract"}: { "TestRootDistinctOutputWitnessBindingsAreExact", }, + {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-output-schema-evolution"}: { + "TestRequirementCoverageViewBreakingRootUsesVersionedOutputContract", + }, {"REQ-PROOFKIT-QUALITY-005", "proofkit.supply-chain-quality.codeql-permission-separation"}: { "TestSecurityScannerWorkflowsSeparateProviderPublicationPermissions", }, @@ -333,6 +339,9 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { "TestVerifyWheelContentsAcceptsDarwinTagAtOrAboveMachOMinimum", "TestVerifyWheelContentsRejectsDarwinTagBelowMachOMinimum", }, + {"REQ-PROOFKIT-QUALITY-015", "proofkit.supply-chain-quality.release-closeout-completion-criteria"}: { + "TestBuildInputFailsClosedForEachBlockingEvidenceClass", + }, {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.release-change-record-projection"}: { "TestAdmitEnforcesVersionedChangeClass", "TestCurrentChangeRecordNamesReviewedSemanticChanges", @@ -373,6 +382,28 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { "TestServeOneShotReturnsCleanupFailuresWithoutWritingTerminalPacket", "TestServeOneShotWaitsForDoneBeforeWritingTerminalPacket", }, + {"REQ-PROOFKIT-SPEC-006", "proofkit.spec-proof-core.test-inventory-and-coverage-view"}: { + "TestAdmitOutputRejectsCompactProjectionDrift", + "TestAdmitOutputRejectsMissingInverseParentProjection", + "TestAdmitOutputRejectsNonCanonicalWireProjectionText", + "TestAdmitOutputRejectsRemovedValidUnmappedInventoryEntry", + "TestAdmitOutputReplaysFailedInventoryQualitySemantics", + "TestAdmitOutputReplaysFullRepositorySourceOwnerScopeFailures", + "TestAdmitOutputReplaysOwnerScopeFailures", + "TestAdmitOutputRequiresEveryCoverageBasisField", + "TestAdmitOutputRequiresEveryDeclaredRootField", + "TestAdmitOutputRetainsFailedInventoryEntriesWithoutProjectedParents", + "TestAdmitOutputValidatesEveryCoverageRowMetadataField", + }, + {"REQ-PROOFKIT-SPEC-006", "proofkit.spec-proof-core.declared-route-mapping-without-assurance"}: { + "TestBuildJSONMissingSelectorRemainsMappingOnly", + }, + {"REQ-PROOFKIT-SPEC-012", "proofkit.spec-proof-core.requirement-authoring-ref-provenance"}: { + "TestBuildPreservesDigestBoundAuthoringRefIdentity", + }, + {"REQ-PROOFKIT-RETIRE-006", "proofkit.consumer-infra-retirement.migration-parity-admission"}: { + "TestBuildProjectsEveryCallerDeclaredStatusAndSummaryField", + }, } requiredPaths := map[inventoryKey]string{ {"REQ-PROOFKIT-PACKAGE-001", "proofkit.package-boundary.root-export-and-deep-import-denial"}: "internal/tools/packageverify/main_test.go", @@ -392,6 +423,7 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-abi-golden"}: "internal/app/cli_abi_test.go", {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-contract-topology"}: "internal/app/cli_contract_test.go", {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-output-witness-contract"}: "internal/app/cli_output_witness_contract_test.go", + {"REQ-PROOFKIT-QUALITY-004", "proofkit.supply-chain-quality.cli-output-schema-evolution"}: "internal/app/cli_contract_test.go", {"REQ-PROOFKIT-QUALITY-005", "proofkit.supply-chain-quality.codeql-permission-separation"}: "scripts/workflow_security_scanner_oracles_test.go", {"REQ-PROOFKIT-QUALITY-006", "proofkit.supply-chain-quality.osv-permission-separation"}: "scripts/workflow_security_scanner_oracles_test.go", {"REQ-PROOFKIT-QUALITY-007", "proofkit.supply-chain-quality.scorecard-permission-and-publication-inputs"}: "scripts/workflow_security_scanner_oracles_test.go", @@ -406,17 +438,22 @@ func validateRequiredBindingWitnessSelectors(bindings bindingFile) error { {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.npm-registry-workflow-delegation"}: "scripts/validate-self-hosting-receipts_test.go", {"REQ-PROOFKIT-QUALITY-022", "proofkit.supply-chain-quality.browser-failure-diagnostics-retention"}: "scripts/workflow_browser_runtime_oracle_test.go", {"REQ-PROOFKIT-QUALITY-023", "proofkit.supply-chain-quality.python-wheel-platform-byte-compatibility"}: "internal/tools/pythonpackage/metadata_test.go", + {"REQ-PROOFKIT-QUALITY-015", "proofkit.supply-chain-quality.release-closeout-completion-criteria"}: "internal/tools/releasecloseoutinput/main_test.go", {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.release-change-record-projection"}: "internal/tools/releasechange/record_test.go", {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.retained-evidence-artifact-topology"}: "internal/tools/retainedevidence/manifest_test.go", {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.release-closeout-change-record"}: "internal/tools/releasecloseoutinput/main_test.go", {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.release-predecessor-lineage"}: "internal/tools/releasepreflight/main_test.go", {"REQ-PROOFKIT-QUALITY-024", "proofkit.supply-chain-quality.release-predecessor-lineage-workflow"}: "scripts/validate-self-hosting-receipts_test.go", {"REQ-PROOFKIT-QUALITY-025", "proofkit.supply-chain-quality.workflow-source-oracles"}: "scripts/workflow_source_oracles_test.go", + {"REQ-PROOFKIT-RETIRE-006", "proofkit.consumer-infra-retirement.migration-parity-admission"}: "internal/command/migrationparityadmission/migrationparityadmission_test.go", {"REQ-PROOFKIT-SPEC-011", "proofkit.spec-proof-core.adoption-contract-envelope-cli-abi"}: "internal/app/cli_abi_test.go", {"REQ-PROOFKIT-SPEC-007", "proofkit.spec-proof-core.canonical-command-input-admission"}: "internal/app/command_coverage_test.go", {"REQ-PROOFKIT-SPEC-007", "proofkit.spec-proof-core.canonical-input-admission"}: "internal/kernel/admission/json_test.go", {"REQ-PROOFKIT-SPEC-013", "proofkit.spec-proof-core.receipt-trust-status-vocabulary-admission"}: "internal/command/receipttrustclass/receipt_trust_class_test.go", {"REQ-PROOFKIT-SPEC-021", "proofkit.spec-proof-core.requirement-browser-one-shot-cleanup"}: "internal/command/requirementbrowser/server_test.go", + {"REQ-PROOFKIT-SPEC-006", "proofkit.spec-proof-core.test-inventory-and-coverage-view"}: "internal/command/requirementcoverageview/output_closure_test.go", + {"REQ-PROOFKIT-SPEC-006", "proofkit.spec-proof-core.declared-route-mapping-without-assurance"}: "internal/command/requirementcoverageview/requirementcoverageview_test.go", + {"REQ-PROOFKIT-SPEC-012", "proofkit.spec-proof-core.requirement-authoring-ref-provenance"}: "internal/command/requirementauthoringplan/requirement_authoring_plan_test.go", } if len(requiredPaths) != len(required) { return fmt.Errorf("required selector path inventory=%d, selector inventory=%d", len(requiredPaths), len(required)) @@ -799,6 +836,7 @@ func buildMetrics(requirements []requirementRecord, bindings bindingFile, witnes sort.Strings(requirementWithoutBinding) sort.Strings(scenarioWithoutCommand) sort.Strings(scenarioWithoutRequirement) + contractCommands := cliContractCommandNames(contract) commandRoutes := buildCommandRouteMetrics(contract, app.CommandCoverageSummaries(), commandInventory) return metrics{ ArtifactKind: "proofkit.coverage-metrics.v1", @@ -815,7 +853,7 @@ func buildMetrics(requirements []requirementRecord, bindings bindingFile, witnes WitnessBackedRequirementCount: len(witnessBacked), }, WitnessPlan: witnessPlanMetrics{CommandCount: len(commandIDs)}, - CLIContract: cliContractMetrics{CommandCount: len(contract.Commands)}, + CLIContract: cliContractMetrics{CommandCount: len(contractCommands), Commands: contractCommands}, CommandRoutes: commandRoutes, DeadZones: deadZoneMetrics{ BindingWithoutRequirementIDs: bindingWithoutRequirement, @@ -825,13 +863,22 @@ func buildMetrics(requirements []requirementRecord, bindings bindingFile, witnes }, NonClaims: []string{ "Coverage metrics report explicit requirement, binding, witness, and CLI inventory linkage only.", - "Coverage metrics classify static command route metadata as proof_route_candidate; route prose, source markers, test existence, and failure-capable AST nodes do not count as semantic_falsifier evidence.", + "Coverage metrics classify static command route metadata as proof_route_candidate; route prose, source markers, test existence, and failure-capable AST nodes do not become execution-backed semantic evidence.", "Coverage metrics do not execute command route candidates or observe a concrete falsification event.", "Coverage metrics do not claim line coverage, semantic correctness, command execution, receipt freshness, or merge satisfaction.", }, } } +func cliContractCommandNames(contract cliContract) []string { + commands := make([]string, 0, len(contract.Commands)) + for _, command := range contract.Commands { + commands = append(commands, command.Command) + } + sort.Strings(commands) + return commands +} + func readCommandCoverageInventory() (testevidenceinventory.Inventory, error) { raw, err := commandCoverageInventoryInput() if err != nil { @@ -853,14 +900,14 @@ func readCommandCoverageInventoryFrom(raw any) (testevidenceinventory.Inventory, func buildCommandRouteMetrics(contract cliContract, summaries []app.CommandCoverageSummary, inventory testevidenceinventory.Inventory) commandRouteMetrics { missingCandidates := []string{} - missingSemantic := []string{} + missingDeclaredSemanticRoutes := []string{} contractRefs := map[string]string{} knownRefs := map[string]struct{}{} candidateRefs := map[string]struct{}{} - semanticRefs := map[string]struct{}{} + declaredSemanticRouteRefs := map[string]struct{}{} routeOnlyCount := 0 candidateEntryCount := 0 - semanticEntryCount := 0 + declaredSemanticRouteEntryCount := 0 for _, command := range contract.Commands { contractRefs[app.CommandCoverageCommandRef(command.Command)] = command.Command } @@ -869,12 +916,12 @@ func buildCommandRouteMetrics(contract cliContract, summaries []app.CommandCover } for _, entry := range inventory.Entries { switch entry.EvidenceClass { - case "semantic_falsifier": - semanticEntryCount++ + case testevidenceinventory.EvidenceClassDeclaredSemanticFalsifierRoute: + declaredSemanticRouteEntryCount++ for _, commandRef := range entry.CommandRefs { - semanticRefs[commandRef] = struct{}{} + declaredSemanticRouteRefs[commandRef] = struct{}{} } - case "proof_route_candidate": + case testevidenceinventory.EvidenceClassProofRouteCandidate: candidateEntryCount++ for _, commandRef := range entry.CommandRefs { candidateRefs[commandRef] = struct{}{} @@ -883,10 +930,10 @@ func buildCommandRouteMetrics(contract cliContract, summaries []app.CommandCover routeOnlyCount++ } } - unknownRefs := []string{} - for ref := range semanticRefs { + unknownDeclaredSemanticRouteRefs := []string{} + for ref := range declaredSemanticRouteRefs { if _, ok := knownRefs[ref]; !ok { - unknownRefs = append(unknownRefs, ref) + unknownDeclaredSemanticRouteRefs = append(unknownDeclaredSemanticRouteRefs, ref) } } unknownCandidateRefs := []string{} @@ -910,54 +957,55 @@ func buildCommandRouteMetrics(contract cliContract, summaries []app.CommandCover sort.Strings(contractOnly) sort.Strings(routeOnly) sort.Strings(unknownCandidateRefs) - sort.Strings(unknownRefs) + sort.Strings(unknownDeclaredSemanticRouteRefs) out := commandRouteMetrics{ - AdmittedInventoryEntryCount: len(inventory.Entries), - CommandCount: len(summaries), - ContractOnlyCommands: contractOnly, - ContractOnlyCommandCount: len(contractOnly), - RouteOnlyCommands: routeOnly, - RouteOnlyCommandCount: len(routeOnly), - RouteSmokeCount: routeOnlyCount, - ProofRouteCandidateInventoryEntryCount: candidateEntryCount, - SemanticInventoryEntryCount: semanticEntryCount, - UnknownProofRouteCandidateRefs: unknownCandidateRefs, - UnknownProofRouteCandidateRefCount: len(unknownCandidateRefs), - UnknownSemanticCommandRefs: unknownRefs, - UnknownSemanticCommandRefCount: len(unknownRefs), + AdmittedInventoryEntryCount: len(inventory.Entries), + CommandCount: len(summaries), + ContractOnlyCommands: contractOnly, + ContractOnlyCommandCount: len(contractOnly), + RouteOnlyCommands: routeOnly, + RouteOnlyCommandCount: len(routeOnly), + RouteSmokeCount: routeOnlyCount, + ProofRouteCandidateInventoryEntryCount: candidateEntryCount, + DeclaredSemanticFalsifierRouteEntryCount: declaredSemanticRouteEntryCount, + UnknownProofRouteCandidateRefs: unknownCandidateRefs, + UnknownProofRouteCandidateRefCount: len(unknownCandidateRefs), + UnknownDeclaredSemanticRouteCommandRefs: unknownDeclaredSemanticRouteRefs, + UnknownDeclaredSemanticRouteCommandRefCount: len(unknownDeclaredSemanticRouteRefs), } for _, summary := range summaries { + out.Commands = append(out.Commands, summary.Command) out.RouteCount += summary.RouteCount out.ProofRouteCandidateRouteCount += summary.ProofRouteCandidateCount - out.SemanticRouteCount += summary.SemanticRouteCount if _, ok := candidateRefs[summary.CommandRef]; !ok { missingCandidates = append(missingCandidates, summary.Command) } - if _, ok := semanticRefs[summary.CommandRef]; !ok { - missingSemantic = append(missingSemantic, summary.Command) + if _, ok := declaredSemanticRouteRefs[summary.CommandRef]; !ok { + missingDeclaredSemanticRoutes = append(missingDeclaredSemanticRoutes, summary.Command) } } + sort.Strings(out.Commands) sort.Strings(missingCandidates) - sort.Strings(missingSemantic) + sort.Strings(missingDeclaredSemanticRoutes) out.CommandsWithoutProofRouteCandidate = missingCandidates out.CommandWithoutProofRouteCandidateCount = len(missingCandidates) - out.CommandsWithoutSemanticFalsifierRoute = missingSemantic - out.CommandWithoutSemanticFalsifierRouteCount = len(missingSemantic) + out.CommandsWithoutDeclaredSemanticFalsifierRoute = missingDeclaredSemanticRoutes + out.CommandWithoutDeclaredSemanticFalsifierRouteCount = len(missingDeclaredSemanticRoutes) return out } func requireCommandRouteInventoryClosure(metrics commandRouteMetrics) error { if len(metrics.CommandsWithoutProofRouteCandidate) == 0 && len(metrics.UnknownProofRouteCandidateRefs) == 0 && - len(metrics.UnknownSemanticCommandRefs) == 0 && + len(metrics.UnknownDeclaredSemanticRouteCommandRefs) == 0 && len(metrics.ContractOnlyCommands) == 0 && len(metrics.RouteOnlyCommands) == 0 { return nil } - return fmt.Errorf("command proof-route inventory defects: missingCandidates=%v unknownCandidateRefs=%v unknownSemanticRefs=%v contractOnly=%v routeOnly=%v", + return fmt.Errorf("command proof-route inventory defects: missingCandidates=%v unknownCandidateRefs=%v unknownDeclaredSemanticRouteRefs=%v contractOnly=%v routeOnly=%v", metrics.CommandsWithoutProofRouteCandidate, metrics.UnknownProofRouteCandidateRefs, - metrics.UnknownSemanticCommandRefs, + metrics.UnknownDeclaredSemanticRouteCommandRefs, metrics.ContractOnlyCommands, metrics.RouteOnlyCommands, ) diff --git a/internal/tools/coveragemetrics/main_test.go b/internal/tools/coveragemetrics/main_test.go index ce0f5cb..ce468e1 100644 --- a/internal/tools/coveragemetrics/main_test.go +++ b/internal/tools/coveragemetrics/main_test.go @@ -63,35 +63,35 @@ func TestReadRequirementsRequiresOwnerAdmission(t *testing.T) { } } -func TestBuildCommandRouteMetricsReportsMissingSemanticRoutes(t *testing.T) { +func TestBuildCommandRouteMetricsReportsMissingDeclaredSemanticRoutes(t *testing.T) { metrics := buildCommandRouteMetrics(cliContractWithCommands("covered", "z-route-only", "a-route-only"), []app.CommandCoverageSummary{ {Command: "covered", CommandRef: app.CommandCoverageCommandRef("covered"), RouteCount: 2, ProofRouteCandidateCount: 1}, {Command: "z-route-only", CommandRef: app.CommandCoverageCommandRef("z-route-only"), RouteCount: 1, ProofRouteCandidateCount: 1}, {Command: "a-route-only", CommandRef: app.CommandCoverageCommandRef("a-route-only"), RouteCount: 1, ProofRouteCandidateCount: 1}, }, testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{ - {CommandRefs: []string{app.CommandCoverageCommandRef("covered")}, EvidenceClass: "semantic_falsifier"}, + {CommandRefs: []string{app.CommandCoverageCommandRef("covered")}, EvidenceClass: "declared_semantic_falsifier_route"}, {CommandRefs: []string{app.CommandCoverageCommandRef("z-route-only")}, EvidenceClass: "proof_route_candidate"}, {CommandRefs: []string{app.CommandCoverageCommandRef("covered")}, EvidenceClass: "routing_smoke_nonclaim"}, }}) - if metrics.CommandCount != 3 || metrics.RouteCount != 4 || metrics.ProofRouteCandidateRouteCount != 3 || metrics.SemanticRouteCount != 0 || metrics.RouteSmokeCount != 1 { + if metrics.CommandCount != 3 || metrics.RouteCount != 4 || metrics.ProofRouteCandidateRouteCount != 3 || metrics.RouteSmokeCount != 1 { t.Fatalf("unexpected metrics: %#v", metrics) } - if metrics.AdmittedInventoryEntryCount != 3 || metrics.ProofRouteCandidateInventoryEntryCount != 1 || metrics.SemanticInventoryEntryCount != 1 { + if metrics.AdmittedInventoryEntryCount != 3 || metrics.ProofRouteCandidateInventoryEntryCount != 1 || metrics.DeclaredSemanticFalsifierRouteEntryCount != 1 { t.Fatalf("unexpected inventory metrics: %#v", metrics) } - if metrics.CommandWithoutSemanticFalsifierRouteCount != 2 { - t.Fatalf("missing semantic route count=%d, want 2", metrics.CommandWithoutSemanticFalsifierRouteCount) + if metrics.CommandWithoutDeclaredSemanticFalsifierRouteCount != 2 { + t.Fatalf("missing declared semantic route count=%d, want 2", metrics.CommandWithoutDeclaredSemanticFalsifierRouteCount) } want := []string{"a-route-only", "z-route-only"} - if strings.Join(metrics.CommandsWithoutSemanticFalsifierRoute, ",") != strings.Join(want, ",") { - t.Fatalf("missing commands=%#v, want %#v", metrics.CommandsWithoutSemanticFalsifierRoute, want) + if strings.Join(metrics.CommandsWithoutDeclaredSemanticFalsifierRoute, ",") != strings.Join(want, ",") { + t.Fatalf("missing commands=%#v, want %#v", metrics.CommandsWithoutDeclaredSemanticFalsifierRoute, want) } } -func TestBuildCommandRouteMetricsRequiresMatchingSemanticInventoryCommandRef(t *testing.T) { +func TestBuildCommandRouteMetricsRequiresMatchingDeclaredSemanticInventoryCommandRef(t *testing.T) { summaries := []app.CommandCoverageSummary{ - {Command: "target", CommandRef: app.CommandCoverageCommandRef("target"), RouteCount: 1, SemanticRouteCount: 1}, + {Command: "target", CommandRef: app.CommandCoverageCommandRef("target"), RouteCount: 1}, } contract := cliContractWithCommands("target") cases := []struct { @@ -101,13 +101,13 @@ func TestBuildCommandRouteMetricsRequiresMatchingSemanticInventoryCommandRef(t * { name: "mismatched command ref", inventory: testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{ - {CommandRefs: []string{app.CommandCoverageCommandRef("other")}, EvidenceClass: "semantic_falsifier"}, + {CommandRefs: []string{app.CommandCoverageCommandRef("other")}, EvidenceClass: "declared_semantic_falsifier_route"}, }}, }, { name: "unknown command ref", inventory: testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{ - {CommandRefs: []string{"proofkit.cli.unknown"}, EvidenceClass: "semantic_falsifier"}, + {CommandRefs: []string{"proofkit.cli.unknown"}, EvidenceClass: "declared_semantic_falsifier_route"}, }}, }, { @@ -125,47 +125,48 @@ func TestBuildCommandRouteMetricsRequiresMatchingSemanticInventoryCommandRef(t * { name: "contract evidence is not semantic command falsifier", inventory: testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{ - {CommandRefs: []string{app.CommandCoverageCommandRef("target")}, EvidenceClass: "contract_admission"}, + {CommandRefs: []string{app.CommandCoverageCommandRef("target")}, EvidenceClass: "declared_contract_admission_route"}, }}, }, } for _, item := range cases { t.Run(item.name, func(t *testing.T) { metrics := buildCommandRouteMetrics(contract, summaries, item.inventory) - if metrics.CommandWithoutSemanticFalsifierRouteCount != 1 || - strings.Join(metrics.CommandsWithoutSemanticFalsifierRoute, ",") != "target" { - t.Fatalf("metrics=%#v, want target missing semantic command coverage", metrics) + if metrics.CommandWithoutDeclaredSemanticFalsifierRouteCount != 1 || + strings.Join(metrics.CommandsWithoutDeclaredSemanticFalsifierRoute, ",") != "target" { + t.Fatalf("metrics=%#v, want target missing declared semantic route", metrics) } }) } } -func TestBuildCommandRouteMetricsReportsUnknownSemanticCommandRefs(t *testing.T) { +func TestBuildCommandRouteMetricsReportsUnknownDeclaredSemanticRouteCommandRefs(t *testing.T) { summaries := []app.CommandCoverageSummary{ - {Command: "target", CommandRef: app.CommandCoverageCommandRef("target"), RouteCount: 1, SemanticRouteCount: 1}, + {Command: "target", CommandRef: app.CommandCoverageCommandRef("target"), RouteCount: 1}, } metrics := buildCommandRouteMetrics(cliContractWithCommands("target"), summaries, testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{ - {CommandRefs: []string{app.CommandCoverageCommandRef("target")}, EvidenceClass: "semantic_falsifier"}, - {CommandRefs: []string{"proofkit.cli.unknown"}, EvidenceClass: "semantic_falsifier"}, + {CommandRefs: []string{app.CommandCoverageCommandRef("target")}, EvidenceClass: "declared_semantic_falsifier_route"}, + {CommandRefs: []string{app.CommandCoverageCommandRef("target")}, EvidenceClass: "proof_route_candidate"}, + {CommandRefs: []string{"proofkit.cli.unknown"}, EvidenceClass: "declared_semantic_falsifier_route"}, }}) - if metrics.CommandWithoutSemanticFalsifierRouteCount != 0 { + if metrics.CommandWithoutDeclaredSemanticFalsifierRouteCount != 0 { t.Fatalf("covered target was reported missing: %#v", metrics) } - if metrics.UnknownSemanticCommandRefCount != 1 || strings.Join(metrics.UnknownSemanticCommandRefs, ",") != "proofkit.cli.unknown" { + if metrics.UnknownDeclaredSemanticRouteCommandRefCount != 1 || strings.Join(metrics.UnknownDeclaredSemanticRouteCommandRefs, ",") != "proofkit.cli.unknown" { t.Fatalf("unknown semantic command refs not reported: %#v", metrics) } - if err := requireCommandRouteInventoryClosure(metrics); err == nil || !strings.Contains(err.Error(), "unknownSemanticRefs") { + if err := requireCommandRouteInventoryClosure(metrics); err == nil || !strings.Contains(err.Error(), "unknownDeclaredSemanticRouteRefs") { t.Fatalf("requireCommandRouteInventoryClosure() error=%v, want unknown ref failure", err) } } func TestBuildCommandRouteMetricsReportsContractRouteDrift(t *testing.T) { metrics := buildCommandRouteMetrics(cliContractWithCommands("contract-only", "shared"), []app.CommandCoverageSummary{ - {Command: "route-only", CommandRef: app.CommandCoverageCommandRef("route-only"), RouteCount: 1, SemanticRouteCount: 1}, - {Command: "shared", CommandRef: app.CommandCoverageCommandRef("shared"), RouteCount: 1, SemanticRouteCount: 1}, + {Command: "route-only", CommandRef: app.CommandCoverageCommandRef("route-only"), RouteCount: 1}, + {Command: "shared", CommandRef: app.CommandCoverageCommandRef("shared"), RouteCount: 1}, }, testevidenceinventory.Inventory{Entries: []testevidenceinventory.Entry{ - {CommandRefs: []string{app.CommandCoverageCommandRef("shared")}, EvidenceClass: "semantic_falsifier"}, + {CommandRefs: []string{app.CommandCoverageCommandRef("shared")}, EvidenceClass: "declared_semantic_falsifier_route"}, }}) if got := strings.Join(metrics.ContractOnlyCommands, ","); got != "contract-only" { @@ -184,7 +185,7 @@ func TestReadCommandCoverageInventoryRejectsUnanchoredCandidate(t *testing.T) { firstCandidateInventoryEntry(t, mutated)["ownerInvariantRefs"] = []any{} _, err := readCommandCoverageInventoryFrom(mutated) - if err == nil || !strings.Contains(err.Error(), "missing_semantic_anchor") { + if err == nil || !strings.Contains(err.Error(), "missing_declared_route_anchor") { t.Fatalf("readCommandCoverageInventoryFrom() error = %v, want missing candidate anchor failure", err) } } @@ -227,13 +228,13 @@ func TestRunWritesCurrentMetricsWhenCommandCoverageInventoryFails(t *testing.T) ArtifactKind: "proofkit.coverage-metrics.v1", SchemaVersion: 1, CommandRoutes: commandRouteMetrics{ - CommandWithoutSemanticFalsifierRouteCount: 0, + CommandWithoutDeclaredSemanticFalsifierRouteCount: 0, }, }, nil); err != nil { t.Fatalf("write stale success: %v", err) } err = run() - if err == nil || !strings.Contains(err.Error(), "missing_semantic_anchor") { + if err == nil || !strings.Contains(err.Error(), "missing_declared_route_anchor") { t.Fatalf("run() error = %v, want failed inventory error", err) } content, err := os.ReadFile(outputPath) @@ -241,8 +242,8 @@ func TestRunWritesCurrentMetricsWhenCommandCoverageInventoryFails(t *testing.T) t.Fatalf("read current metrics: %v", err) } if !strings.Contains(string(content), `"admittedInventoryEntryCount": 0`) || - !strings.Contains(string(content), `"commandsWithoutSemanticFalsifierRoute"`) || - strings.Contains(string(content), `"commandWithoutSemanticFalsifierRouteCount": 0`) { + !strings.Contains(string(content), `"commandsWithoutDeclaredSemanticFalsifierRoute"`) || + strings.Contains(string(content), `"commandWithoutDeclaredSemanticFalsifierRouteCount": 0`) { t.Fatalf("failed inventory did not replace stale success artifact:\n%s", string(content)) } } @@ -272,7 +273,7 @@ func TestRunWritesCurrentMetricsWhenCommandRouteInventoryBuilderFails(t *testing ArtifactKind: "proofkit.coverage-metrics.v1", SchemaVersion: 1, CommandRoutes: commandRouteMetrics{ - CommandWithoutSemanticFalsifierRouteCount: 0, + CommandWithoutDeclaredSemanticFalsifierRouteCount: 0, }, }, nil); err != nil { t.Fatalf("write stale success: %v", err) @@ -286,7 +287,7 @@ func TestRunWritesCurrentMetricsWhenCommandRouteInventoryBuilderFails(t *testing t.Fatalf("read current metrics: %v", err) } if !strings.Contains(string(content), `"admittedInventoryEntryCount": 0`) || - strings.Contains(string(content), `"commandWithoutSemanticFalsifierRouteCount": 0`) { + strings.Contains(string(content), `"commandWithoutDeclaredSemanticFalsifierRouteCount": 0`) { t.Fatalf("failed route inventory builder did not replace stale success artifact:\n%s", string(content)) } } @@ -299,7 +300,7 @@ func TestEachCommandRouteClosureConjunctHasIndependentFalsifier(t *testing.T) { }{ {name: "missing candidate", metrics: commandRouteMetrics{CommandsWithoutProofRouteCandidate: []string{"target"}}, want: "missingCandidates=[target]"}, {name: "unknown candidate ref", metrics: commandRouteMetrics{UnknownProofRouteCandidateRefs: []string{"proofkit.unknown"}}, want: "unknownCandidateRefs=[proofkit.unknown]"}, - {name: "unknown semantic ref", metrics: commandRouteMetrics{UnknownSemanticCommandRefs: []string{"proofkit.unknown"}}, want: "unknownSemanticRefs=[proofkit.unknown]"}, + {name: "unknown declared route ref", metrics: commandRouteMetrics{UnknownDeclaredSemanticRouteCommandRefs: []string{"proofkit.unknown"}}, want: "unknownDeclaredSemanticRouteRefs=[proofkit.unknown]"}, {name: "contract only", metrics: commandRouteMetrics{ContractOnlyCommands: []string{"contract-only"}}, want: "contractOnly=[contract-only]"}, {name: "route only", metrics: commandRouteMetrics{RouteOnlyCommands: []string{"route-only"}}, want: "routeOnly=[route-only]"}, } @@ -401,7 +402,10 @@ func TestBindingWitnessSelectorsRequireExactCriticalInventories(t *testing.T) { "proofkit.supply-chain-quality.workflow-package-gate-oracle", "proofkit.supply-chain-quality.workflow-source-oracles", "proofkit.spec-proof-core.adoption-contract-envelope-cli-abi", + "proofkit.spec-proof-core.declared-route-mapping-without-assurance", + "proofkit.spec-proof-core.requirement-authoring-ref-provenance", "proofkit.spec-proof-core.requirement-browser-one-shot-cleanup", + "proofkit.spec-proof-core.test-inventory-and-coverage-view", } { index := -1 for candidate := range bindings.Bindings { @@ -497,7 +501,10 @@ func TestBindingWitnessSelectorsRequireExactCriticalInventories(t *testing.T) { "proofkit.supply-chain-quality.scorecard-permission-and-publication-inputs", "proofkit.supply-chain-quality.workflow-source-oracles", "proofkit.spec-proof-core.adoption-contract-envelope-cli-abi", + "proofkit.spec-proof-core.declared-route-mapping-without-assurance", + "proofkit.spec-proof-core.requirement-authoring-ref-provenance", "proofkit.spec-proof-core.requirement-browser-one-shot-cleanup", + "proofkit.spec-proof-core.test-inventory-and-coverage-view", } { index := -1 for candidate := range bindings.Bindings { @@ -521,6 +528,45 @@ func TestBindingWitnessSelectorsRequireExactCriticalInventories(t *testing.T) { } } +func TestExactSelectorInventoryRejectsExistingUnrelatedTests(t *testing.T) { + root := filepath.Join("..", "..", "..") + bindings, err := readJSON[bindingFile](filepath.Join(root, "proofkit", "requirement-bindings.json")) + if err != nil { + t.Fatal(err) + } + unrelated := map[string]witnessSelector{ + "proofkit.spec-proof-core.declared-route-mapping-without-assurance": { + Selector: "TestBuildJSONRequiresExactlyOneProofInput", + Command: "go test ./internal/command/requirementcoverageview -run '^TestBuildJSONRequiresExactlyOneProofInput$'", + }, + "proofkit.spec-proof-core.requirement-authoring-ref-provenance": { + Selector: "TestBuildDoesNotReadAuthoringRefPaths", + Command: "go test ./internal/command/requirementauthoringplan -run '^TestBuildDoesNotReadAuthoringRefPaths$'", + }, + "proofkit.spec-proof-core.test-inventory-and-coverage-view": { + Selector: "TestBuildJSONRequiresExactlyOneProofInput", + Command: "go test ./internal/command/requirementcoverageview -run '^TestBuildJSONRequiresExactlyOneProofInput$'", + }, + } + for scenarioID, replacement := range unrelated { + t.Run(scenarioID, func(t *testing.T) { + mutated := cloneBindingFile(bindings) + for index := range mutated.Bindings { + if mutated.Bindings[index].ScenarioID != scenarioID { + continue + } + mutated.Bindings[index].WitnessSelectors = []witnessSelector{replacement} + err := validateBindingWitnessSelectorsAtRoot(root, mutated) + if err == nil || !strings.Contains(err.Error(), "witness selectors=") { + t.Fatalf("unrelated existing selector error=%v", err) + } + return + } + t.Fatalf("binding %s is missing", scenarioID) + }) + } +} + func cloneBindingFile(source bindingFile) bindingFile { clone := bindingFile{ Requirements: append([]bindingRequirement(nil), source.Requirements...), @@ -712,6 +758,19 @@ func TestBuildMetricsReportsScenarioWithoutRequirement(t *testing.T) { } } +func TestBuildMetricsProjectsExactCLIContractCommandInventory(t *testing.T) { + metrics := buildMetrics( + nil, + bindingFile{}, + witnessPlan{}, + cliContractWithCommands("z-command", "a-command"), + testevidenceinventory.Inventory{}, + ) + if metrics.CLIContract.CommandCount != 2 || strings.Join(metrics.CLIContract.Commands, ",") != "a-command,z-command" { + t.Fatalf("CLIContract=%#v, want exact sorted command inventory", metrics.CLIContract) + } +} + func TestWriteMetricsWritesCurrentReportBeforeRouteFailure(t *testing.T) { root := t.TempDir() oldwd, err := os.Getwd() @@ -731,8 +790,8 @@ func TestWriteMetricsWritesCurrentReportBeforeRouteFailure(t *testing.T) { ArtifactKind: "proofkit.coverage-metrics.v1", SchemaVersion: 1, CommandRoutes: commandRouteMetrics{ - CommandWithoutSemanticFalsifierRouteCount: 1, - CommandsWithoutSemanticFalsifierRoute: []string{"route-only"}, + CommandWithoutDeclaredSemanticFalsifierRouteCount: 1, + CommandsWithoutDeclaredSemanticFalsifierRoute: []string{"route-only"}, }, }, errors.New("route failure")) if err == nil || !strings.Contains(err.Error(), "route failure") { @@ -743,7 +802,7 @@ func TestWriteMetricsWritesCurrentReportBeforeRouteFailure(t *testing.T) { if err != nil { t.Fatalf("read output: %v", err) } - if !strings.Contains(string(content), `"commandsWithoutSemanticFalsifierRoute"`) || !strings.Contains(string(content), `"route-only"`) { + if !strings.Contains(string(content), `"commandsWithoutDeclaredSemanticFalsifierRoute"`) || !strings.Contains(string(content), `"route-only"`) { t.Fatalf("output did not preserve current route failure report:\n%s", string(content)) } } @@ -784,10 +843,10 @@ func TestBuildMetricsCarriesRealCommandRouteCandidatesAndNonClaim(t *testing.T) if metrics.CommandRoutes.CommandCount == 0 || metrics.CommandRoutes.ProofRouteCandidateRouteCount == 0 || metrics.CommandRoutes.ProofRouteCandidateInventoryEntryCount == 0 { t.Fatalf("real command route inventory was not loaded: %#v", metrics.CommandRoutes) } - if metrics.CommandRoutes.SemanticRouteCount != 0 || metrics.CommandRoutes.SemanticInventoryEntryCount != 0 { + if metrics.CommandRoutes.DeclaredSemanticFalsifierRouteEntryCount != 0 { t.Fatalf("static command routes were counted as semantic evidence: %#v", metrics.CommandRoutes) } - if metrics.CommandRoutes.CommandWithoutSemanticFalsifierRouteCount != metrics.CommandRoutes.CommandCount { + if metrics.CommandRoutes.CommandWithoutDeclaredSemanticFalsifierRouteCount != metrics.CommandRoutes.CommandCount { t.Fatalf("candidate-only commands did not remain missing semantic evidence: %#v", metrics.CommandRoutes) } if !containsNonClaim(metrics.NonClaims, "do not execute command route candidates") { diff --git a/internal/tools/releasechange/record_test.go b/internal/tools/releasechange/record_test.go index 184d313..61fbe34 100644 --- a/internal/tools/releasechange/record_test.go +++ b/internal/tools/releasechange/record_test.go @@ -194,19 +194,33 @@ func TestCurrentChangeRecordNamesReviewedSemanticChanges(t *testing.T) { assertCurrentChangeRecordNotesRejected(t, "appended duplicate change section", record, notes+"## Breaking Contract Changes\n\n- `proofkit.surplus.section`: Surplus section.\n") } -var currentBreakingChanges = []Change{} +var currentBreakingChanges = []Change{ + {ChangeID: "proofkit.authoring.lossless-reference-projection", Summary: "Requirement authoring plan output advances from schema v1 to v2, requires the complete admitted authoringRefs projection, and composes current and candidate requirement records only from canonical requirement-source owner projections."}, + {ChangeID: "proofkit.coverage.declaration-only-vocabulary", Summary: "Caller-authored test routes, falsifier, oracle-signal, and supersession metadata now use declaration-only classes and mapped states; mapping no longer emits covered, verified, semantic-execution, oracle-review, or dominance-proof vocabulary, and related classifications and coverage metric fields are renamed."}, + {ChangeID: "proofkit.coverage.output-parent-reference-closure", Summary: "Requirement coverage output advances from schema v1 to v2, retains complete test projections on requirement, owner-invariant, command, and unmapped-inventory surfaces, and re-admits the exact root and row shapes, row metadata, command-owned non-claims, canonical row order, coverage-basis owner scope and inventory commitments, inventory warnings and failures, compact scenario route unions, command state, parent references, retained unknown-reference diagnostics, and declared dead-zone diagnostics."}, + {ChangeID: "proofkit.migration.caller-declared-parity", Summary: "Migration parity input statuses, diagnostics, and summaries now identify caller-declared claims, preserve each record's admitted non-claims, and no longer imply that Proofkit computed native parity."}, + {ChangeID: "proofkit.proof.requirement-state-vocabulary-closure", Summary: "Requirement proof states now use one closed vocabulary across structured bindings, compact contracts, and re-admitted coverage output; arbitrary rule IDs and assurance-sounding states are rejected."}, +} var currentAdditions = []Change{ - {ChangeID: "proofkit.admission.canonical-boundary-hardening", Summary: "Shared admission now preserves exact typed JSON numbers and field spelling, separates canonical path-array policy from prose-array policy, rejects complete secret-shaped path and Windows executable alias classes, and requires higher receipt-trust ranks to refine lower-rank obligations."}, - {ChangeID: "proofkit.cli.argument-constraint-closure", Summary: "The public CLI contract and generated help now expose descriptor-owned singleton, mutual-exclusion, presence, enum-domain, and value-dependent flag constraints, including requirement-browser session, scope, view, host, and cross-flag admission."}, - {ChangeID: "proofkit.coverage.witness-anti-vacuity", Summary: "Self-hosting coverage now rejects Go witness selectors that expose no statically reachable failure-capable candidate or contain statically resolved testing.T skip calls, including direct aliases and helper propagation; this remains candidate evidence rather than semantic proof."}, - {ChangeID: "proofkit.proof.delegation-authority", Summary: "Proof obligation algebra now fails closed on unresolved delegation references and cross-requirement child edges until an owner-admitted delegation authority exists."}, - {ChangeID: "proofkit.release.migration-support-baseline", Summary: "Release-history migration support starts at 0.2.0, and any future cumulative plan must be derived from contiguous owner-reviewed per-release records without backfilling pre-baseline release history."}, - {ChangeID: "proofkit.release.npm-predecessor-lineage", Summary: "Release candidate preflight binds a new candidate previousVersion to npm latest, while an exact already-published idempotent candidate requires npm latest to equal the candidate version."}, - {ChangeID: "proofkit.release.registry-authority-closure", Summary: "Release metadata now promotes npm publication only from repository-produced typed registry evidence whose unique package set exactly matches retained provider records and local candidate identity and bytes."}, + {ChangeID: "proofkit.coverage.classification-contract-closure", Summary: "The requirement coverage output contract now declares every reachable diagnostic classification, including proof_route_candidate_only and the fail-closed unclassified_gap fallback."}, + {ChangeID: "proofkit.release.coverage-metric-reachability", Summary: "Release closeout now uses overflow-safe exact coverage-producer relations for inventory entries, route classes, proof-route candidates, the complete command inventory, and commands missing declaration-only routes, and binds both command projections to the actual CLI contract inventory from the same source snapshot."}, } -var currentMigrationSteps = []string{} +var currentMigrationSteps = []string{ + "Replace test inventory evidenceClass values semantic_falsifier, contract_admission, and property_or_fuzz with declared_semantic_falsifier_route, declared_contract_admission_route, and declared_property_or_fuzz_route.", + "Replace test inventory summary fields semanticFalsifierCount and weakOracleFailureCount with declaredSemanticFalsifierRouteCount and incompleteDeclaredOracleMetadataFailureCount; replace diagnostics missing_semantic_anchor and weak_or_empty_oracle with missing_declared_route_anchor and incomplete_declared_oracle_metadata; and replace rule IDs test_inventory.semantic_entries_have_anchors and test_inventory.strong_oracles with test_inventory.declared_routes_have_anchors and test_inventory.declared_oracle_metadata.", + "Within test-evidence-inventory-discovery-draft output, replace summary field weakOracleWarningCount with missingDeclaredAssertionSignalWarningCount, warning and action type weak_or_empty_oracle with missing_declared_assertion_signal, candidate quality finding suffix weak-oracle with missing-declared-assertion-signal, class empty_oracle with missing_edge, and suffix missing-semantic-anchor with missing-declared-route-anchor.", + "Within agent-route inspect_coverage stopConditions, replace weak-oracle language with missing declared assertion-signal language; treat the condition as routing guidance rather than an oracle-quality classification.", + "Replace falsifier field supersessionProofRef with supersessionDeclarationRef, diagnostic reasons missing_dominance_proof and unowned_dominance_proof with missing_supersession_declaration and unowned_supersession_declaration, and rule ID test_inventory.no_duplicate_falsifiers with test_inventory.declared_falsifier_supersession_consistency.", + "Replace requirement coverage states covered_by_semantic_falsifier, covered_by_property_or_fuzz, covered_by_contract_admission, covered_by_governance_invariant_nonproduct, command_semantic_falsifier_present, and missing_command_semantic_falsifier with mapped_to_declared_semantic_falsifier_route, mapped_to_declared_property_or_fuzz_route, mapped_to_declared_contract_admission_route, mapped_to_declared_governance_or_release_route_nonproduct, command_declared_semantic_falsifier_route_present, and missing_command_declared_semantic_falsifier_route; replace classification missing_semantic_test with missing_declared_test_route.", + "Consume proofkit.requirement-coverage-view.output.v2 with schemaVersion 2; preserve every admitted test-entry field, retain entries without selected parents in the required unmappedTests root array, consume the required coverageBasis owner scope, full-repository out-of-scope source identities, and retained-inventory digest, and consume compact scenarios with required verifyCommands and witnessSelectors arrays. Output re-admission now requires exact root and row shapes, exact row metadata, command-owned non-claims, canonical row order, owner-scoped compact scenario identities, exact missing-inventory warnings, and rejects owner-scope, inventory-failure, proof-route union, parent-reference, repeated-test, command-state, retained unknown-reference, mutable-fragment, dead-zone, and row-derived or scope-derived diagnostic drift; coverageBasis does not authenticate source inputs or producer identity.", + "Use only explicitly_deferred, not_bound, or witness_backed for requirement proofState values; compact binding rows contain positive and falsification witnesses and therefore require witness_backed proof_contract_state. Arbitrary rule IDs and assurance-sounding states are no longer admitted.", + "Replace coverage metric fields commandWithoutSemanticFalsifierRouteCount, commandsWithoutSemanticFalsifierRoute, semanticInventoryEntryCount, unknownSemanticCommandRefCount, and unknownSemanticCommandRefs with commandWithoutDeclaredSemanticFalsifierRouteCount, commandsWithoutDeclaredSemanticFalsifierRoute, declaredSemanticFalsifierRouteEntryCount, unknownDeclaredSemanticRouteCommandRefCount, and unknownDeclaredSemanticRouteCommandRefs; stop consuming removed semanticRouteCount, consume commandRoutes.commands and cliContract.commands, and require both inventories to equal the actual CLI contract command inventory from the same source snapshot.", + "Replace migration parity statuses matched, mismatched, not_comparable, and not_run with caller_declared_match, caller_declared_mismatch, caller_declared_not_comparable, and caller_declared_not_run.", + "Replace migration parity output fields admittedParityEvidenceCount, matchedCount, mismatchedCount, notComparableCount, notRunCount, and admittedParityEvidenceRefs with admittedParityClaimCount, callerDeclaredMatchCount, callerDeclaredMismatchCount, callerDeclaredNotComparableCount, callerDeclaredNotRunCount, and admittedParityClaimRefs; consume each migrationParity record's required nonClaims projection.", + "Consume proofkit.requirement-authoring-plan.output.v2 with schemaVersion 2, preserve its required authoringRefs array, treat nonAuthoritativeAdmissionPreview.requirementSourcePreview as a canonical requirement-source projection in which absent optional fields are omitted, and handle candidateRequirement owner-admission failures as input errors before composition.", +} func validateCurrentChangeRecord(record Record, notes string) error { if !slices.Equal(record.BreakingChanges, currentBreakingChanges) { @@ -226,11 +240,13 @@ func validateCurrentChangeRecord(record Record, notes string) error { func currentExpectedReleaseNotes() string { lines := []string{ - "# @research-engineering/agentic-proofkit 0.2.1", + "# @research-engineering/agentic-proofkit 0.3.0", "", "## Breaking Contract Changes", "", - "- None.", + } + for _, change := range currentBreakingChanges { + lines = append(lines, currentChangeBullet(change)) } lines = append(lines, "", "## Additions", "") for _, change := range currentAdditions { @@ -240,7 +256,13 @@ func currentExpectedReleaseNotes() string { "", "## Migration", "", - "No consumer migration is required.", + "Migration is required:", + "", + ) + for _, step := range currentMigrationSteps { + lines = append(lines, "- "+step) + } + lines = append(lines, "", "## Platform Requirements", "", @@ -248,16 +270,16 @@ func currentExpectedReleaseNotes() string { "", "## Known Limitations", "", + "- Execution-backed semantic command evidence remains open under COVERAGE-01; declared routes prove mapping only.", + "- Requirement-source v2 representation and any textual DSL remain unselected until the typed-model and codec replacement gates pass.", "- TSX source parsing remains unsupported.", - "- Static route coverage does not prove semantic execution coverage.", - "- Release-history migration support starts at 0.2.0; no machine release-history chain is claimed for earlier releases.", "", "## Install", "", "Primary npm channel:", "", "```bash", - "npm install --save-dev --save-exact @research-engineering/agentic-proofkit@0.2.1", + "npm install --save-dev --save-exact @research-engineering/agentic-proofkit@0.3.0", "```", "", "Pre-1.0 npm consumers must keep this dependency exact-pinned.", @@ -268,7 +290,7 @@ func currentExpectedReleaseNotes() string { "", "## Rollback", "", - "- Pin npm consumers to the previous admitted version 0.2.0 with `npm install --save-dev --save-exact @research-engineering/agentic-proofkit@0.2.0`.", + "- Pin npm consumers to the previous admitted version 0.2.1 with `npm install --save-dev --save-exact @research-engineering/agentic-proofkit@0.2.1`.", "- Treat local package artifacts as candidates until registry identity is proven.", ) return strings.Join(lines, "\n") + "\n" diff --git a/internal/tools/releasecloseoutinput/main.go b/internal/tools/releasecloseoutinput/main.go index 3e37948..4957503 100644 --- a/internal/tools/releasecloseoutinput/main.go +++ b/internal/tools/releasecloseoutinput/main.go @@ -33,6 +33,7 @@ import ( const ( completionID = "proofkit.release_closeout.current_package_gate" ciProvenancePath = "artifacts/proofkit/ci-provenance.json" + cliContractPath = "proofkit/cli-contract.v2.json" coverageMetricsPath = "artifacts/proofkit/coverage-metrics.json" proofReceiptReportPath = "artifacts/proofkit/self-hosting-proof-receipt-admission-report.json" proofReceiptsPath = "artifacts/proofkit/self-hosting-proof-receipts.json" @@ -120,7 +121,7 @@ type cyclonedxBOM struct { type coverageMetricsEvidence struct { ArtifactKind string `json:"artifactKind"` - CLIContract map[string]any `json:"cliContract"` + CLIContract coverageCLIContractMetrics `json:"cliContract"` CommandRoutes coverageCommandRouteMetrics `json:"commandRoutes"` DeadZones coverageDeadZoneMetrics `json:"deadZones"` NonClaims []string `json:"nonClaims"` @@ -131,6 +132,19 @@ type coverageMetricsEvidence struct { WitnessPlan map[string]any `json:"witnessPlan"` } +type coverageCLIContractMetrics struct { + CommandCount *int `json:"commandCount"` + Commands *[]string `json:"commands"` +} + +type cliContractEvidence struct { + Commands []cliContractCommandEvidence `json:"commands"` +} + +type cliContractCommandEvidence struct { + Command string `json:"command"` +} + type coverageMetricsProvenance struct { GeneratedAt string `json:"generatedAt"` ProducerCommandID string `json:"producerCommandId"` @@ -139,26 +153,26 @@ type coverageMetricsProvenance struct { } type coverageCommandRouteMetrics struct { - AdmittedInventoryEntryCount *int `json:"admittedInventoryEntryCount"` - CommandCount *int `json:"commandCount"` - CommandWithoutProofRouteCandidateCount *int `json:"commandWithoutProofRouteCandidateCount"` - CommandsWithoutProofRouteCandidate *[]string `json:"commandsWithoutProofRouteCandidate"` - ContractOnlyCommandCount *int `json:"contractOnlyCommandCount"` - ContractOnlyCommands *[]string `json:"contractOnlyCommands"` - CommandWithoutSemanticFalsifierRouteCount *int `json:"commandWithoutSemanticFalsifierRouteCount"` - CommandsWithoutSemanticFalsifierRoute *[]string `json:"commandsWithoutSemanticFalsifierRoute"` - RouteCount *int `json:"routeCount"` - RouteOnlyCommandCount *int `json:"routeOnlyCommandCount"` - RouteOnlyCommands *[]string `json:"routeOnlyCommands"` - RouteSmokeCount *int `json:"routeSmokeCount"` - ProofRouteCandidateInventoryEntryCount *int `json:"proofRouteCandidateInventoryEntryCount"` - ProofRouteCandidateRouteCount *int `json:"proofRouteCandidateRouteCount"` - SemanticInventoryEntryCount *int `json:"semanticInventoryEntryCount"` - SemanticRouteCount *int `json:"semanticRouteCount"` - UnknownProofRouteCandidateRefCount *int `json:"unknownProofRouteCandidateRefCount"` - UnknownProofRouteCandidateRefs *[]string `json:"unknownProofRouteCandidateRefs"` - UnknownSemanticCommandRefCount *int `json:"unknownSemanticCommandRefCount"` - UnknownSemanticCommandRefs *[]string `json:"unknownSemanticCommandRefs"` + AdmittedInventoryEntryCount *int `json:"admittedInventoryEntryCount"` + CommandCount *int `json:"commandCount"` + Commands *[]string `json:"commands"` + CommandWithoutProofRouteCandidateCount *int `json:"commandWithoutProofRouteCandidateCount"` + CommandsWithoutProofRouteCandidate *[]string `json:"commandsWithoutProofRouteCandidate"` + ContractOnlyCommandCount *int `json:"contractOnlyCommandCount"` + ContractOnlyCommands *[]string `json:"contractOnlyCommands"` + CommandWithoutDeclaredSemanticFalsifierRouteCount *int `json:"commandWithoutDeclaredSemanticFalsifierRouteCount"` + CommandsWithoutDeclaredSemanticFalsifierRoute *[]string `json:"commandsWithoutDeclaredSemanticFalsifierRoute"` + RouteCount *int `json:"routeCount"` + RouteOnlyCommandCount *int `json:"routeOnlyCommandCount"` + RouteOnlyCommands *[]string `json:"routeOnlyCommands"` + RouteSmokeCount *int `json:"routeSmokeCount"` + ProofRouteCandidateInventoryEntryCount *int `json:"proofRouteCandidateInventoryEntryCount"` + ProofRouteCandidateRouteCount *int `json:"proofRouteCandidateRouteCount"` + DeclaredSemanticFalsifierRouteEntryCount *int `json:"declaredSemanticFalsifierRouteEntryCount"` + UnknownProofRouteCandidateRefCount *int `json:"unknownProofRouteCandidateRefCount"` + UnknownProofRouteCandidateRefs *[]string `json:"unknownProofRouteCandidateRefs"` + UnknownDeclaredSemanticRouteCommandRefCount *int `json:"unknownDeclaredSemanticRouteCommandRefCount"` + UnknownDeclaredSemanticRouteCommandRefs *[]string `json:"unknownDeclaredSemanticRouteCommandRefs"` } type coverageDeadZoneMetrics struct { @@ -228,6 +242,7 @@ type selfEvidenceDocument[T any] struct { // SelfEvidenceSnapshot is constructed once and exposes no mutable document state. type SelfEvidenceSnapshot struct { root string + cliContractCommands []string ciProvenance selfEvidenceDocument[map[string]any] coverageMetrics selfEvidenceDocument[coverageMetricsEvidence] execution packageartifactrecord.Record @@ -527,6 +542,14 @@ func selfEvidenceValid(root string) bool { } func readSelfEvidenceSnapshot(root string) (SelfEvidenceSnapshot, error) { + cliContract, err := readTypedJSON[cliContractEvidence](root, cliContractPath) + if err != nil { + return SelfEvidenceSnapshot{}, err + } + cliContractCommands, err := admittedCLIContractCommands(cliContract) + if err != nil { + return SelfEvidenceSnapshot{}, err + } ciProvenance, err := readSelfEvidenceDocument[map[string]any](root, ciProvenancePath) if err != nil { return SelfEvidenceSnapshot{}, err @@ -568,6 +591,7 @@ func readSelfEvidenceSnapshot(root string) (SelfEvidenceSnapshot, error) { } return SelfEvidenceSnapshot{ root: root, + cliContractCommands: cliContractCommands, ciProvenance: ciProvenance, coverageMetrics: coverageMetrics, execution: execution, @@ -593,7 +617,7 @@ func readSelfEvidenceDocument[T any](root string, path string) (selfEvidenceDocu } func (snapshot SelfEvidenceSnapshot) valid() bool { - return coverageMetricsRecordMatches(snapshot.coverageMetrics.value) && + return coverageMetricsRecordMatches(snapshot.coverageMetrics.value, snapshot.cliContractCommands) && coverageMetricsMatchExecution(snapshot.coverageMetrics.value, snapshot.execution) && proofReceiptReportMatchesDocument(snapshot.proofReceiptReport, snapshot.proofReceipts) && proofReceiptDocumentMatches(snapshot.proofReceipts) && @@ -854,16 +878,23 @@ func packageArtifactCommandDigest(execution packageartifactrecord.Record) (strin }) } -func coverageMetricsRecordMatches(record coverageMetricsEvidence) bool { +func coverageMetricsRecordMatches(record coverageMetricsEvidence, cliContractCommands []string) bool { return record.SchemaVersion == 1 && record.ArtifactKind == "proofkit.coverage-metrics.v1" && - len(record.CLIContract) > 0 && + coverageCLIContractMatches(record.CLIContract, cliContractCommands) && len(record.NonClaims) > 0 && len(record.ProofBindings) > 0 && len(record.Requirements) > 0 && len(record.WitnessPlan) > 0 && coverageDeadZonesEmpty(record.DeadZones) && - commandRouteDefectsEmpty(record.CommandRoutes) + commandRouteDefectsEmpty(record.CommandRoutes, cliContractCommands) +} + +func coverageCLIContractMatches(metrics coverageCLIContractMetrics, expected []string) bool { + return metrics.CommandCount != nil && metrics.Commands != nil && + *metrics.CommandCount == len(expected) && + reflect.DeepEqual(*metrics.Commands, expected) && + sortedUniqueStringSlice(metrics.Commands) } func reportRecordMatches(record selfEvidenceReport, wantKind string, wantID string, wantState string, wantRuleIDs []string) bool { @@ -955,27 +986,106 @@ func coverageDeadZonesEmpty(deadZones coverageDeadZoneMetrics) bool { emptyStringSlice(deadZones.ScenarioWithoutRequirementIDs) } -func commandRouteDefectsEmpty(routes coverageCommandRouteMetrics) bool { +func commandRouteDefectsEmpty(routes coverageCommandRouteMetrics, cliContractCommands []string) bool { return positiveInt(routes.AdmittedInventoryEntryCount) && positiveInt(routes.CommandCount) && positiveInt(routes.RouteCount) && nonNegativeInt(routes.RouteSmokeCount) && positiveInt(routes.ProofRouteCandidateInventoryEntryCount) && positiveInt(routes.ProofRouteCandidateRouteCount) && - nonNegativeInt(routes.SemanticInventoryEntryCount) && - nonNegativeInt(routes.SemanticRouteCount) && - nonNegativeInt(routes.CommandWithoutSemanticFalsifierRouteCount) && + nonNegativeInt(routes.DeclaredSemanticFalsifierRouteEntryCount) && + commandRouteMetricsProducerReachable(routes, cliContractCommands) && + countMatchesStringSlice(routes.CommandWithoutDeclaredSemanticFalsifierRouteCount, routes.CommandsWithoutDeclaredSemanticFalsifierRoute) && zeroInt(routes.CommandWithoutProofRouteCandidateCount) && zeroInt(routes.ContractOnlyCommandCount) && zeroInt(routes.RouteOnlyCommandCount) && zeroInt(routes.UnknownProofRouteCandidateRefCount) && - zeroInt(routes.UnknownSemanticCommandRefCount) && + zeroInt(routes.UnknownDeclaredSemanticRouteCommandRefCount) && emptyStringSlice(routes.CommandsWithoutProofRouteCandidate) && emptyStringSlice(routes.ContractOnlyCommands) && emptyStringSlice(routes.RouteOnlyCommands) && emptyStringSlice(routes.UnknownProofRouteCandidateRefs) && - nonNilStringSlice(routes.CommandsWithoutSemanticFalsifierRoute) && - emptyStringSlice(routes.UnknownSemanticCommandRefs) + emptyStringSlice(routes.UnknownDeclaredSemanticRouteCommandRefs) +} + +func commandRouteMetricsProducerReachable(routes coverageCommandRouteMetrics, cliContractCommands []string) bool { + if routes.AdmittedInventoryEntryCount == nil || + routes.CommandCount == nil || + routes.Commands == nil || + routes.CommandWithoutProofRouteCandidateCount == nil || + routes.CommandsWithoutDeclaredSemanticFalsifierRoute == nil || + routes.RouteCount == nil || + routes.RouteSmokeCount == nil || + routes.ProofRouteCandidateInventoryEntryCount == nil || + routes.ProofRouteCandidateRouteCount == nil || + routes.DeclaredSemanticFalsifierRouteEntryCount == nil || + routes.CommandWithoutDeclaredSemanticFalsifierRouteCount == nil { + return false + } + admitted := *routes.AdmittedInventoryEntryCount + commands := *routes.CommandCount + routeCount := *routes.RouteCount + routeSmokeCount := *routes.RouteSmokeCount + candidateEntries := *routes.ProofRouteCandidateInventoryEntryCount + candidateRoutes := *routes.ProofRouteCandidateRouteCount + declaredEntries := *routes.DeclaredSemanticFalsifierRouteEntryCount + missingCandidates := *routes.CommandWithoutProofRouteCandidateCount + missingDeclared := *routes.CommandWithoutDeclaredSemanticFalsifierRouteCount + if admitted < 0 || commands < 0 || routeCount < 0 || routeSmokeCount < 0 || + candidateEntries < 0 || candidateRoutes < 0 || declaredEntries < 0 || + missingCandidates < 0 || missingDeclared < 0 || + admitted != routeCount || candidateEntries != candidateRoutes || + candidateEntries > admitted || routeSmokeCount != admitted-candidateEntries || + declaredEntries != 0 || missingCandidates > commands || missingDeclared != commands || + candidateRoutes < commands-missingCandidates || len(*routes.Commands) != commands || + !reflect.DeepEqual(*routes.Commands, *routes.CommandsWithoutDeclaredSemanticFalsifierRoute) || + !reflect.DeepEqual(*routes.Commands, cliContractCommands) { + return false + } + return sortedUniqueStringSlice(routes.Commands) && + sortedUniqueStringSlice(routes.CommandsWithoutDeclaredSemanticFalsifierRoute) && + sortedUniqueStringSlice(routes.CommandsWithoutProofRouteCandidate) && + sortedUniqueStringSlice(routes.ContractOnlyCommands) && + sortedUniqueStringSlice(routes.RouteOnlyCommands) && + sortedUniqueStringSlice(routes.UnknownProofRouteCandidateRefs) && + sortedUniqueStringSlice(routes.UnknownDeclaredSemanticRouteCommandRefs) +} + +func admittedCLIContractCommands(contract cliContractEvidence) ([]string, error) { + if len(contract.Commands) == 0 { + return nil, fmt.Errorf("%s must declare at least one command", cliContractPath) + } + commands := make([]string, 0, len(contract.Commands)) + for index, command := range contract.Commands { + value, err := admit.RuleID(command.Command, fmt.Sprintf("%s commands[%d] command", cliContractPath, index)) + if err != nil { + return nil, err + } + commands = append(commands, value) + } + sort.Strings(commands) + for index := 1; index < len(commands); index++ { + if commands[index-1] == commands[index] { + return nil, fmt.Errorf("%s commands must be unique", cliContractPath) + } + } + return commands, nil +} + +func sortedUniqueStringSlice(values *[]string) bool { + if values == nil { + return false + } + for index, value := range *values { + if value == "" || (index > 0 && (*values)[index-1] >= value) { + return false + } + } + return true +} + +func countMatchesStringSlice(count *int, values *[]string) bool { + return count != nil && values != nil && *count >= 0 && *count == len(*values) } func positiveInt(value *int) bool { @@ -994,10 +1104,6 @@ func emptyStringSlice(value *[]string) bool { return value != nil && len(*value) == 0 } -func nonNilStringSlice(value *[]string) bool { - return value != nil -} - func numericValueIsZero(value any) bool { switch typed := value.(type) { case json.Number: diff --git a/internal/tools/releasecloseoutinput/main_test.go b/internal/tools/releasecloseoutinput/main_test.go index 4915d7a..3210e20 100644 --- a/internal/tools/releasecloseoutinput/main_test.go +++ b/internal/tools/releasecloseoutinput/main_test.go @@ -124,7 +124,7 @@ func TestCompleteFixtureSelfEvidenceSnapshotMatches(t *testing.T) { name string ok bool }{ - {name: "coverage metrics", ok: coverageMetricsRecordMatches(snapshot.coverageMetrics.value)}, + {name: "coverage metrics", ok: coverageMetricsRecordMatches(snapshot.coverageMetrics.value, snapshot.cliContractCommands)}, {name: "proof receipt report", ok: reportRecordMatches(snapshot.proofReceiptReport.value, "proofkit.proof-receipt-admission", "proofkit.self-hosting.proof-receipts", "passed", []string{"proofkit.proof-receipt-admission.boundary", "proofkit.proof-receipt-admission.receipts"})}, {name: "proof receipts", ok: proofReceiptDocumentMatches(snapshot.proofReceipts)}, {name: "producer report", ok: reportRecordMatches(snapshot.producerReport.value, "proofkit.receipt-producer-admission", "proofkit.receipt-producer-policy", "passed", []string{"proofkit.receipt-producer-admission.boundary", "proofkit.receipt-producer-admission.coverage", "proofkit.receipt-producer-admission.receipts"})}, @@ -179,6 +179,9 @@ func TestBuildInputRejectsMixedSelfEvidenceIdentity(t *testing.T) { } func TestBuildInputFailsClosedForEachBlockingEvidenceClass(t *testing.T) { + if !commandRouteMetricsProducerReachable(producerReachableCommandRouteMetricsFixture(), producerReachableCLIContractCommands()) { + t.Fatal("release closeout rejected a coverage-metrics producer-reachable count relation") + } cases := []struct { name string criterionID string @@ -554,6 +557,165 @@ func TestBuildInputFailsClosedForEachBlockingEvidenceClass(t *testing.T) { writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) }, }, + { + name: "self evidence coverage with contradictory declared route count", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["commandWithoutDeclaredSemanticFalsifierRouteCount"] = 0 + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{"ghost"} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with substituted declared-route command identity", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + record["commandRoutes"].(map[string]any)["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{"proofkit.command.substituted"} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with coordinated alien command inventories", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + alien := []any{"proofkit.command.substituted"} + record["cliContract"].(map[string]any)["commands"] = alien + routes := record["commandRoutes"].(map[string]any) + routes["commands"] = alien + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = alien + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with inventory route count mismatch", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + record["commandRoutes"].(map[string]any)["admittedInventoryEntryCount"] = 2 + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with candidate projection mismatch", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + record["commandRoutes"].(map[string]any)["proofRouteCandidateInventoryEntryCount"] = 0 + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with impossible inventory class partition", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["routeSmokeCount"] = 1 + routes["declaredSemanticFalsifierRouteEntryCount"] = 1 + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with declared entries unavailable from producer", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + record["commandRoutes"].(map[string]any)["declaredSemanticFalsifierRouteEntryCount"] = 1 + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with candidate routes unable to cover commands", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["commandCount"] = 2 + routes["commands"] = []any{"proofkit.command.a", "proofkit.command.b"} + routes["commandWithoutDeclaredSemanticFalsifierRouteCount"] = 2 + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{"proofkit.command.a", "proofkit.command.b"} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with overflow-shaped route partition", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + maximum := int(^uint(0) >> 1) + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["admittedInventoryEntryCount"] = maximum + routes["routeCount"] = maximum + routes["routeSmokeCount"] = maximum + routes["proofRouteCandidateInventoryEntryCount"] = maximum + routes["proofRouteCandidateRouteCount"] = maximum + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with candidate routes above total routes", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["proofRouteCandidateRouteCount"] = 2 + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with incomplete declared route missing set", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["commandWithoutDeclaredSemanticFalsifierRouteCount"] = 0 + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with missing declared routes above command count", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["commandCount"] = 1 + routes["commands"] = []any{"proofkit.command.a"} + routes["commandWithoutDeclaredSemanticFalsifierRouteCount"] = 2 + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{"proofkit.command.a", "proofkit.command.b"} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with duplicate missing declared route ids", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["commandCount"] = 2 + routes["commands"] = []any{"proofkit.command.a", "proofkit.command.b"} + routes["commandWithoutDeclaredSemanticFalsifierRouteCount"] = 2 + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{"proofkit.command.a", "proofkit.command.a"} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, + { + name: "self evidence coverage with unsorted missing declared route ids", + criterionID: "proofkit.release_closeout.self_evidence", + mutate: func(root string) { + record := coverageMetricsFixture() + routes := record["commandRoutes"].(map[string]any) + routes["commandCount"] = 2 + routes["commands"] = []any{"proofkit.command.a", "proofkit.command.b"} + routes["commandWithoutDeclaredSemanticFalsifierRouteCount"] = 2 + routes["commandsWithoutDeclaredSemanticFalsifierRoute"] = []any{"proofkit.command.b", "proofkit.command.a"} + writeJSON(t, filepath.Join(root, "artifacts/proofkit/coverage-metrics.json"), record) + }, + }, { name: "self evidence arbitrary receipt object", criterionID: "proofkit.release_closeout.self_evidence", @@ -646,6 +808,96 @@ func TestBuildInputFailsClosedForEachBlockingEvidenceClass(t *testing.T) { } } +func TestCommandRouteMetricsProducerReachabilityMatchesExactProducerPartition(t *testing.T) { + routes := producerReachableCommandRouteMetricsFixture() + expectedCommands := producerReachableCLIContractCommands() + if !commandRouteMetricsProducerReachable(routes, expectedCommands) { + t.Fatal("producer-reachable route partition was rejected") + } + mutations := []struct { + name string + mutate func(*coverageCommandRouteMetrics) + }{ + {name: "inventory route mismatch", mutate: func(value *coverageCommandRouteMetrics) { value.AdmittedInventoryEntryCount = testIntPointer(4) }}, + {name: "missing command inventory", mutate: func(value *coverageCommandRouteMetrics) { value.Commands = nil }}, + {name: "source contract mismatch", mutate: func(value *coverageCommandRouteMetrics) { + commands := []string{"proofkit.cli.a", "proofkit.cli.c"} + value.Commands = &commands + value.CommandsWithoutDeclaredSemanticFalsifierRoute = &commands + }}, + {name: "missing candidate count", mutate: func(value *coverageCommandRouteMetrics) { value.CommandWithoutProofRouteCandidateCount = nil }}, + {name: "missing declared command identities", mutate: func(value *coverageCommandRouteMetrics) { value.CommandsWithoutDeclaredSemanticFalsifierRoute = nil }}, + {name: "candidate projection mismatch", mutate: func(value *coverageCommandRouteMetrics) { + value.ProofRouteCandidateInventoryEntryCount = testIntPointer(1) + }}, + {name: "route partition mismatch", mutate: func(value *coverageCommandRouteMetrics) { value.RouteSmokeCount = testIntPointer(2) }}, + {name: "unreachable declared entry", mutate: func(value *coverageCommandRouteMetrics) { + value.DeclaredSemanticFalsifierRouteEntryCount = testIntPointer(1) + }}, + {name: "declared missing partition mismatch", mutate: func(value *coverageCommandRouteMetrics) { + value.CommandWithoutDeclaredSemanticFalsifierRouteCount = testIntPointer(1) + }}, + {name: "declared missing command identity mismatch", mutate: func(value *coverageCommandRouteMetrics) { + commands := []string{"arbitrary.a", "arbitrary.z"} + value.CommandsWithoutDeclaredSemanticFalsifierRoute = &commands + }}, + {name: "candidate route cannot cover commands", mutate: func(value *coverageCommandRouteMetrics) { value.CommandCount = testIntPointer(3) }}, + {name: "overflow-shaped counts", mutate: func(value *coverageCommandRouteMetrics) { + maximum := int(^uint(0) >> 1) + value.AdmittedInventoryEntryCount = testIntPointer(maximum) + value.RouteCount = testIntPointer(maximum) + value.RouteSmokeCount = testIntPointer(maximum) + value.ProofRouteCandidateInventoryEntryCount = testIntPointer(maximum) + value.ProofRouteCandidateRouteCount = testIntPointer(maximum) + }}, + } + for _, item := range mutations { + t.Run(item.name, func(t *testing.T) { + mutated := producerReachableCommandRouteMetricsFixture() + item.mutate(&mutated) + if commandRouteMetricsProducerReachable(mutated, expectedCommands) { + t.Fatal("producer-impossible command-route metrics were admitted") + } + }) + } +} + +func producerReachableCLIContractCommands() []string { + return []string{"proofkit.cli.a", "proofkit.cli.b"} +} + +func producerReachableCommandRouteMetricsFixture() coverageCommandRouteMetrics { + empty := []string{} + missingDeclared := []string{"proofkit.cli.a", "proofkit.cli.b"} + commands := []string{"proofkit.cli.a", "proofkit.cli.b"} + return coverageCommandRouteMetrics{ + AdmittedInventoryEntryCount: testIntPointer(3), + CommandCount: testIntPointer(2), + Commands: &commands, + CommandWithoutProofRouteCandidateCount: testIntPointer(0), + CommandsWithoutProofRouteCandidate: &empty, + ContractOnlyCommandCount: testIntPointer(0), + ContractOnlyCommands: &empty, + CommandWithoutDeclaredSemanticFalsifierRouteCount: testIntPointer(2), + CommandsWithoutDeclaredSemanticFalsifierRoute: &missingDeclared, + RouteCount: testIntPointer(3), + RouteOnlyCommandCount: testIntPointer(0), + RouteOnlyCommands: &empty, + RouteSmokeCount: testIntPointer(1), + ProofRouteCandidateInventoryEntryCount: testIntPointer(2), + ProofRouteCandidateRouteCount: testIntPointer(2), + DeclaredSemanticFalsifierRouteEntryCount: testIntPointer(0), + UnknownProofRouteCandidateRefCount: testIntPointer(0), + UnknownProofRouteCandidateRefs: &empty, + UnknownDeclaredSemanticRouteCommandRefCount: testIntPointer(0), + UnknownDeclaredSemanticRouteCommandRefs: &empty, + } +} + +func testIntPointer(value int) *int { + return &value +} + func TestBuildInputAdmitsWorkflowPublicationTrustedPublisherIdentity(t *testing.T) { root := completeFixture(t) writeReleaseManifest(t, root, releaseManifestFixtureWithNpmPublication("published_by_workflow", trustedPublisherFixture("npm-production", "publish"))) @@ -928,6 +1180,9 @@ func populateCompleteFixture(t *testing.T, root string) { writeFile(t, filepath.Join(root, "go.sum"), "") writeJSON(t, filepath.Join(root, "proofkit", "requirement-bindings.json"), map[string]any{"schemaVersion": 1}) writeJSON(t, filepath.Join(root, "proofkit", "witness-plan.json"), map[string]any{"schemaVersion": 1}) + writeJSON(t, filepath.Join(root, filepath.FromSlash(cliContractPath)), map[string]any{ + "commands": []any{map[string]any{"command": "proofkit.coverage.command"}}, + }) writeFile(t, filepath.Join(root, "source.txt"), "source-v1\n") writeJSON(t, filepath.Join(root, "package.json"), map[string]any{ "name": testNPMPackageName, @@ -1202,28 +1457,28 @@ func coverageMetricsFixture() map[string]any { "requirements": map[string]any{"blocking": 1}, "proofBindings": map[string]any{"boundRequirementCount": 1}, "witnessPlan": map[string]any{"commandCount": 1}, - "cliContract": map[string]any{"commandCount": 1}, + "cliContract": map[string]any{"commandCount": 1, "commands": []any{"proofkit.coverage.command"}}, "commandRoutes": map[string]any{ - "admittedInventoryEntryCount": 1, - "commandCount": 1, - "commandWithoutProofRouteCandidateCount": 0, - "commandsWithoutProofRouteCandidate": []any{}, - "commandWithoutSemanticFalsifierRouteCount": 0, - "commandsWithoutSemanticFalsifierRoute": []any{}, - "contractOnlyCommandCount": 0, - "contractOnlyCommands": []any{}, - "routeCount": 1, - "routeOnlyCommandCount": 0, - "routeOnlyCommands": []any{}, - "routeSmokeCount": 0, - "proofRouteCandidateInventoryEntryCount": 1, - "proofRouteCandidateRouteCount": 1, - "semanticInventoryEntryCount": 1, - "semanticRouteCount": 1, - "unknownProofRouteCandidateRefCount": 0, - "unknownProofRouteCandidateRefs": []any{}, - "unknownSemanticCommandRefCount": 0, - "unknownSemanticCommandRefs": []any{}, + "admittedInventoryEntryCount": 1, + "commandCount": 1, + "commands": []any{"proofkit.coverage.command"}, + "commandWithoutProofRouteCandidateCount": 0, + "commandsWithoutProofRouteCandidate": []any{}, + "commandWithoutDeclaredSemanticFalsifierRouteCount": 1, + "commandsWithoutDeclaredSemanticFalsifierRoute": []any{"proofkit.coverage.command"}, + "contractOnlyCommandCount": 0, + "contractOnlyCommands": []any{}, + "routeCount": 1, + "routeOnlyCommandCount": 0, + "routeOnlyCommands": []any{}, + "routeSmokeCount": 0, + "proofRouteCandidateInventoryEntryCount": 1, + "proofRouteCandidateRouteCount": 1, + "declaredSemanticFalsifierRouteEntryCount": 0, + "unknownProofRouteCandidateRefCount": 0, + "unknownProofRouteCandidateRefs": []any{}, + "unknownDeclaredSemanticRouteCommandRefCount": 0, + "unknownDeclaredSemanticRouteCommandRefs": []any{}, }, "deadZones": map[string]any{ "bindingWithoutRequirementIds": []any{}, diff --git a/package-lock.json b/package-lock.json index 941ab42..4cb186c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@research-engineering/agentic-proofkit", - "version": "0.2.1", + "version": "0.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@research-engineering/agentic-proofkit", - "version": "0.2.1", + "version": "0.3.0", "cpu": [ "arm64", "x64" diff --git a/package.json b/package.json index e40b490..06e30aa 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@research-engineering/agentic-proofkit", "description": "Reusable proof profile, report, graph, and witness-planning primitives.", - "version": "0.2.1", + "version": "0.3.0", "type": "module", "license": "MIT", "sideEffects": false, diff --git a/proofkit/cli-contract.v2.json b/proofkit/cli-contract.v2.json index 1e87311..5a99633 100644 --- a/proofkit/cli-contract.v2.json +++ b/proofkit/cli-contract.v2.json @@ -510,7 +510,7 @@ "rootDefinitionDigest": "sha256:fd1121285d1b357ed848febc248700a75417d06619e22ed36991631d7526375b", "nativeSource": { "path": "internal/command/agentroute", - "canonicalDigest": "sha256:bcbf099540f203e32c0b6d9cb38dff0dc20e8018644569d647d2979d0d74ac15", + "canonicalDigest": "sha256:21ffaf8d53060e31f3dec92d7f39cf6e7654cc2268bde47599211d1d914acfeb", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -586,7 +586,7 @@ "rootDefinitionDigest": "sha256:db7d5b29083a939eea0f23253632394212b0d73b5f1b6206d0811d236d6a84ca", "nativeSource": { "path": "internal/command/agentroute", - "canonicalDigest": "sha256:bcbf099540f203e32c0b6d9cb38dff0dc20e8018644569d647d2979d0d74ac15", + "canonicalDigest": "sha256:21ffaf8d53060e31f3dec92d7f39cf6e7654cc2268bde47599211d1d914acfeb", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -1276,7 +1276,7 @@ "rootDefinitionDigest": "sha256:7ffe236ab203cf411dbfda6008ec8c10cc15691261031d4ed9ed90d81525dc12", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -1304,7 +1304,7 @@ "rootDefinitionDigest": "sha256:2be3545f9329bb4e4ba4971fc04bf5811959cf0087fd7ab3a7c2c8b45b8d714a", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -1840,7 +1840,7 @@ "inputContract": { "contractId": "proofkit.migration-parity-admission.input.v1", "schemaVersion": 1, - "authority": "caller-owned migration parity evidence", + "authority": "caller-owned migration parity claims", "requiredFields": [ "nonClaims", "parityRecords", @@ -1856,13 +1856,13 @@ "rootDefinitionDigest": "sha256:a1f8ab777249d7ff613bfb3256cd96514235ac90d02ad39ce8720330cbd71d38", "nativeSource": { "path": "internal/command/migrationparityadmission", - "canonicalDigest": "sha256:e6d7db500dea081cbf5c50a0b56051c3f5f193d5dbcc314871cd2bae0c0b3e6a", + "canonicalDigest": "sha256:37ea4422b751635bfd27f1afef8e7815b3efec2a8f71f96a7074e6185a18ee5b", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { "path": "internal/command/migrationparityadmission/migrationparityadmission_test.go", - "test": "TestBuildAdmitsMatchedParityAndRejectsDigestDrift", - "command": "go test ./internal/command/migrationparityadmission -run '^TestBuildAdmitsMatchedParityAndRejectsDigestDrift$'", + "test": "TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift", + "command": "go test ./internal/command/migrationparityadmission -run '^TestBuildAdmitsCallerDeclaredMatchAndRejectsDigestDrift$'", "evidenceClass": "source_checkout" }, "compatibilitySummary": [ @@ -1888,13 +1888,13 @@ "rootDefinitionDigest": "sha256:aaea5c7d9d12e1fbc406d080dcda5aca84351c5a2048d40535a1b3895cc37440", "nativeSource": { "path": "internal/command/migrationparityadmission", - "canonicalDigest": "sha256:e6d7db500dea081cbf5c50a0b56051c3f5f193d5dbcc314871cd2bae0c0b3e6a", + "canonicalDigest": "sha256:37ea4422b751635bfd27f1afef8e7815b3efec2a8f71f96a7074e6185a18ee5b", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { "path": "internal/command/migrationparityadmission/migrationparityadmission_test.go", - "test": "TestBuildAdmitsMatchedParityAndRejectsDigestDrift", - "command": "go test ./internal/command/migrationparityadmission -run '^TestBuildAdmitsMatchedParityAndRejectsDigestDrift$'", + "test": "TestBuildProjectsEveryCallerDeclaredStatusAndSummaryField", + "command": "go test ./internal/command/migrationparityadmission -run '^TestBuildProjectsEveryCallerDeclaredStatusAndSummaryField$'", "evidenceClass": "source_checkout" }, "compatibilitySummary": [ @@ -2231,7 +2231,7 @@ "nativeSources": [ { "path": "internal/app", - "canonicalDigest": "sha256:0bea47d33304e528ffbe087688ea2dee07cf42e55632eda3b479732e221841d9", + "canonicalDigest": "sha256:b106e4c87900417f84f8bd999ddc67ac1d47bc54c2cb0bcb0e85993433761c82", "evidenceClass": "source_checkout" }, { @@ -2486,7 +2486,7 @@ "rootDefinitionDigest": "sha256:71914c95c70dbace0ebce1dc4609624e3b91c994cd8c5adc20ed84d4200f0ac9", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -2514,7 +2514,7 @@ "rootDefinitionDigest": "sha256:5fbdf25c90a2020525f97453c9d9fe5a774fe0e2eacf00d1b2aea3ff5f4fa0f0", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -3196,7 +3196,7 @@ "rootDefinitionDigest": "sha256:86f8f4913b5316caa63c40d79125199b282d108cbd8b9711db21c2023d1a4ca4", "nativeSource": { "path": "internal/command/requirementauthoringplan", - "canonicalDigest": "sha256:91d82aa48a4c10b33f89b7b2db7443182b7702de4b0c2763a8104f2b0a997bdf", + "canonicalDigest": "sha256:ec094da2d08e360992266402ec87fc361ef6b34023a91139015493d277bd3e76", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -3216,26 +3216,26 @@ ] }, "outputContract": { - "contractId": "proofkit.requirement-authoring-plan.output.v1", - "schemaVersion": 1, + "contractId": "proofkit.requirement-authoring-plan.output.v2", + "schemaVersion": 2, "rootType": "object", "closed": true, - "rootDefinitionRef": "proofkit.requirement-authoring-plan.output.v1.root-shape", - "rootDefinitionDigest": "sha256:019c920b72a4d2fba8f6364bff554442b0194ac42364d645d5b0b71f28df24b1", + "rootDefinitionRef": "proofkit.requirement-authoring-plan.output.v2.root-shape", + "rootDefinitionDigest": "sha256:c2264058757e40789005b7cf7417906461f064b9b9bca0c5ddda86704e3b36cc", "nativeSource": { "path": "internal/command/requirementauthoringplan", - "canonicalDigest": "sha256:91d82aa48a4c10b33f89b7b2db7443182b7702de4b0c2763a8104f2b0a997bdf", + "canonicalDigest": "sha256:ec094da2d08e360992266402ec87fc361ef6b34023a91139015493d277bd3e76", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { - "path": "internal/command/requirementauthoringplan/requirement_authoring_plan_test.go", - "test": "TestBuildRejectsCandidateSourceAdmissionFailure", - "command": "go test ./internal/command/requirementauthoringplan -run '^TestBuildRejectsCandidateSourceAdmissionFailure$'", + "path": "internal/app/cli_abi_test.go", + "test": "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", + "command": "go test ./internal/app -run '^TestRequirementAuthoringPlanOutputUsesVersionedRootShape$'", "evidenceClass": "source_checkout" }, "compatibilitySummary": [ - "schemaVersion=1", - "root-shape-only definition proofkit.requirement-authoring-plan.output.v1.root-shape; nested fields, types, and cardinalities are non-claims" + "schemaVersion=2", + "root-shape-only definition proofkit.requirement-authoring-plan.output.v2.root-shape; nested fields, types, and cardinalities are non-claims" ], "ownerRequirementRefs": [ "REQ-PROOFKIT-PACKAGE-002", @@ -3266,7 +3266,7 @@ "rootDefinitionDigest": "sha256:e7da70a2267f2ca13771e864bba6a1a0416132c2d56decb5db9f9694de021daf", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -3294,7 +3294,7 @@ "rootDefinitionDigest": "sha256:f31dd686f28fce4bf74265b557dc25428179cbc282753b4bea01439cd98f73c2", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -3878,8 +3878,8 @@ "--format" ], "outputContract": { - "contractId": "proofkit.requirement-coverage-view.output.v1", - "schemaVersion": 1, + "contractId": "proofkit.requirement-coverage-view.output.v2", + "schemaVersion": 2, "authority": "requirement coverage view output contract", "classificationRecords": { "failureClassifications": { @@ -3895,15 +3895,18 @@ "declared_dead_zone", "failed_test_inventory", "missing_requirement_binding", - "missing_semantic_test", + "missing_declared_test_route", "nonsemantic_command_evidence", "nonsemantic_governance_evidence", "not_applicable_with_reason", "owner_scope_violation", + "proof_route_candidate_only", "routing_smoke_only", + "unclassified_gap", "unknown_reference" ], "requiredFields": [ + "coverageBasis", "failureClassifications", "failures", "ownerInvariantCoverage", @@ -3912,15 +3915,16 @@ "warnings" ], "nonClaims": [ - "Classification records describe Proofkit coverage diagnostics only; consumer repositories own severity policy, merge admission, escalation, and native witness execution." + "Classification records describe Proofkit coverage diagnostics only; consumer repositories own severity policy, merge admission, escalation, and native witness execution.", + "Coverage basis commitments prove internal projection consistency only; they do not authenticate source inputs or producer identity." ], "rootType": "object", "closed": true, - "rootDefinitionRef": "proofkit.requirement-coverage-view.output.v1.root-shape", - "rootDefinitionDigest": "sha256:697d24d12d4772b67283f24da2f265372de241bd4741ac23f7b984ca96023818", + "rootDefinitionRef": "proofkit.requirement-coverage-view.output.v2.root-shape", + "rootDefinitionDigest": "sha256:71a3a1245159e03eb3f81e2b117e1d1690c99e7e84625670a1fd42530b53684c", "nativeSource": { "path": "internal/command/requirementcoverageview", - "canonicalDigest": "sha256:c2a999a43db2e7b9ed65a599865bd291c7a0a4daf64973aefb40e684dee34c5a", + "canonicalDigest": "sha256:a6f804ad55b0101c934c782468c217ab8f3ce0cc42828975fbc35621bd43bc97", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -3930,13 +3934,14 @@ "evidenceClass": "source_checkout" }, "compatibilitySummary": [ + "coverageBasis", "failureClassifications", "failures", "ownerInvariantCoverage", "state", "warningClassifications", "warnings", - "root-shape-only definition proofkit.requirement-coverage-view.output.v1.root-shape; nested fields, types, and cardinalities are non-claims" + "root-shape-only definition proofkit.requirement-coverage-view.output.v2.root-shape; nested fields, types, and cardinalities are non-claims" ], "ownerRequirementRefs": [ "REQ-PROOFKIT-PACKAGE-002", @@ -3953,7 +3958,7 @@ "rootDefinitionDigest": "sha256:b5ddab0a55a381f002b39ece0f5727171f7984856e553200bbb62a49821ff6ac", "nativeSource": { "path": "internal/command/requirementcoverageview", - "canonicalDigest": "sha256:c2a999a43db2e7b9ed65a599865bd291c7a0a4daf64973aefb40e684dee34c5a", + "canonicalDigest": "sha256:a6f804ad55b0101c934c782468c217ab8f3ce0cc42828975fbc35621bd43bc97", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -4111,7 +4116,7 @@ "rootDefinitionDigest": "sha256:68ade17ac7e9fb3496ad0e7ea8ae9fbcf734afa4e6b8a44442e1af3c4f87bf15", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -4138,7 +4143,7 @@ "rootDefinitionDigest": "sha256:750c75c08376a986a18e8739b7476d58af4133c19c40f43d216ee009607be225", "nativeSource": { "path": "internal/command/requirementbinding", - "canonicalDigest": "sha256:90bada8c868751cef16683475839174de48dd18936e3d79a74e24febcda1e09b", + "canonicalDigest": "sha256:c1382a5e461974baf8e5a45d95e99552c241ea16590b1e1a781748c331cb5ffe", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -4464,7 +4469,7 @@ "rootDefinitionDigest": "sha256:b2b3982ca053794da607673db6258933960a4f67653871205ff03caab2012a9f", "nativeSource": { "path": "internal/command/requirementsourceadmission", - "canonicalDigest": "sha256:5767016688b34b4843a4c2da3f817b354c2ade0e9dfcfae5009b8fbf7cf5f76d", + "canonicalDigest": "sha256:d68c56d1566166937da8ca3f2e0afe302d21a9349a0b520322a434ab88270b30", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -4491,7 +4496,7 @@ "rootDefinitionDigest": "sha256:2527a19394b22efb4b14e39bf3b1f14283f179592c369b9c2ae34b612c7c69af", "nativeSource": { "path": "internal/command/requirementsourceadmission", - "canonicalDigest": "sha256:5767016688b34b4843a4c2da3f817b354c2ade0e9dfcfae5009b8fbf7cf5f76d", + "canonicalDigest": "sha256:d68c56d1566166937da8ca3f2e0afe302d21a9349a0b520322a434ab88270b30", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -5465,7 +5470,7 @@ "rootDefinitionDigest": "sha256:3c842174dff5361e7f83166469b832805e05aa314b073c16234b5b64e346281e", "nativeSource": { "path": "internal/app", - "canonicalDigest": "sha256:0bea47d33304e528ffbe087688ea2dee07cf42e55632eda3b479732e221841d9", + "canonicalDigest": "sha256:b106e4c87900417f84f8bd999ddc67ac1d47bc54c2cb0bcb0e85993433761c82", "evidenceClass": "source_checkout" }, "nativeAdmissionWitnessSelector": { @@ -5494,7 +5499,7 @@ "rootDefinitionDigest": "sha256:0ea95e277ebe44cd2de42c29b47c38686ac0b6b390d8965367437b3fe138e209", "nativeSource": { "path": "internal/app", - "canonicalDigest": "sha256:0bea47d33304e528ffbe087688ea2dee07cf42e55632eda3b479732e221841d9", + "canonicalDigest": "sha256:b106e4c87900417f84f8bd999ddc67ac1d47bc54c2cb0bcb0e85993433761c82", "evidenceClass": "source_checkout" }, "nativeOutputWitnessSelector": { @@ -5770,7 +5775,7 @@ "candidateKind": "proofkit.test-inventory-discovery-draft.candidate-inventory", "strictInventoryAdmission": "rejected_until_owner_materialized" }, - "nonClaim": "Discovery draft reports candidate-only guidance from explicit caller-owned facts; it does not scan repositories, execute tests, emit semantic_falsifier evidence, or close coverage." + "nonClaim": "Discovery draft reports candidate-only guidance from explicit caller-owned facts; it does not scan repositories, execute tests, emit execution-backed semantic evidence, or close coverage." }, "proof-binding-derived": { "authority": "proof-binding-derived test inventory projection admission", @@ -5845,14 +5850,14 @@ }, { "path": "internal/command/testevidenceinventory", - "canonicalDigest": "sha256:04febdd7e002567d50bb16fded119ef6ca0ce0d2eb063c92b1664312652ac395", + "canonicalDigest": "sha256:95618ec2824e5510d55d2c6bb86da69b057679a39404cca5f2ab5c8fd4826c6a", "evidenceClass": "source_checkout" } ], "nativeAdmissionWitnessSelector": { "path": "internal/command/testevidenceinventory/testevidenceinventory_test.go", - "test": "TestBuildRejectsWeakOracleAndDuplicateFalsifier", - "command": "go test ./internal/command/testevidenceinventory -run '^TestBuildRejectsWeakOracleAndDuplicateFalsifier$'", + "test": "TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier", + "command": "go test ./internal/command/testevidenceinventory -run '^TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier$'", "evidenceClass": "source_checkout" }, "compatibilitySummary": [ @@ -5899,7 +5904,7 @@ "invalid_falsifier_supersession", "missing_edge", "missing_executable_command_ref", - "missing_semantic_anchor", + "missing_declared_route_anchor", "mock_tests_mock", "over_broad_integration", "proof_route_candidate", @@ -5908,7 +5913,7 @@ "snapshot_without_oracle", "tautology", "unasserted_diagnostic", - "weak_or_empty_oracle", + "incomplete_declared_oracle_metadata", "wrong_boundary", "wrong_evidence_boundary" ], @@ -5921,8 +5926,8 @@ "severity": "failure|warning" }, "oracleFields": { - "assertionSummary": "non-empty assertion summary for semantic_falsifier, contract_admission, and property_or_fuzz entries", - "expectedPublicOutcome": "non-empty expected public outcome for semantic_falsifier, contract_admission, and property_or_fuzz entries", + "assertionSummary": "non-empty caller-authored assertion summary for declared semantic-falsifier, contract-admission, and property-or-fuzz routes", + "expectedPublicOutcome": "non-empty caller-authored expected public outcome for declared semantic-falsifier, contract-admission, and property-or-fuzz routes", "oracleId": "stable rule id", "oracleKind": "stable rule id" }, @@ -5942,14 +5947,14 @@ }, { "path": "internal/command/testevidenceinventory", - "canonicalDigest": "sha256:04febdd7e002567d50bb16fded119ef6ca0ce0d2eb063c92b1664312652ac395", + "canonicalDigest": "sha256:95618ec2824e5510d55d2c6bb86da69b057679a39404cca5f2ab5c8fd4826c6a", "evidenceClass": "source_checkout" } ], "nativeOutputWitnessSelector": { "path": "internal/command/testevidenceinventory/testevidenceinventory_test.go", - "test": "TestBuildRejectsWeakOracleAndDuplicateFalsifier", - "command": "go test ./internal/command/testevidenceinventory -run '^TestBuildRejectsWeakOracleAndDuplicateFalsifier$'", + "test": "TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier", + "command": "go test ./internal/command/testevidenceinventory -run '^TestBuildRejectsIncompleteDeclaredOracleMetadataAndDuplicateFalsifier$'", "evidenceClass": "source_checkout" }, "compatibilitySummary": [ @@ -11109,8 +11114,8 @@ "canonicalDigest": "sha256:86f8f4913b5316caa63c40d79125199b282d108cbd8b9711db21c2023d1a4ca4" }, { - "definitionId": "proofkit.requirement-authoring-plan.output.v1.root-shape", - "schemaVersion": 1, + "definitionId": "proofkit.requirement-authoring-plan.output.v2.root-shape", + "schemaVersion": 2, "rootType": "object", "closed": true, "definitionRefs": [], @@ -11124,6 +11129,7 @@ { "allowedFields": [ "authoringPlanId", + "authoringRefs", "candidateChangeSet", "mode", "nonAuthoritativeAdmissionPreview", @@ -11138,6 +11144,7 @@ ], "requiredFields": [ "authoringPlanId", + "authoringRefs", "candidateChangeSet", "mode", "nonAuthoritativeAdmissionPreview", @@ -11158,7 +11165,7 @@ } ] }, - "canonicalDigest": "sha256:019c920b72a4d2fba8f6364bff554442b0194ac42364d645d5b0b71f28df24b1" + "canonicalDigest": "sha256:c2264058757e40789005b7cf7417906461f064b9b9bca0c5ddda86704e3b36cc" }, { "definitionId": "proofkit.requirement-bindings.input.v1.root-shape", @@ -11886,8 +11893,8 @@ "canonicalDigest": "sha256:b5ddab0a55a381f002b39ece0f5727171f7984856e553200bbb62a49821ff6ac" }, { - "definitionId": "proofkit.requirement-coverage-view.output.v1.root-shape", - "schemaVersion": 1, + "definitionId": "proofkit.requirement-coverage-view.output.v2.root-shape", + "schemaVersion": 2, "rootType": "object", "closed": true, "definitionRefs": [], @@ -11945,6 +11952,7 @@ "commandCoverageCount", "completenessDeclaration", "contractId", + "coverageBasis", "coverageUniverseId", "deadZones", "failureClassifications", @@ -11962,6 +11970,7 @@ "sourceId", "state", "testInventoryId", + "unmappedTests", "viewInputId", "viewKind", "warningClassifications", @@ -11975,6 +11984,7 @@ "commandCoverageCount", "completenessDeclaration", "contractId", + "coverageBasis", "coverageUniverseId", "deadZones", "failureClassifications", @@ -11992,6 +12002,7 @@ "sourceId", "state", "testInventoryId", + "unmappedTests", "viewInputId", "viewKind", "warningClassifications", @@ -12006,7 +12017,7 @@ } ] }, - "canonicalDigest": "sha256:697d24d12d4772b67283f24da2f265372de241bd4741ac23f7b984ca96023818" + "canonicalDigest": "sha256:71a3a1245159e03eb3f81e2b117e1d1690c99e7e84625670a1fd42530b53684c" }, { "definitionId": "proofkit.requirement-impact-input-compose.input.v1.root-shape", diff --git a/proofkit/requirement-bindings.json b/proofkit/requirement-bindings.json index f42266b..f306044 100644 --- a/proofkit/requirement-bindings.json +++ b/proofkit/requirement-bindings.json @@ -129,7 +129,7 @@ "claimLevel": "blocking", "proofState": "witness_backed", "nonClaims": [ - "This requirement does not claim repository scanning, native test execution, proof freshness, receipt authenticity, merge approval, or global coverage completeness outside the caller-provided universe." + "This requirement does not claim repository scanning, source authenticity, producer identity, selector resolution, oracle quality, native test execution, proof freshness, receipt authenticity, trust, sensitivity, assurance closure, merge approval, or global coverage completeness outside the caller-provided universe." ] }, { @@ -988,6 +988,10 @@ "selector": "TestAdoptionContractEnvelopeCLIABI", "command": "go test ./internal/app -run '^TestAdoptionContractEnvelopeCLIABI$'" }, + { + "selector": "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", + "command": "go test ./internal/app -run '^TestRequirementAuthoringPlanOutputUsesVersionedRootShape$'" + }, { "selector": "TestSelfCheckOutputUsesExactRootShape", "command": "go test ./internal/app -run '^TestSelfCheckOutputUsesExactRootShape$'" @@ -1452,7 +1456,53 @@ "scenarioId": "proofkit.spec-proof-core.test-inventory-and-coverage-view", "witnessId": "proofkit.test-inventory-and-coverage-view.boundary", "witnessKind": "contract", - "witnessPath": "internal/command/requirementcoverageview/requirementcoverageview_test.go", + "witnessPath": "internal/command/requirementcoverageview/output_closure_test.go", + "witnessSelectors": [ + { + "selector": "TestAdmitOutputRejectsCompactProjectionDrift", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRejectsCompactProjectionDrift$'" + }, + { + "selector": "TestAdmitOutputRejectsMissingInverseParentProjection", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRejectsMissingInverseParentProjection$'" + }, + { + "selector": "TestAdmitOutputRejectsNonCanonicalWireProjectionText", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRejectsNonCanonicalWireProjectionText$'" + }, + { + "selector": "TestAdmitOutputRejectsRemovedValidUnmappedInventoryEntry", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRejectsRemovedValidUnmappedInventoryEntry$'" + }, + { + "selector": "TestAdmitOutputReplaysFailedInventoryQualitySemantics", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputReplaysFailedInventoryQualitySemantics$'" + }, + { + "selector": "TestAdmitOutputReplaysFullRepositorySourceOwnerScopeFailures", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputReplaysFullRepositorySourceOwnerScopeFailures$'" + }, + { + "selector": "TestAdmitOutputReplaysOwnerScopeFailures", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputReplaysOwnerScopeFailures$'" + }, + { + "selector": "TestAdmitOutputRequiresEveryCoverageBasisField", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRequiresEveryCoverageBasisField$'" + }, + { + "selector": "TestAdmitOutputRequiresEveryDeclaredRootField", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRequiresEveryDeclaredRootField$'" + }, + { + "selector": "TestAdmitOutputRetainsFailedInventoryEntriesWithoutProjectedParents", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputRetainsFailedInventoryEntriesWithoutProjectedParents$'" + }, + { + "selector": "TestAdmitOutputValidatesEveryCoverageRowMetadataField", + "command": "go test ./internal/command/requirementcoverageview -run '^TestAdmitOutputValidatesEveryCoverageRowMetadataField$'" + } + ], "commandIds": [ "proofkit.go-test" ], @@ -1473,6 +1523,25 @@ "local-go" ] }, + { + "requirementId": "REQ-PROOFKIT-SPEC-006", + "scenarioId": "proofkit.spec-proof-core.declared-route-mapping-without-assurance", + "witnessId": "proofkit.requirement-coverage.declared-route-mapping-falsifier", + "witnessKind": "contract", + "witnessPath": "internal/command/requirementcoverageview/requirementcoverageview_test.go", + "witnessSelectors": [ + { + "selector": "TestBuildJSONMissingSelectorRemainsMappingOnly", + "command": "go test ./internal/command/requirementcoverageview -run '^TestBuildJSONMissingSelectorRemainsMappingOnly$'" + } + ], + "commandIds": [ + "proofkit.go-test" + ], + "environmentClasses": [ + "local-go" + ] + }, { "requirementId": "REQ-PROOFKIT-SPEC-007", "scenarioId": "proofkit.spec-proof-core.canonical-command-input-admission", @@ -1672,6 +1741,25 @@ "local-go" ] }, + { + "requirementId": "REQ-PROOFKIT-SPEC-012", + "scenarioId": "proofkit.spec-proof-core.requirement-authoring-ref-provenance", + "witnessId": "proofkit.requirement-authoring-plan.authoring-ref-provenance-falsifier", + "witnessKind": "contract", + "witnessPath": "internal/command/requirementauthoringplan/requirement_authoring_plan_test.go", + "witnessSelectors": [ + { + "selector": "TestBuildPreservesDigestBoundAuthoringRefIdentity", + "command": "go test ./internal/command/requirementauthoringplan -run '^TestBuildPreservesDigestBoundAuthoringRefIdentity$'" + } + ], + "commandIds": [ + "proofkit.go-test" + ], + "environmentClasses": [ + "local-go" + ] + }, { "requirementId": "REQ-PROOFKIT-SPEC-013", "scenarioId": "proofkit.spec-proof-core.private-proof-vocabulary-owner", @@ -1943,7 +2031,13 @@ "scenarioId": "proofkit.consumer-infra-retirement.migration-parity-admission", "witnessId": "proofkit.migration-parity-admission.boundary", "witnessKind": "contract", - "witnessPath": "internal/app/self_hosting_semantics_test.go", + "witnessPath": "internal/command/migrationparityadmission/migrationparityadmission_test.go", + "witnessSelectors": [ + { + "selector": "TestBuildProjectsEveryCallerDeclaredStatusAndSummaryField", + "command": "go test ./internal/command/migrationparityadmission -run '^TestBuildProjectsEveryCallerDeclaredStatusAndSummaryField$'" + } + ], "commandIds": [ "proofkit.go-test" ], @@ -2068,6 +2162,10 @@ "selector": "TestAdoptionContractEnvelopeCLIABI", "command": "go test ./internal/app -run '^TestAdoptionContractEnvelopeCLIABI$'" }, + { + "selector": "TestRequirementAuthoringPlanOutputUsesVersionedRootShape", + "command": "go test ./internal/app -run '^TestRequirementAuthoringPlanOutputUsesVersionedRootShape$'" + }, { "selector": "TestRequirementBrowserOneShotCLIOutputVariants", "command": "go test ./internal/app -run '^TestRequirementBrowserOneShotCLIOutputVariants$'" @@ -2167,7 +2265,13 @@ "scenarioId": "proofkit.supply-chain-quality.cli-output-schema-evolution", "witnessId": "proofkit.cli-contract.output-schema-evolution", "witnessKind": "contract", - "witnessPath": "internal/app/app_test.go", + "witnessPath": "internal/app/cli_contract_test.go", + "witnessSelectors": [ + { + "selector": "TestRequirementCoverageViewBreakingRootUsesVersionedOutputContract", + "command": "go test ./internal/app -run '^TestRequirementCoverageViewBreakingRootUsesVersionedOutputContract$'" + } + ], "commandIds": [ "proofkit.go-test" ], @@ -2531,6 +2635,12 @@ "witnessId": "proofkit.release-closeout.completion-criteria-gate", "witnessKind": "contract", "witnessPath": "internal/tools/releasecloseoutinput/main_test.go", + "witnessSelectors": [ + { + "selector": "TestBuildInputFailsClosedForEachBlockingEvidenceClass", + "command": "go test ./internal/tools/releasecloseoutinput -run '^TestBuildInputFailsClosedForEachBlockingEvidenceClass$'" + } + ], "commandIds": [ "proofkit.go-test", "proofkit.package-gate" diff --git a/proofkit/witness-plan.json b/proofkit/witness-plan.json index bd70514..95ad99b 100644 --- a/proofkit/witness-plan.json +++ b/proofkit/witness-plan.json @@ -771,6 +771,7 @@ "internal/kernel/compactproofcontract", "internal/kernel/digest", "internal/kernel/markdownfmt", + "internal/kernel/proofvocab", "internal/kernel/report", "internal/kernel/secretjson", "internal/kernel/stablejson", diff --git a/release/change-record.v2.json b/release/change-record.v2.json index 681f39a..b07ec18 100644 --- a/release/change-record.v2.json +++ b/release/change-record.v2.json @@ -1,50 +1,64 @@ { "schemaVersion": 2, - "previousVersion": "0.2.0", - "version": "0.2.1", - "changeClass": "compatible", - "breakingChanges": [], - "additions": [ + "previousVersion": "0.2.1", + "version": "0.3.0", + "changeClass": "breaking", + "breakingChanges": [ { - "changeId": "proofkit.admission.canonical-boundary-hardening", - "summary": "Shared admission now preserves exact typed JSON numbers and field spelling, separates canonical path-array policy from prose-array policy, rejects complete secret-shaped path and Windows executable alias classes, and requires higher receipt-trust ranks to refine lower-rank obligations." + "changeId": "proofkit.authoring.lossless-reference-projection", + "summary": "Requirement authoring plan output advances from schema v1 to v2, requires the complete admitted authoringRefs projection, and composes current and candidate requirement records only from canonical requirement-source owner projections." }, { - "changeId": "proofkit.cli.argument-constraint-closure", - "summary": "The public CLI contract and generated help now expose descriptor-owned singleton, mutual-exclusion, presence, enum-domain, and value-dependent flag constraints, including requirement-browser session, scope, view, host, and cross-flag admission." + "changeId": "proofkit.coverage.declaration-only-vocabulary", + "summary": "Caller-authored test routes, falsifier, oracle-signal, and supersession metadata now use declaration-only classes and mapped states; mapping no longer emits covered, verified, semantic-execution, oracle-review, or dominance-proof vocabulary, and related classifications and coverage metric fields are renamed." }, { - "changeId": "proofkit.coverage.witness-anti-vacuity", - "summary": "Self-hosting coverage now rejects Go witness selectors that expose no statically reachable failure-capable candidate or contain statically resolved testing.T skip calls, including direct aliases and helper propagation; this remains candidate evidence rather than semantic proof." + "changeId": "proofkit.coverage.output-parent-reference-closure", + "summary": "Requirement coverage output advances from schema v1 to v2, retains complete test projections on requirement, owner-invariant, command, and unmapped-inventory surfaces, and re-admits the exact root and row shapes, row metadata, command-owned non-claims, canonical row order, coverage-basis owner scope and inventory commitments, inventory warnings and failures, compact scenario route unions, command state, parent references, retained unknown-reference diagnostics, and declared dead-zone diagnostics." }, { - "changeId": "proofkit.proof.delegation-authority", - "summary": "Proof obligation algebra now fails closed on unresolved delegation references and cross-requirement child edges until an owner-admitted delegation authority exists." + "changeId": "proofkit.migration.caller-declared-parity", + "summary": "Migration parity input statuses, diagnostics, and summaries now identify caller-declared claims, preserve each record's admitted non-claims, and no longer imply that Proofkit computed native parity." }, { - "changeId": "proofkit.release.migration-support-baseline", - "summary": "Release-history migration support starts at 0.2.0, and any future cumulative plan must be derived from contiguous owner-reviewed per-release records without backfilling pre-baseline release history." - }, + "changeId": "proofkit.proof.requirement-state-vocabulary-closure", + "summary": "Requirement proof states now use one closed vocabulary across structured bindings, compact contracts, and re-admitted coverage output; arbitrary rule IDs and assurance-sounding states are rejected." + } + ], + "additions": [ { - "changeId": "proofkit.release.npm-predecessor-lineage", - "summary": "Release candidate preflight binds a new candidate previousVersion to npm latest, while an exact already-published idempotent candidate requires npm latest to equal the candidate version." + "changeId": "proofkit.coverage.classification-contract-closure", + "summary": "The requirement coverage output contract now declares every reachable diagnostic classification, including proof_route_candidate_only and the fail-closed unclassified_gap fallback." }, { - "changeId": "proofkit.release.registry-authority-closure", - "summary": "Release metadata now promotes npm publication only from repository-produced typed registry evidence whose unique package set exactly matches retained provider records and local candidate identity and bytes." + "changeId": "proofkit.release.coverage-metric-reachability", + "summary": "Release closeout now uses overflow-safe exact coverage-producer relations for inventory entries, route classes, proof-route candidates, the complete command inventory, and commands missing declaration-only routes, and binds both command projections to the actual CLI contract inventory from the same source snapshot." } ], "migration": { - "required": false, - "steps": [] + "required": true, + "steps": [ + "Replace test inventory evidenceClass values semantic_falsifier, contract_admission, and property_or_fuzz with declared_semantic_falsifier_route, declared_contract_admission_route, and declared_property_or_fuzz_route.", + "Replace test inventory summary fields semanticFalsifierCount and weakOracleFailureCount with declaredSemanticFalsifierRouteCount and incompleteDeclaredOracleMetadataFailureCount; replace diagnostics missing_semantic_anchor and weak_or_empty_oracle with missing_declared_route_anchor and incomplete_declared_oracle_metadata; and replace rule IDs test_inventory.semantic_entries_have_anchors and test_inventory.strong_oracles with test_inventory.declared_routes_have_anchors and test_inventory.declared_oracle_metadata.", + "Within test-evidence-inventory-discovery-draft output, replace summary field weakOracleWarningCount with missingDeclaredAssertionSignalWarningCount, warning and action type weak_or_empty_oracle with missing_declared_assertion_signal, candidate quality finding suffix weak-oracle with missing-declared-assertion-signal, class empty_oracle with missing_edge, and suffix missing-semantic-anchor with missing-declared-route-anchor.", + "Within agent-route inspect_coverage stopConditions, replace weak-oracle language with missing declared assertion-signal language; treat the condition as routing guidance rather than an oracle-quality classification.", + "Replace falsifier field supersessionProofRef with supersessionDeclarationRef, diagnostic reasons missing_dominance_proof and unowned_dominance_proof with missing_supersession_declaration and unowned_supersession_declaration, and rule ID test_inventory.no_duplicate_falsifiers with test_inventory.declared_falsifier_supersession_consistency.", + "Replace requirement coverage states covered_by_semantic_falsifier, covered_by_property_or_fuzz, covered_by_contract_admission, covered_by_governance_invariant_nonproduct, command_semantic_falsifier_present, and missing_command_semantic_falsifier with mapped_to_declared_semantic_falsifier_route, mapped_to_declared_property_or_fuzz_route, mapped_to_declared_contract_admission_route, mapped_to_declared_governance_or_release_route_nonproduct, command_declared_semantic_falsifier_route_present, and missing_command_declared_semantic_falsifier_route; replace classification missing_semantic_test with missing_declared_test_route.", + "Consume proofkit.requirement-coverage-view.output.v2 with schemaVersion 2; preserve every admitted test-entry field, retain entries without selected parents in the required unmappedTests root array, consume the required coverageBasis owner scope, full-repository out-of-scope source identities, and retained-inventory digest, and consume compact scenarios with required verifyCommands and witnessSelectors arrays. Output re-admission now requires exact root and row shapes, exact row metadata, command-owned non-claims, canonical row order, owner-scoped compact scenario identities, exact missing-inventory warnings, and rejects owner-scope, inventory-failure, proof-route union, parent-reference, repeated-test, command-state, retained unknown-reference, mutable-fragment, dead-zone, and row-derived or scope-derived diagnostic drift; coverageBasis does not authenticate source inputs or producer identity.", + "Use only explicitly_deferred, not_bound, or witness_backed for requirement proofState values; compact binding rows contain positive and falsification witnesses and therefore require witness_backed proof_contract_state. Arbitrary rule IDs and assurance-sounding states are no longer admitted.", + "Replace coverage metric fields commandWithoutSemanticFalsifierRouteCount, commandsWithoutSemanticFalsifierRoute, semanticInventoryEntryCount, unknownSemanticCommandRefCount, and unknownSemanticCommandRefs with commandWithoutDeclaredSemanticFalsifierRouteCount, commandsWithoutDeclaredSemanticFalsifierRoute, declaredSemanticFalsifierRouteEntryCount, unknownDeclaredSemanticRouteCommandRefCount, and unknownDeclaredSemanticRouteCommandRefs; stop consuming removed semanticRouteCount, consume commandRoutes.commands and cliContract.commands, and require both inventories to equal the actual CLI contract command inventory from the same source snapshot.", + "Replace migration parity statuses matched, mismatched, not_comparable, and not_run with caller_declared_match, caller_declared_mismatch, caller_declared_not_comparable, and caller_declared_not_run.", + "Replace migration parity output fields admittedParityEvidenceCount, matchedCount, mismatchedCount, notComparableCount, notRunCount, and admittedParityEvidenceRefs with admittedParityClaimCount, callerDeclaredMatchCount, callerDeclaredMismatchCount, callerDeclaredNotComparableCount, callerDeclaredNotRunCount, and admittedParityClaimRefs; consume each migrationParity record's required nonClaims projection.", + "Consume proofkit.requirement-authoring-plan.output.v2 with schemaVersion 2, preserve its required authoringRefs array, treat nonAuthoritativeAdmissionPreview.requirementSourcePreview as a canonical requirement-source projection in which absent optional fields are omitted, and handle candidateRequirement owner-admission failures as input errors before composition." + ] }, "platformRequirements": [ "Published Darwin package binaries require macOS 12.0 or later on arm64 and x86_64." ], "knownLimitations": [ - "TSX source parsing remains unsupported.", - "Static route coverage does not prove semantic execution coverage.", - "Release-history migration support starts at 0.2.0; no machine release-history chain is claimed for earlier releases." + "Execution-backed semantic command evidence remains open under COVERAGE-01; declared routes prove mapping only.", + "Requirement-source v2 representation and any textual DSL remain unselected until the typed-model and codec replacement gates pass.", + "TSX source parsing remains unsupported." ], "rollback": { "strategy": "previous_admitted_version" diff --git a/tests/browser/workspace.spec.mjs b/tests/browser/workspace.spec.mjs index b8ccdc1..3f71d65 100644 --- a/tests/browser/workspace.spec.mjs +++ b/tests/browser/workspace.spec.mjs @@ -1335,7 +1335,6 @@ async function assertRenderedContrast(page, row) { for (const colorScheme of ["light", "dark"]) { await page.emulateMedia({colorScheme}); await assertWorkspaceState(page, row); - await page.keyboard.press("Tab"); const samples = await page.evaluate(() => { const parse = (value) => { if (value === "transparent") return [0, 0, 0, 0]; @@ -1384,7 +1383,8 @@ async function assertRenderedContrast(page, row) { text[3] *= opacity; const border = parse(style.borderTopColor); border[3] *= opacity; - control.focus(); + control.blur(); + control.focus({focusVisible: true}); const focused = getComputedStyle(control); const outlineStyle = focused.outlineStyle; const outlineWidth = Number.parseFloat(focused.outlineWidth);