Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ADOPTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ caller-owned capability or test observations

Use `test-evidence-inventory --projection discovery-draft` only for explicit
caller-owned test discovery facts. The command does not scan repositories,
execute tests, emit `semantic_falsifier` evidence, or close coverage. It emits
execute tests, emit execution-backed semantic evidence, or close coverage. It emits
candidate inventory guidance with a non-strict candidate authority so an agent
can ask the right owner questions and materialize strict inventory rows later.
Candidate inventory diagnostics are rejected by strict inventory admission until
Expand Down
8 changes: 7 additions & 1 deletion BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,13 @@ records, generated release manifests, or the owning docs named above.

| Status | ID | Scope | Completion condition |
|---|---|---|---|
| NEXT | COVERAGE-01 | Replace command proof-route candidates with an owner-admitted executable oracle ledger; static route records are explicitly non-semantic. | `CommandCoverageInventory` consumes a separate execution-backed owner ledger whose rows bind `commandRef`, selector, concrete falsification event, assertion oracle, expected public outcome, and owner invariant; until then route metadata, prose, source markers, test existence, and failure-capable AST nodes emit only `proof_route_candidate`, candidate-route closure remains blocking, and missing `semantic_falsifier` evidence remains an explicit non-blocking metric and non-claim. |
| NEXT | COVERAGE-01 | Replace command proof-route candidates with an owner-admitted executable oracle ledger; static route records are explicitly non-semantic. | `CommandCoverageInventory` consumes a separate execution-backed owner ledger whose rows bind `commandRef`, selector, concrete falsification event, assertion oracle, expected public outcome, and owner invariant; an independently authored versioned counterfeit corpus covers every shipped policy, evidence class, identity coordinate, and single or correlated substitution axis with positive controls and exact expected decisions; until then route metadata, prose, legacy source markers, test existence, and failure-capable AST nodes emit only declared routes or `proof_route_candidate`, candidate-route closure remains blocking, and semantic execution evidence remains an explicit non-claim. |
| NEXT | COMPACT-01 | Replace compact proof caller labels and synthetic counts with an honest declaration-only schema/profile. | One atomic schema/profile cutover prefixes caller-owned proof and mutation fields with `declared`, removes checked/finding counts without independent evidence owners, rejects collapsed witness roles, updates every compact producer and consumer, and proves exact role and round-trip closure without claiming execution or assurance. |
| NEXT | SOURCE-MODEL-01 | Define one representation-neutral typed requirement-source v2 model before selecting a source syntax. | A private bounded model admits atomic requirement identities, grouped authoring, premises, scenarios, definitions, vocabulary, lifecycle, references, and deterministic normalization; an independently authored field/variant completeness manifest plus mutant corpus proves every normative field reaches each required downstream owner, with no production parser or persisted normalized mirror. |
| BLOCKED | SOURCE-CODEC-01 | Select at most one compact source codec without creating dual authority. | After `SOURCE-MODEL-01`, one versioned experiment manifest freezes disjoint role sets: the flat-v1 baseline control, grouped-model ablations, and exactly complete grouped-JSON plus at most one complete restricted-DSL codec candidate over the same model. Only codec candidates can win the predeclared replacement relation; controls and ablations measure causality and cannot become production grammars. A newly discovered candidate requires a new manifest version and complete experiment. A frozen corpus and strict `Replace(candidate, grouped-json)` predicate cover grammar completeness, safety, semantic parity, diagnostics, canonical bytes, review accuracy, token cost, diff amplification, parse/format cost, and unknowns. Every metric is classified exactly once by a versioned registry with role, direction, baseline pair, aggregation, material threshold, primary decision requirement, and missing-observation semantics; duplicate or unclassified metrics fail admission, hard constraints cannot trade off, report-only metrics cannot decide replacement, promised byte/token reductions must be materially better, and bounded diff/parse costs must be noninferior. If grouped JSON fails its hard gate, retain the current flat v1 source and perform no v2 cutover; otherwise select the restricted text candidate only when it is the unique strict replacement, while a tie, unknown, incomparability, or non-material improvement selects grouped JSON. The losing parser and formatter are deleted before experiment closeout, and production admits exactly one grammar. |
| BLOCKED | SOURCE-CUTOVER-01 | Migrate self-hosted requirement sources only after one codec, the typed v2 model, nested structural contracts, and the complete evidence counterfeit corpus pass their gates. | `COVERAGE-01` and `SCHEMA-01` are complete; a digest-bound clause ledger proves representation-only equality or owner-reviewed semantic decomposition for every legacy requirement; all bindings/scenarios/contracts/context/diff/graph/browser owners cut over atomically; v1 admission and the losing codec are removed; and active-v1 inventory is zero. |
| BLOCKED | SCHEMA-01 | Replace root-shape-only public contracts with one independent complete nested structural-contract owner. | A versioned schema owner covers nested fields, variants, cardinalities, bounds, enums, defaults, duplicate and unknown-field policy, and cross-field constraints; generated artifacts pass parity against an independently authored completeness manifest and mutant corpus without becoming semantic or policy authority. |
| BLOCKED | SOURCE-PILOT-01 | Validate the selected source-v2 model and agent routing against heterogeneous external repositories without mutating them. | At least two independent repository classes complete no-push dual runs whose frozen inputs compare incumbent and candidate mapping, diagnostics, token cost, authoring accuracy, proof-route gaps, and rollback; unresolved parity or authority gaps keep incumbent owners active. |
| BLOCKED | GOVERNANCE-01 | Evaluate a generic explicit-inventory governance-observation command without promoting one consumer's policy into Proofkit; detailed candidate contract is retained in [issue #64](https://github.com/research-engineering/agentic-proofkit/issues/64). | A sanitized reproducible fixture detects one named failure without classifying its false-positive counterexample, existing owners are proven insufficient, and either a second independent consumer reproduces the predicate or the owner admits recurring first-consumer cost; otherwise retire the candidate. |
| DEFERRED | VALUE-01 | Admit exact value-evidence comparisons only after a real producer and downstream consumer establish the public record boundary; detailed candidate contract is retained in [issue #65](https://github.com/research-engineering/agentic-proofkit/issues/65). | A real execution-receipt projection, baseline producer, and downstream consumer prove an exact producer-output-to-admission round trip plus compact/full-graph inclusion or an intentional omission non-claim; otherwise no public command is added. |
| BLOCKED | RELOCATION-01 | Add provenance-bounded witness relocation candidates without introducing a second binding path or trusting a caller-authored prior digest; detailed candidate contract is retained in [issue #66](https://github.com/research-engineering/agentic-proofkit/issues/66). | An owner-admitted content-addressed baseline binds witness id, prior path and digest, source revision, evidence class, authentication non-claims, and freshness non-claims; the scanner then proves the zero/one/many match partition while remaining non-current until fresh execution evidence exists. |
Expand Down
4 changes: 2 additions & 2 deletions docs/proofkit-contract-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ owner boundaries. It is not a second command-family inventory.
| Adoption and scaffolding | `init`, `adoption-contract-envelope`, `adoption-workflow-plan`, `adoption-checklist`, `adoption-doctor`, `gradual-adoption`, `gradual-adoption-bootstrap`, `gradual-adoption-guidance`, `capability-map-admission`, `pilot-admission`, `scaffold-profile-plan`, `scaffold-project-structure`, `stack-preset` | adoption intent, aggregate adoption contract envelope, checklist facts, target paths, owner routes, caller-extracted stale authority vocabulary facts, explicit pre-spec capability observations, pilot records, stack preset id, optional init preset id | dry-run route selection, aggregate contract-envelope admission, deterministic starter plans, checklist/report admission, bounded guidance envelopes, dry-run manifests, pre-spec trust-mode admission, adoption gap and stale-authority classification, pilot shape admission | final files, final requirements, rollout policy, text extraction from files, code observation extraction, pilot truth | selected child output, plan, report, seed packet, or agent envelope |
| Requirement source | `capability-map-admission`, `requirement-authoring-plan`, `requirement-source-admission`, `requirement-source-transition`, `spec-overview-claims`, `requirement-spec-tree`, `requirement-spec-tree-view`, `requirement-source-view`, `requirement-browser-server` | `requirements.v1.json`, caller-owned capability maps, caller-owned authoring facts, overview claim extraction, explicit spec hierarchy, view options | candidate seed admission, candidate-only authoring packets, source-shape admission, lifecycle checks, explicit tree topology/source-ref admission, shared safe renderer fragments, presentation-only views | requirement meaning, extraction completeness, Markdown extraction completeness, hierarchy ownership, proof adequacy, file materialization | capability map report, authoring packet, source report, spec-tree report, rendered view, or browser presentation |
| Requirement proof binding | `requirement-bindings`, `binding-partition`, `proof-slice`, `evidence-graph`, `requirement-proof-resolver`, `requirement-proof-source-set`, `requirement-proof-view`, `spec-proof-bundle-admission` | requirement records, bindings, witness commands, source-set facts, receipt reports, partition policy | graph validation, binding partition projection, compact slices, typed compact proof contract projections, resolver projections, bundle linkage checks | test semantics, witness execution, proof freshness, merge policy | proof report, partition report, slice, lookup graph, or view |
| Test inventory and coverage | `test-evidence-inventory`, `test-evidence-inventory --projection discovery-draft`, `test-evidence-inventory --normalized-inventory`, `requirement-coverage-input-compose`, `requirement-coverage-view`, `requirement-browser-server --view coverage` | caller-owned direct or source-set test inventory, caller-owned explicit test discovery facts, declared quality findings, requirement source, proof binding or compact proof contract, coverage universe, optional owner-invariant registry, aggregate coverage compose input | strict inventory/source-set admission, candidate-only discovery draft projection, fail-closed normalized inventory projection, deterministic coverage-view input composition from explicit facts, weak-oracle and declared-quality classification, bounded agent action guidance, requirement/test/command/owner-invariant joins, nonsemantic command-evidence classification, stable coverage failure/warning classifications, presentation-only coverage view | inventory completeness, weak-test truth, test discovery extraction, native test execution, receipt freshness, producer trust, merge policy | candidate inventory guidance, inventory report, normalized inventory data product, coverage-view input, coverage view, or browser presentation |
| Test inventory and coverage | `test-evidence-inventory`, `test-evidence-inventory --projection discovery-draft`, `test-evidence-inventory --normalized-inventory`, `requirement-coverage-input-compose`, `requirement-coverage-view`, `requirement-browser-server --view coverage` | caller-owned direct or source-set test inventory, caller-owned explicit test discovery facts, declared quality findings, requirement source, proof binding or compact proof contract, coverage universe, optional owner-invariant registry, aggregate coverage compose input | strict inventory/source-set admission, candidate-only discovery draft projection, fail-closed normalized inventory projection, deterministic coverage-view input composition from explicit facts, missing declared assertion-signal and declared-quality classification, bounded agent action guidance, requirement/test/command/owner-invariant joins, nonsemantic command-evidence classification, stable coverage failure/warning classifications, presentation-only coverage view | inventory completeness, oracle quality, test quality, test discovery extraction, native test execution, receipt freshness, producer trust, merge policy | candidate inventory guidance, inventory report, normalized inventory data product, coverage-view input, coverage view, or browser presentation |
| Selective planning | `changed-path-set`, `requirement-impact-input-compose`, `impact`, `selective-gate-plan`, `selective-gate-evidence`, `selective-gate-obligation-decision-input`, `proof-obligation-algebra`, `obligation-decision` | changed paths, base/current requirement sources, base/current single-binding-per-requirement proof contracts, generated-artifact policy, local environment policy, proof-like path policy, scan obligation ownership, planned receipts, obligation routes, obligation algebra records | fail-closed impact input composition, fail-closed planning, receipt comparison, obligation algebra admission, bounded agent packets | git diff truth, repository scanning, command execution, producer trust, final admission | composed impact input, plan, evidence report, obligation algebra report, or obligation input |
| Receipts and producers | `proof-receipt-admission`, `receipt-producer-admission`, `receipt-currentness-scope`, `receipt-trust-class`, `producer-policy-self-proof` | receipt sets, producer policy, scope/currentness facts, trust classes | receipt shape, producer/receipt compatibility, self-proof diagnostics | producer authentication, freshness policy, CI trust roots | receipt/provenance report |
| Release and deployment | `release-authority`, `external-consumer`, `registry-consumer-proof-input-compose`, `registry-consumer`, `deployment-evidence-admission`, `completion-criteria`, `branch-authority`, `readiness-closeout` | package facts, tarball/registry facts, explicit primitive registry/install/smoke facts, deployment evidence, criteria, branch facts | artifact/channel boundary checks, registry-consumer input composition, release diagnostics, falsifiable criteria shape | package publication, registry fetch, package-manager execution, deployment, rollback, approval | composed input, release/deployment/readiness report |
Expand Down Expand Up @@ -131,7 +131,7 @@ Semantic context routes are `requirement-context-compose`,
| An agent needs a bounded semantic subset instead of whole specification files. | `requirement-context-compose --repo-root <caller-selected-root>` over an explicit catalog, then `requirement-context-slice` over the materialized snapshot. | The snapshot and slice are content-bound derived projections. Stop before inferring hierarchy, scanning ambient paths, treating omissions as absence, or promoting the slice to requirement, proof, freshness, or merge authority. |
| Overview prose may contain durable claims. | `spec-overview-claims` | Escalate when normative claims are not tied to `REQ-*` records. |
| Requirements have no verified proof route. | `requirement-bindings`, `witness-plan` from either an explicit `witness_command_catalog` or a complete `binding_witness_plan_input`, `proof-slice`, or `requirement-proof-resolver` | Stop before claiming proof adequacy; native witness semantics stay with the consumer. A binding-derived witness plan still needs caller-owned vocabulary and conservative command policy. |
| Tests or proof evidence need inventory. | `test-evidence-inventory`; use `--projection discovery-draft` only for explicit discovered-test facts, then `requirement-coverage-input-compose` when an aggregate `coverage_compose_input` exists, then `requirement-coverage-view` when a `coverage_view_input` exists | Compose only from explicit caller-owned facts. Discovery drafts are candidate-only and cannot close coverage. Use `failureClassifications[]`, `warningClassifications[]`, and `agentActionPlan[]` for machine routing. Escalate when tests are route-only, weak-oracle, unbound, or outside the caller-owned coverage universe. |
| Tests or proof evidence need inventory. | `test-evidence-inventory`; use `--projection discovery-draft` only for explicit discovered-test facts, then `requirement-coverage-input-compose` when an aggregate `coverage_compose_input` exists, then `requirement-coverage-view` when a `coverage_view_input` exists | Compose only from explicit caller-owned facts. Discovery drafts are candidate-only and cannot close coverage. Use `failureClassifications[]`, `warningClassifications[]`, and `agentActionPlan[]` for machine routing. Escalate when tests are route-only, lack a declared assertion signal, are unbound, or are outside the caller-owned coverage universe. Proofkit reports declared signals; the consumer owns oracle-quality and test-quality conclusions. |
| A change set is known. | `changed-path-set`, optionally `requirement-impact-input-compose`, `impact`, then `selective-gate-plan --agent-envelope` | Raw `knownChangedPaths` in `agent-route` are diagnostic only. Materialize a caller-owned `changed_path_set`, compose a caller-owned `impact_input` before `impact`, and compose a caller-owned `selective_gate_plan_input` before `selective-gate-plan`. Use `scanObligation` to name whether a `text-policy`, `secret-scan`, or caller-owned external scanner is required. Fail closed on unknown scope, dynamic edges, missing owner routes, unbound proof-like paths, or full-gate escalation. |
| Caller-owned file contents need secret-like text detection. | `secret-scan` | Provide explicit sorted file inventory with content. Stop before claiming repository-wide discovery, credential validity, provider ingestion, merge readiness, or replacement of GitHub secret scanning. |
| Does a TypeScript package public API match a caller-owned manifest? | `agent-route` with `goal: "verify_typescript_public_api"` and explicit `typescript_public_api_manifest` plus `typescript_public_api_repo_root`, then `typescript-public-api-surfaces --repo-root <caller-selected-root>` | The manifest must name each referenced `package.json`, sorted-unique export conditions, and a non-JSX `.ts`, `.mts`, or `.cts` `sourcePath` whose canonical target has the same admitted extension class. The bounded scanner accepts only the fail-closed export grammar in `proofkit/cli-contract.v2.json`; it does not parse unrestricted TypeScript or TSX, infer conventional layouts, or prove compiler output provenance, checkout freshness, package-manager truth, or merge readiness. |
Expand Down
8 changes: 5 additions & 3 deletions docs/specs/proofkit-consumer-infra-retirement/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,11 @@ approval, rollout approval, and production decisions.
gradual adoption, and release-channel scenarios to existing Proofkit
primitives through bounded structured command refs.
- `REQ-PROOFKIT-RETIRE-006`: migration parity admission validates
caller-provided parity evidence shape, source/target closure, typed
equivalence dimensions, and matched digest equality without owning evidence
authenticity, freshness, semantic correctness, or retirement approval.
caller-provided parity-claim shape, source/target closure, typed equivalence
dimensions, consistency between caller-declared status and supplied digests,
and lossless record-level non-claim projection without owning digest
computation, evidence authenticity, freshness, semantic correctness, or
retirement approval.
- `REQ-PROOFKIT-RETIRE-007`: gradual adoption guidance keeps caller-provided
candidate boundaries advisory, exposes owner-review questions, routes mode
and checked-scope semantics through one private owner, and fails closed for
Expand Down
Loading
Loading