Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

## Unreleased

- Add machine-readable execution accounting for the bounded `TR-APR-001`,
`TR-POL-003`, and `TR-SCA-002` pilot to CLI JSON reports. Accounted findings
and accounting come from one immutable execution snapshot; unreconciled
accounting is rejected on that path, the operational policy-correspondence rule
is separated from supporting schema locators and all carry value digests for
comparison against the referenced trace-spec bytes, scheduler non-execution
carries a reason, and existing verdict policy and CLI exit behavior are unchanged.

## v0.5.1 — 2026-08-22

- Level 1 and Level 2 verification now requires a verifier-issued challenge via
Expand Down
36 changes: 36 additions & 0 deletions LIMITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,39 @@ A report is produced by whoever ran the suite, on evidence they supplied. There
assessor and no certification programme behind it. This is why the generated report tells a reader
who does not trust the sender to go and check the record themselves rather than trusting the
summary.

## Bounded obligation accounting

**`accounting_complete` describes the pilot, not all of TRACE.**
The accounting extension covers only `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002`.
Complete means that every attempted level has exactly one reconciled row for each of those three
obligations. It does not mean every TRACE obligation was accounted for, every obligation was
evaluated, or that the report independently proves which code ran or which fields it accessed.
An obligation absent from both the bounded registry and its rows is outside this completeness
claim and cannot be discovered by registry-to-row reconciliation alone.

The pilot records `TR-SCA-002` at Level 0 as applicable but not attempted: the pinned schema
requires `build_provenance.digest`, while the scheduler first runs `TR-SCA` at Level 1. The row
carries that scheduler reason rather than silently presenting the state without an explanation.

Each source locator includes a digest of the exact value resolved at its pinned trace-spec
revision. JSON sources use RFC 6901 pointers and RFC 8785-canonical value bytes. The operational
TR-POL-003 rule uses the unique exact text of verification item 5 in the pinned specification;
its schema fragments are listed separately as structural support. A reader who holds those
pinned bytes can resolve and compare the values. The suite does not fetch or authenticate
trace-spec at report time, and a matching value does not prove that a locator is sufficient or
that its checker is correct.

Findings and accounting share one validated execution snapshot during supported report
construction. The emitted report remains editable and unsigned. Its registry hash identifies
registry content; it does not authenticate the rows, the report, or its producer.

The optional policy resolver is trusted in-process Python code supplied by the caller. The
accounted path isolates nested public runs and freezes ordinary decision inputs, but it is not a
sandbox against a callback that rewrites interpreter globals, functions, classes, or source files.

`producer_branch` is a checker-owned same-execution diagnostic. Some branches with the same
accounting meaning return indistinguishable findings, so the emitted report alone cannot
independently reconstruct every branch label. Applicability, evaluation state, prerequisite,
contribution, findings, and report tallies are revalidated at emission; the label is not
presented as independent proof of checker control flow.
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,25 @@ conformance report that looks authoritative and cannot be checked is the same sh
thing as a control plane writing its own log.

`report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards
and CI.
and CI. Reports produced by the CLI include an additive, version-tagged
`obligation_accounting` object for the bounded `TR-APR-001`, `TR-POL-003`, and
`TR-SCA-002` pilot. During supported report construction, its rows are reconciled
against the executable registry identified by `registry_id` and `registry_sha256`.
The operational `TR-POL-003` rule is identified separately from schema fragments
that support its field shape. Every source locator carries a digest of the exact
resolved value so a reader holding the pinned trace-spec bytes can re-resolve and
compare it. Accounted JSON, HTML, badge, and verdict projections consume one
immutable execution-derived snapshot. A JSON-only request does not pre-render
unrequested formats; when multiple formats are requested, they are emitted in the
CLI's existing order. The existing contribution policy continues to determine the
report verdict.
The tag identifies this emitted object shape; the repository does not currently
ship a separate formal JSON Schema for it.

This treats `report/1` as additively extensible: existing members retain their meaning,
and `obligation_accounting` is the sole new top-level member. Compatibility with
consumers that require the exact historical key set is not established. The report
remains an unsigned self-report; see [Known Limitations](LIMITATIONS.md).

## Test modules

Expand Down
8 changes: 8 additions & 0 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ trace-tests report --record trust-record.json --html report.html --json report.j

Use `--fail-under 1` to gate CI on a level. Without it the command always exits `0`, which is what you want when you are producing an artifact rather than enforcing a threshold. `report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards and CI.

CLI reports add an independently versioned `obligation_accounting` member for a
bounded three-obligation pilot: `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002`.
The rows and findings come from one execution snapshot, and the report refuses
an incomplete pilot matrix. This does not claim complete TRACE accounting.
The extension treats `report/1` as additively extensible; compatibility with
consumers requiring the exact historical top-level key set is not established.
See [Known limitations](LIMITATIONS.md) for the trust and replay boundary.

A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log. So the report tells a reader who does not trust the sender to go and check the record instead, and gives them what they need to do it.

## Where to start
Expand Down
Loading
Loading