docs: align runtime claims and module navigation with the suite - #99
Conversation
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
|
@agentrust-io/maintainers please review and approve this documentation correction once CI is green. It states what the runtime checks actually do, corrects the module count, and adds Appraisal to navigation. All 13 runtime tests, the strict site build, and 1,475 local links pass. |
lywinged
left a comment
There was a problem hiding this comment.
Read the description and the diff at 5d3e878, then checked docs/modules/tr-rte.md against tr_rte.py. The new table matches the code line for line: four findings, TR-RTE-001 to 004; the ten-platform set named in the same words; [0-9a-f] with 64 or 96 digits, so all-zero passes the format check as the table says and uppercase hex fails it; rim_uri tested only for a string starting https://, with absence reported as SKIP; and TR-RTE-004 requiring a nonempty expected nonce at Level 1 or above and comparing with hmac.compare_digest. Eight is the count of tr_* modules under src/trace_tests/modules/; tr-apr.md exists and renders in the nav; and docs/modules.md, untouched by this PR, already listed eight on main. The three ci.yml invocations pass here at 152, 234 and 553 tests; the 5 xpassed are the same five test_level2.py hardware cases main reports; the 13 runtime tests pass; the strict build, assembled the way docs.yml assembles it, is clean and produces 23 pages, the same count your description gives. Not reproduced: the 1,475-link count, which the strict build does not stand in for. Approving on the rest.
One thing, not blocking, and outside this PR's stated scope: README.md line 32 still opens with "Seven test modules covering the full specification", and the badge on line 24 says 7 modules. It is the same number on the surface GitHub shows first, and README is in exclude_docs, so the site build cannot see it. A separate one-line change to eight keeps the front page and the site saying the same thing.
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
|
Included the README correction in this same audit PR: the badge now says eight and links to the module index, and the introduction uses the same bounded scope as the site. Also corrected the privacy page to describe optional input files, caller-supplied resolvers, and retained reports. Strict site build and 1,475 local links pass. |
lywinged
left a comment
There was a problem hiding this comment.
Re-reviewing at 5574b02. My approval was at 5d3e878 and does not carry to this head, so this
is a fresh read of the two files the new commit touches, plus a re-run of the checks.
The README item I raised is fixed: the badge reads eight and links to the module index, and the
introduction states a bounded scope.
What I ran on this head. The full suite: 553 passed, 5 xpassed, with tests/unit/test_tr_rte.py
at 13. The strict site build, assembling the build directory the way .github/workflows/docs.yml
does: 23 HTML pages, and a pass over the built site finding 1,002 local links with no broken
target. My count differs from yours by method rather than by result, since I count href and
src values that resolve to a local path and drop fragments and external hosts; a planted bad
link is caught, so the pass is over real work. The same build on 7a43155 also gives 23 pages,
and its llms.txt carries no appraisal entry where this head's does, so the LLM index change does
what it says. The runtime table matches tr_rte.py check by check: the Level 0 and Level 1 split
on software-only, the lowercase-hex digest pattern across both algorithms, an absent rim_uri
reading as skipped, and the nonce comparison requiring both sides to be nonempty before
hmac.compare_digest. The ten platform values are exactly _VALID_PLATFORMS, and the note that
an all-zero measurement passes the format check corrects a table that had listed it as a negative
case.
Three things before this merges.
PRIVACY.md names an input the suite cannot accept. The new first paragraph says CLI options
can also load trusted keys. There is no such option: verify and report between them take
--record, --level, --max-level, --max-age, --expected-nonce, --receipt,
--policy-dir, --html, --json, --badge and --fail-under. Nothing under src/ handles a
trusted key, and the signature module verifies with the key the record itself carries rather than
one supplied alongside it. Outside this sentence the repository never mentions a trusted key at
all. That is the same class of statement this PR opened to remove, in the file a reader consults
to learn what the tool touches. Deleting the two words leaves the rest of the sentence true; the
policy bundles, receipts and local files it names are all real.
Two seven-module lines survive, one of them in a file this PR edits. index.md:57 reads "The
seven test modules, the TR-* error codes they emit, and what each one checks", 47 lines below
the line this PR corrects to eight, and it is the card that links to docs/modules.md.
docs/tutorials/writing-conformance-tests.md:245 reads "the single interface across all seven
modules".
docs/modules.md still describes three runtime checks. Its Runtime row at line 9 reads "TEE
platform enum, measurement format, RIM URI scheme". The nonce check is absent, so the page the new
badge links to describes the module this PR is correcting as it was before the correction.
Not blocking, and separable: nothing mechanically checks the module count, which is why the seven
survived the eighth module landing in #82 on 2026-08-28. tests/test_docs_match_the_modules.py is
where this repository puts guards of exactly this shape, and its own opening states the reason.
The count is derivable from the runner's level table, so an assertion against the published pages
would hold the next module addition to the standard the rest of that file already sets. I am happy
to open it separately rather than widen this PR again.
Tool-assisted: the runs above and this write-up.
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
|
@lywinged addressed all three requests in this same PR. Removed the nonexistent trusted-key CLI input from the privacy page, corrected both remaining seven-module lines, and added nonce comparison to the Runtime overview row. Also added a count guard in Validation: 19 documentation/runtime tests pass; strict MkDocs build passes; 23 HTML files and 1,475 local href/fragment targets checked with zero broken targets. Please re-review and approve if these corrections resolve your requests. |
lywinged
left a comment
There was a problem hiding this comment.
Approving at 2375dbb. All three items are addressed, and the suggestion was implemented rather
than noted.
The PRIVACY.md sentence now reads "CLI options can also load policy bundles, receipts, and
related local files", and the two inputs it names concretely are real: --policy-dir and
--receipt. The phrase that named an input the suite has no way to accept is gone from the
repository, not moved. I also checked the two claims around it by running the tool rather than
reading it: a report written with --json carries record values in its finding messages: the
subject, the rim_uri and the platform, for the record I used. --badge writes the file the
paragraph says it writes. So the retention paragraph describes artifacts that exist.
index.md:57 and docs/tutorials/writing-conformance-tests.md:245 both say eight. The Runtime
row in docs/modules.md now ends "RIM URI scheme, and verifier-issued nonce comparison", which
is what TR-RTE-004 does. The one seven left in the tree is at CHANGELOG.md:60, "Initial release
with 7 test modules", which is accurate as history and is correctly outside the new check.
test_published_module_counts_match_the_runner is the part worth the most here. It derives the
number from _LEVEL_MODULES rather than restating it, checks the four reader entry pages and the
README badge, and refuses to pass when a page carries no count at all, so the check cannot
degrade into passing over nothing. I ran four mutations against it: wrong count in index.md,
wrong number in the badge, a count deleted from the tutorial, and a ninth module added to the
runner. It failed on all four and passes on the unmutated tree, so it is load-bearing rather than
decorative.
On this head: 554 passed and 5 xpassed, one more than the previous head, and the extra one is
that test. The strict site build assembles and builds clean, 23 HTML pages, and the pass over the
built site finds 1,002 local links with no broken target, unchanged from before.
One forward note, not blocking. The count check asserts that every match on those four pages
equals the total, so a page that later says something true and narrower, such as two policy
modules, would fail it. If that happens the fix is to scope the pattern, not to delete the check.
Tool-assisted: the runs above, the mutations, and this write-up.
|
@agentrust-io/maintainers the latest head, |
The homepage claimed full specification coverage across seven modules, while the runtime page described network and hardware checks the code does not perform. Document all four actual runtime checks, their boundaries, and the eight-module scope. Add the omitted Appraisal module to site navigation and the LLM documentation index. Align the README, homepage card, module overview, and tutorial; correct privacy guidance to name only actual CLI inputs.
Validation: 19 documentation/runtime tests pass, including a new count guard derived from the runner's module table; the guard rejects a planted stale count. Focused lint and strict MkDocs build pass; 23 HTML files and 1,475 local href/fragment targets checked with zero broken targets. No conformance behavior changed. All further audit changes remain in this PR.