Skip to content

feat(audit): signed audit.heads snapshot and paged audit.export - #1996

Merged
joshuajbouw merged 7 commits into
astrid-runtime:mainfrom
unicity-aos:feat/audit-heads-export
Sep 29, 2026
Merged

joshuajbouw merged 7 commits into
astrid-runtime:mainfrom
unicity-aos:feat/audit-heads-export

Conversation

@MastaP

@MastaP MastaP commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Linked Issue

Closes #1995

Summary

Adds two read-only, admin-scoped audit operations so the audit log can be anchored and verified outside the runtime:

  • audit.heads returns a snapshot of every chain head, signed by the runtime key over a fixed byte layout.
  • audit.export pages one chain's raw signed entries, with their hashes and exact signing bytes.

To give entries an absolute position after pruning, chain metadata now counts the entries pruned from each chain. This is the chain-head publication follow-up that #678 leaves out of scope. It also provides an admin read path that a remote-CLI verifier can use.

Changes

  • astrid-audit: per-chain pruned-entry counter. ChainMetadata gains omitted_total and omitted_generation.
    • Prune finalization raises omitted_total in the same compare-and-swap that lowers count, under the durable append lock. A resumed finalization counts its receipt once.
    • Legacy metadata derives the earlier total from the installed receipt where that is exact. Otherwise it stores u64::MAX (unknown).
    • Absent fields are not serialized, so re-encoding legacy metadata reproduces its stored bytes.
  • astrid-audit: read functions.
    • heads_snapshot() reads each chain's metadata and prune state together, one chain at a time, and re-reads the head entry to confirm it hashes to the recorded head.
    • chain_entries_page() reads entries in bounded batches of 32. A page never mixes pre- and post-prune state: it fails with a retryable error while a prune is in progress or if one completes mid-page.
    • A cursor is accepted only if it names a stored entry of the requested chain.
    • A single entry larger than the 1 MiB page budget is returned alone, up to 1.5 MiB.
    • New AuditLog::prune_in_progress and AuditLog::chain_cursor_entry.
  • astrid-core: request and response types. AuditHeads / AuditExport requests and their responses, plus AuditHeadsSnapshot::signed_bytes_v1.
    • The layout is length-prefixed and big-endian: domain astrid.audit.heads.v1, snapshot time in ns, then per chain: session, principal or none, count, omitted_total, head hash.
    • u64::MAX signals an unknown omitted_total inside the signed bytes.
    • It is signed with Ed25519 directly, with no pre-hash.
  • astrid-kernel: handlers. Capabilities audit:heads and audit:export. Both methods skip the generic success AdminRequest row, so a poller does not grow the log it reads. Denied requests are recorded before dispatch; authorized requests whose handler fails are recorded as failure rows. Admin request authorization moves to admin/request_authorization.rs to keep admin/mod.rs under the 1,000-line limit.
  • astrid-uplink, astrid-cli, e2e. The admin client, astrid audit heads and astrid audit export --session <id> [--agent <alias>] [--from N] [--cursor C] [--limit N], and CLI scenario entries.
  • Dependencies: Wasmtime 48.0.3. wasmtime and wasmtime-wasi move from 48.0.1 to 48.0.3 for RUSTSEC-2026-0314, -0315 and -0316, published after this branch was cut, which fail the security audit job on main's lockfile as well. RUSTSEC-2026-0315 (fuel accounting skipped around call_ref) reaches capsules, since fuel metering is on and function references are enabled; the other two are not reachable by guests today. The lockfile change is limited to the wasmtime, cranelift, pulley and wiggle crates. 49.0.1 needs Rust 1.96, above the 1.95 MSRV. Changelog fragment changes/1995.security.md.
  • Changelog fragment.

Verification

  • Test suites: astrid-audit 82 (+2 doc), astrid-core 387 (+1), astrid-uplink 58, CLI 818, and kernel router all pass. The kernel lib passes 581; its 15 remaining failures are host-environment ones that also fail on the base commit: 14 depend on umask and pass under umask 0022, and 1 needs a copy-on-write workspace backend.
  • Unit and handler tests:
    • the byte-layout known-answer vector, ordering rules and the unknown sentinel;
    • chain enumeration across sessions and storage pages;
    • the counter through three prunes, with a close and reopen of an on-disk store;
    • a replayed finalization, and legacy metadata (never pruned, pruned once, pruned twice);
    • byte-exact re-encoding of legacy metadata;
    • cursor paging, rejected and forged cursors, batch reads, oversized entries, and a prune during an export;
    • kernel handler checks: signature, head hashes, the accumulated total inside the signed bytes, chain links, and the prune-receipt link;
    • the success-row exemption through the admin IPC path, including recorded denials and failures.
  • Dependency update: cargo audit reports no vulnerabilities on the new lockfile, and exactly the three advisories on the previous one. astrid-capsule and astrid-hooks (820) pass, the kernel lib passes 595 of 596 (the copy-on-write backend test), and cargo check --workspace --all-targets is clean.
  • Lint: cargo fmt and cargo clippy --all-features --all-targets -- -D warnings on the touched crates.
  • End to end: the snapshot format and export were exercised by an independent verifier (not Astrid code) against a running daemon, with AOS Community Edition main and its 22 capsules on this runtime. Chain heads were anchored on the Unicity testnet and re-verified against exported entries, including detection of a store rolled back to an earlier backup.
  • Known host-environment test failures on the development machine (umask 0002, no copy-on-write workspace backend) also occur on the unmodified base commit.

AI / Tool Assistance

Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation and tests in all touched crates, and this description.
Assisted-by: Codex CLI. Review passes over the diff.

Checklist

  • Linked to an issue
  • Changelog fragment added under changes/{issue}.{kind}.md (docs/CI-only may skip; release PRs roll fragments into the version section instead of adding one)
  • I understand every change in this PR and can explain its design, risks, and validation.
  • I reviewed and tested any meaningful tool-generated output included in this PR.
  • Every non-bot, non-merge commit has a matching Signed-off-by trailer.

Copilot AI balanced review requested due to automatic review settings September 28, 2026 17:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Chain metadata records only the retained count, and a chain keeps only its
latest prune receipt, whose omitted_count covers that one generation. After
a chain's second prune the total number of entries removed from its front
cannot be recovered, so a retained entry has no known absolute position in
its chain.

ChainMetadata gains omitted_total, the cumulative number of pruned entries,
and omitted_generation, the latest receipt generation counted in it. Prune
finalization raises omitted_total in the same compare-and-swap that lowers
count, under the durable append lock, so the two always come from one
committed state and a known total never decreases. The generation marker
lets a resumed finalization count its receipt once. Appends carry both
fields forward unchanged.

Metadata written before this change has neither field. Its first counted
prune takes the earlier total from the installed receipt: zero before
generation 0, and the generation-0 receipt's omitted_count before
generation 1. Any other earlier total is lost; it is stored as u64::MAX,
which saturating addition keeps. Absent fields are not serialized, so
re-encoding such metadata reproduces its stored bytes, which append-intent
recovery compares exactly.

Tests cover the counter through three prunes with a close and reopen of an
on-disk store in between, the prune-oldest path, a finalization replayed
after its receipt was installed, metadata without the fields pruned once
and twice before, and the byte-exact re-encoding of such metadata.

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Anchoring the audit log externally needs two read-only admin methods that
did not exist: a signed statement of every chain head, and a raw export of
signed entries with their hashes (the gateway view strips hashes and
signatures).

audit.heads (capability audit:heads) enumerates every chain-metadata
record. For each chain it reads the metadata and prune state together
under the durable append lock (one chain at a time, never across the
scan), re-reads the head entry and requires it to hash to the recorded
head. It returns session, principal, retained count, omitted total, head
hash, head id and last timestamp per chain. The runtime Ed25519 key signs a
fixed, length-prefixed encoding (domain "astrid.audit.heads.v1",
AuditHeadsSnapshot::signed_bytes_v1) directly, with no pre-hash; the
response carries the signed bytes, the signature and the public key, so a
verifier needs no serde code.

The omitted total is the per-chain counter in ChainMetadata, read from the
same committed metadata as the retained count. Metadata that has not
counted a prune yet takes it from the latest receipt: zero when there is
none, and the receipt's omitted_count for generation 0. The total is
unknown for a later generation, and while a prune plan that finished
deleting is still pending, because an older binary may have lowered the
count before it was interrupted. An unknown total is signed as u64::MAX
(AUDIT_OMITTED_TOTAL_UNKNOWN), so the claim is covered by the signature
instead of an unsigned side flag.

audit.export (capability audit:export) pages one chain's stored entries in
chain order with id, timestamp, previous hash, content hash
(BLAKE3(signing_data)), signature, embedded key, the exact signing bytes
and the stored entry, plus the chain's latest prune receipt and its signing
bytes. Pages default to 500 entries and are capped at 1000 entries and
about 1 MiB of entries. The returned cursor resumes after later appends and
is rejected unless it names an entry of the requested chain. The retained
index a cursor carries is bounded by the chain's count but not
authenticated, and a later prune makes it stale; verifiers take positions
from entry links and the signed heads instead.

Both methods skip the generic success AdminRequest row, so an anchoring
service polling them does not grow the log it anchors. Denied requests are
still recorded before dispatch, and an authorized request whose handler
fails (unknown chain, rejected cursor, storage error) is recorded after it.
Admin request authorization and its audit rows move from admin/mod.rs into
admin/request_authorization.rs to keep the module under the 1000-line
limit; the context no longer carries the request kind, so it outlives
dispatch. The CLI gains `astrid audit heads` and `astrid audit export
--session <id> [--agent <alias>] [--from N] [--cursor C] [--limit N]`.

Tests cover the encoding (known-answer vector, ordering rules and the
unknown sentinel), chain enumeration across sessions and storage pages,
the omitted total through several prunes, its derivation for metadata
without the counter (never pruned, pruned once, pruned twice) and while a
finished prune is pending, cursor paging, the kernel handlers (signature,
head hashes, the accumulated omitted total inside the signed bytes, paging,
chain links, prune-receipt link, foreign and out-of-range cursors) and,
through the admin IPC path, the success-row exemption with its recorded
failures and denials.

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
An export page read the chain metadata, the cursor entry or the skipped
prefix, the page itself and the prune receipt as separate reads. A prune
running in between could pair entries from before it with the retained
count or receipt from after it, so the first exported entry need not link
to the receipt's omitted terminal hash. An entry above the 1 MiB page
budget was also returned whatever its size, so a large enough entry
produced a response the native uplink cannot deliver in one 2 MiB frame.

AuditLog::prune_in_progress reports whether a chain has a prune plan.
Entries are deleted only while a plan exists, and a finished prune installs
a new receipt. The export checks for a prune in progress and then reads the
receipt, before and after its other reads, and fails with a retryable
error unless no prune was in progress and the receipt is unchanged. A chain
whose prune was interrupted cannot be exported until its next prune resumes
and finishes the plan. An entry above the page budget is still returned
alone, up to 1.5 MiB serialized, which leaves room for the page envelope in
one frame; a larger entry fails the page with an error that names it.

The admin IPC test helper now waits up to 30 s for a response instead of
5 s, since the kernel suite runs it alongside heavier tests.

Tests cover prune_in_progress with and without a pending plan, and an
export over entries of about 1.1 MiB and 1.6 MiB serialized that returns
the first alone and rejects the second.

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…runtime#1995

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
An export page asked storage for limit + 1 entries (up to 1001) and applied
the 1 MiB page budget only afterwards, and skipping to a `from` index read
256 entries per call. An entry can be nearly as large as the page budget, so
one request over a chain of large entries could hold hundreds of megabytes
in memory to return a single entry.

The page is now filled 32 entries per storage call, each call asking for one
more entry than the page still needs when that fits, and reading stops as
soon as the page is full or its byte budget is reached; the extra entry
still tells whether the page is complete. Skipping to a `from` index reads
the same 32-entry batches. At most one batch of entries is held besides the
page itself.

A new test exports a 70-entry chain as one page, with limits of 64 and 70,
and as a 64-entry page followed by a cursor page, crossing several batch
boundaries; the existing paging, cursor and large-entry tests pass
unchanged.

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
An export cursor's storage part is a session-index key
"<session>:<sequence>:<entry id>", and paging resumes after it by lexical
comparison. The kernel checked only that the entry id loaded and belonged
to the requested chain, so a caller could keep a real entry id and raise
the sequence to skip any part of the chain, and the page came back looking
like a valid, even complete, continuation.

AuditLog::chain_cursor_entry returns the entry a cursor names only when the
cursor is a session-index key that storage holds, and the export uses it to
resume. An altered key, or one whose entry a prune removed, is rejected
with an error that asks the caller to restart from an index.

Tests cover a real cursor, an altered sequence, a malformed key and a
pruned entry at the audit-log level, and a forged-sequence cursor through
the export handler alongside the existing foreign and out-of-range cases.

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
@MastaP

MastaP commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up stacked on this PR: #2002 (anchor-safe retention, #2001). Its diff includes this PR's commits until this one merges.

joshuajbouw
joshuajbouw previously approved these changes Sep 29, 2026

@joshuajbouw joshuajbouw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed d9d4d16. This is a good first step for the series: signed heads and raw export are generic Astrid mechanisms, with external anchoring policy kept outside the runtime.

I checked pruning counters and replay, legacy metadata compatibility, cursor validation, paging, and admin authorization. No blocking findings. Independently ran 540 tests across audit/core/uplink and the focused kernel audit handlers (2 audit tests ignored), plus focused Clippy with warnings denied; all passed. The kernel tests cover signatures, paging after pruning, rejected cursors, and recording failed/denied requests without logging every successful poll.

GitHub currently reports no CI checks for this branch. This approval is the code review, not confirmation of CI or a live external-anchoring run. #2002 should follow this merge, as planned.

Wasmtime and wasmtime-wasi 48.0.1 are affected by three advisories
published on 2026-09-24:

- RUSTSEC-2026-0315 (GHSA-m63x-6p34-q65x): code compiled for call_ref,
  and for exception catch blocks, skips fuel accounting, so a guest can
  execute far more instructions than the fuel it was given.
- RUSTSEC-2026-0316 (GHSA-jqpg-j7w6-42pr): lifting a record or tuple
  through the dynamic component Val API can allocate past the hostcall
  fuel limit.
- RUSTSEC-2026-0314 (GHSA-j2g9-4prp-pf6h): a guest can panic the host
  through a datetime overflow in the wasmtime-wasi filesystem
  implementation.

The capsule engine enables fuel metering: each pooled interceptor call is
seeded with a fixed fuel budget that caps a runaway call, and the fuel it
consumes is recorded in the invoking principal's fuel ledger. Function
references are part of the default Wasm 3.0 feature set and stay enabled,
so under 48.0.1 a capsule using call_ref could exceed that budget and
under-report its instruction count. Exceptions are disabled, so only the
call_ref path applies. Host calls use the typed bindgen bindings rather
than the dynamic Val API, and no wasi:* interfaces are linked for capsules
or hooks, so the other two advisories are not reachable by guests today;
both crates are updated so the runtime no longer ships the affected code.

Both workspace requirements move from 48.0.1 to 48.0.3, the first patched
48.x release. The lockfile change is limited to the wasmtime family:
wasmtime, wasmtime-wasi, wasmtime-wasi-io, wasmtime-environ, the
wasmtime-internal-* crates, pulley-interpreter, pulley-macros, wiggle,
wiggle-generate and wiggle-macro move to 48.0.3, and the cranelift-*
crates from 0.135.1 to 0.135.3. wasmtime-wasi 48.0.3 no longer depends on
cap-fs-ext, which leaves the lockfile, and now depends on
rustix-linux-procfs and winx, which were already locked. 49.0.1 is also
patched but requires Rust 1.96, above the workspace MSRV of 1.95.

The compiled-artifact cache is in memory only and keyed by the Wasmtime
48 ABI domain, so no persisted compiled code needs invalidating.

Verified with cargo check --workspace --all-targets, the astrid-capsule
and astrid-hooks test suites, the astrid-kernel library tests, clippy
with -D warnings on astrid-capsule and astrid-hooks, and cargo audit,
which reports no vulnerabilities (the three advisories above against the
previous lockfile).

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>

@joshuajbouw joshuajbouw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked 5984977 after the Wasmtime security update. No blocking findings in the new dependency delta. Wasmtime/WASI and the associated compiler crates are aligned on the patched 48.0.3 release; no audit/export implementation or test changes were added since my earlier review.

Independently ran cargo audit against this exact Cargo.lock: passed with 1,277 advisories loaded and 836 dependencies scanned. This addresses the three advisories without suppressing them. The earlier audit/export review still applies to the unchanged code; the refreshed CI is now running to validate compatibility with the updated runtime dependencies. This approval is not a claim that those pending jobs have passed.

@joshuajbouw
joshuajbouw merged commit ed2df49 into astrid-runtime:main Sep 29, 2026
34 checks passed
@MastaP
MastaP deleted the feat/audit-heads-export branch September 29, 2026 17:07
joshuajbouw added a commit that referenced this pull request Sep 29, 2026
## Linked Issue

Closes #2001

#1996 is merged. The retention-only commits are rebased onto main
`ed2df49b`, preserving the Wasmtime 48.0.3 security update. Current
head: `896c6e4c3d9f5fb305278a3a408250a7908d7204`.

The integration adds one Windows Clippy fix: directory syncing and its
stored path are compiled only on Unix; Windows retains the existing
write-through rename behavior. No global lint suppression.

## Summary

Today, pruning deletes entries permanently. That holds whether an
operator runs it or the global cap triggers it (1,000,000 entries / 1
GiB), and only a chain's latest prune receipt is kept. So history no
external party has seen can be erased, and a verifier cannot connect
retained entries to history it anchored several prunes earlier.

This PR changes that:

- An anchoring service records, per chain, how far it has certified the
log: an anchored watermark.
- Retention never deletes past a watermark. When the cap is reached and
nothing can be pruned, the log keeps the entries and reports degraded
instead of deleting them.
- Every prune receipt is kept and can be exported.
- Operators can require a watermark before any prune, and can archive
pruned segments.

Installs that do not anchor keep today's bounded retention.

## Changes

- **`astrid-audit`: anchored watermarks.**
- `AuditLog::mark_anchored` records that a chain was certified through a
position. The position counts entries from the chain's genesis, pruned
ones included (`omitted_total + count`). The mark carries the head hash
at that position and up to 4,096 bytes of opaque evidence.
- The claim is checked against the chain: the entry at `position - 1`,
or, at the pruned boundary, the latest receipt's terminal hash, must
hash to the given head.
- Verification resumes from the previous watermark's stored cursor, so a
mark costs only the entries appended since.
- Watermarks never decrease. Re-marking the same position and hash is a
no-op.
- Marks live in their own namespace (`audit:anchor_marks`) and are
replaced by compare-and-swap. Chain metadata, which append-intent
recovery compares byte for byte, is untouched.
- **`astrid-audit`: prunes never pass a watermark.**
- Each prune is checked against the chain's pruned total after its
retention scan. The check runs again, under the durable append lock,
when the deletion plan is created.
- Marks are installed under that lock only while no plan exists and the
receipt is unchanged since verification. This closes the window in which
a chain's first mark could land between a prune's check and its plan.
  - A refused prune deletes nothing and writes no receipt.
- `prune_oldest` (used by `audit prune` and the cap) takes the oldest
sealed segment, in seal order, whose removal is allowed. It searches the
whole seal-order index before refusing.
- **`astrid-audit`: the cap holds instead of deleting.** When an append
reaches the cap and every candidate holds unanchored history:
  - a durable retention hold is set in the global metadata;
  - appends are admitted over the cap;
- the global state reports degraded with the reason, and an error is
logged once.

The hold clears when a watermark advances, a prune finishes, a segment
is sealed or the caps change.
- **`astrid-audit`: every receipt is kept.** Receipts are recorded under
`audit:receipt_history/<chain>/<generation>` before installation. A
chain pruned before this change keeps its latest receipt.
`AuditLog::prune_receipts` pages them.
- **`astrid-audit`: archive, then drop.** With an `AuditArchiver` set, a
prune writes the entries it removes to an archive and commits it before
the deletion plan is created. The archive must match the receipt's
count, digest and terminal hash, or the prune is abandoned. Prunes are
serialized from segment selection to plan installation, so two prunes of
one chain cannot archive the same generation.
- **`astrid-core`, `astrid-kernel`: admin methods.**
- `audit.anchor_mark` (capability `audit:anchor`) accepts one
certification's evidence and up to 4,096 chains. Each chain is reported
as advanced, unchanged or rejected with the reason. Malformed evidence,
an empty or oversized list, or a duplicate chain rejects the whole
request.
- The kernel checks the evidence's shape, not the certification itself.
- Successful marks skip the generic success row, as `audit.heads` and
`audit.export` do. Rejections and denials are recorded.
- `audit.anchor_status` (capability `audit:heads`) reports each chain's
count, pruned total, watermark, its time and evidence, whether anchoring
is required before prune, and the retention hold.
- `audit.export` gains `receipts_from`, which pages prune receipts with
their hashes and signing bytes.
  - `audit.stats` reports the hold.
- The file archiver does its filesystem work on the blocking thread
pool.
- **`astrid-config`: `[audit.retention]`, operator-only.**
- `require_anchor` (default `false`) makes every prune require a
watermark.
- `archive_dir` must be an absolute path. It writes each pruned segment
as `<session>/<system|principal.alias>/<generation>.jsonl`: the signed
receipt, then the removed entries. The file is written under a temporary
name, synced and renamed, in directories private to the daemon's user.
  - A workspace layer cannot override either key.
  - `AuditConfig` moves to its own module.
- **`astrid-cli`:**
- `astrid audit anchor-status` shows per-chain watermarks and lag, and
exits 2 while the hold is set.
  - `audit prune` explains a refusal.
  - `audit stats` shows the hold.
  - `audit export --receipts-from` prints prune receipts.
  - Scenario entries.
- Changelog fragment.

New storage fields are optional and absent by default, so existing
metadata re-encodes to its stored bytes.

## Verification

Current integration validation: audit library 99 passed / 2 ignored;
config library 138 passed; kernel retention 2 passed; kernel-library
Clippy with warnings denied, formatting and diff checks passed on macOS.
All 33 checks on 896c6e4 now pass, including Windows and both MUSL
targets. #2008 is merged. The exact combined tree with main bdc1fe8
(tree 6c15a1d) also passes the full
kernel library suite: 605 passed, zero failed or ignored.

Prior author validation of the retention implementation follows:

- **Test suites:** `astrid-audit` 99 (+2 doc), `astrid-core` 390,
`astrid-config` 138, `astrid-uplink` 58 and CLI 818 all pass. The kernel
lib passes 602 of 603 with `--test-threads=6` and umask 0022. The
remaining failure needs a copy-on-write workspace backend and also fails
on the base commit. With full parallelism on a loaded host, some
kernel-router tests exceed their 2-second admin-response timeout;
#1996's branch shows the same under the same load.
- **Unit and handler tests:**
  - watermark verification and monotonicity;
  - positions across prunes and at the pruned boundary;
  - persistence across reopen;
- a first mark recorded while a prune is held between its check and its
plan: the prune is refused with nothing deleted;
  - a mark refused while a prune is pending;
  - prune refusal past a watermark;
  - the require-anchor switch;
  - oldest-eligible segment selection across refused chains;
  - the cap hold, which keeps every append and clears on anchoring;
  - cap pruning unchanged for chains without watermarks;
  - receipt history order and backfill;
  - archive contents and permissions, and abort on archive failure;
  - two concurrent prunes of one chain archiving one at a time;
  - per-chain `anchor_mark` outcomes and whole-request rejections;
  - receipt pages in export;
- no row written for a successful mark, while rejections and denials
write one;
  - config restriction and validation.
- **Lint:** `cargo fmt` and `cargo clippy --all-features --all-targets
-- -D warnings` on the touched crates.

## AI / Tool Assistance

Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation
and tests in all touched crates, and this description.
Assisted-by: Codex CLI. Review passes over the diff.

#1996 and #2008 are merged; all current-head CI checks pass.

## Checklist

- [x] Linked to an issue
- [x] Changelog fragment added under `changes/{issue}.{kind}.md`
(docs/CI-only may skip; release PRs roll fragments into the version
section instead of adding one)
- [x] I understand every change in this PR and can explain its design,
risks, and validation.
- [x] I reviewed and tested any meaningful tool-generated output
included in this PR.
- [x] Every non-bot, non-merge commit has a matching `Signed-off-by`
trailer.

---------

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
Co-authored-by: Joshua J. Bouw <jjb@unicity-labs.com>
joshuajbouw added a commit that referenced this pull request Sep 29, 2026
## Linked Issue

Closes #1999

Part of #1997. #1996, #2002, and the mount cleanup fix #2008 are merged.
Current head `c92562e65087fe47e313e4c029781e89a69927d4` is based on main
`26a500a6`. Remaining merge order: this PR, #2004 (recording coverage),
#2005 (entry format v2).

The integration preserves Pavel's ordered host-audit implementation. The
overlapping configuration is resolved in the existing
`astrid-config::audit` module, retaining both retention validation and
fail-closed host-call settings. Kernel startup loads both settings from
the same audit configuration.

This PR replaces the host-audit writer. #2004 changes the old writer's
fold key to include the capsule. Whichever of the two lands second is
rebased onto the other, and the capsule then becomes part of the run
key.

## Summary

The host-audit sink could lose calls and reorder them without a trace:

- It coalesced allowed calls per class over `host_coalesce_ms`, and
appended `repeats=N` to the outcome details. The entry signature covers
the outcome only as a success bit, so that count was not signed.
- It drained pending work per key, so a chain's entries did not follow
call order.
- A call that met the full queue (`host_queue_capacity`, default 4,096
keys) was dropped, and only a counter moved.
- A failed batch was dropped, and calls still queued when the daemon
died vanished.

As a result, even an externally anchored log could not show that it was
complete.

This PR records host calls in call order, and makes every loss visible
as a signed entry on the chain. It also lets an operator make chosen
host-call classes fail closed.

## Changes

- **`astrid-audit`: signed host-call records.** These actions are
covered by the entry signature:
- `HostCallRun`: consecutive calls of one principal. It holds the call
count, the first and last call times, a tally per class and outcome, and
a fold over every call.
- `HostCallLoss`: calls that were accepted but not recorded
individually.
  - `HostCallGap`: an earlier run of the lane stopped without draining.
  - `HostCallAdmitted`: the write-ahead entry of a fail-closed call.

`astrid_audit::host_call` specifies the per-call digest and the fold, so
a verifier can recompute them without the kernel. Known-answer tests pin
them.
- **`astrid-kernel`: ordered lane** (`audit_sink/lane.rs`, `writer.rs`).
- Each principal chain has a FIFO of pending slots, so FIFO order is
call order.
- The writer takes slots from the front and appends each batch
atomically with `append_batch_with_principal`.
- A failed batch is retried as is, with backoff up to 5 s, before
anything behind it is taken.
- Signing happens at the durable commit, because only the commit fixes a
chain position. The kernel router appends to the same chains directly.
- **`astrid-kernel`: lossless coalescing.**
- Consecutive allowed and failed calls share one run, and consecutive
identical denials share one run.
  - A one-call run is written as the plain action, as before.
- **`astrid-kernel`: loss and gap entries.**
- Queue capacity bounds slots, not calls. Overflow folds into one loss
slot per chain, which is written as `HostCallLoss` at its place in the
chain.
- A lane marker in `system:control:audit-lane` records the run and the
chains it wrote. After an unclean stop, the next start writes
`HostCallGap` into each of those chains and into the session's system
chain.
- `astrid-storage`: the state-owner resolver admits
`system:control:audit-lane` as a system control projection, like
`audit`, `invites` and `pair-tokens`.
- An unreadable marker, or a marker store that fails, also produces a
gap entry.
  - If the writer thread dies, the lane closes and health reports it.
- **`audit.host_fail_closed`** (`astrid-config`, `astrid-capsule`,
`astrid-kernel`).
- It lists host-call classes whose effect runs only after a write-ahead
entry is durable: `file_read`, `file_write`, `file_delete`,
`net_connect`, `net_bind`, `process_spawn`. The default is empty.
- `HostAuditSink::admit()` is called by the fs, net and process host
functions after the security gate and before the effect.
- A refusal fails the call with `unknown("audit unavailable")` and is
recorded as a denial.
  - A workspace layer can add classes but not remove them.
- **Health:** `accepted` and `persisted` now count calls, and `lost`,
`gaps_recorded` and `dropped_after_shutdown` are new. The admin
`AuditHealth` wire type is unchanged.
- Changelog fragment.

**Consequences.**
- A denial between allowed calls closes the run.
- Order is guaranteed among a chain's host-call entries. Admin rows are
still appended when they run.
- Calls reported after shutdown are counted in health, not written.

**Overhead** (release build, in-memory log, per producer thread;
`audit_sink::tests::bench`, ignored by default):

| Workload | Before | After |
|---|---|---|
| 1 thread, 100k mixed allowed calls | 1.61–1.65 µs/call | 0.84–0.92
µs/call |
| 8 threads × 8 principals, 200k calls | 15.1–17.9 µs/call | 8.9–12.8
µs/call |
| 4 threads, reads with 2% denials | 6.7–7.8 µs/call | 4.2–4.8 µs/call |
| 20k distinct denials, 1 thread | 1.35–1.58 µs/call; 15,904 calls
dropped | 1.82–1.91 µs/call; none dropped, overflow in 5–7 loss entries
|

## Verification

Current integrated tree: full kernel library **628 passed, 1 ignored**;
capsule audit tests **45 passed**. Earlier validation of the same
recorder/retention integration: audit 106 passed / 2 ignored, config 140
passed, kernel audit filter 51 passed / 1 ignored; audit/config/kernel
library Clippy passed with warnings denied. The published tree is
byte-identical to the tested combined tree; CI must pass on this new
head before merge.

Prior author verification (before this integration):

- **Test suites:** `astrid-audit` 74, `astrid-capsule` 748,
`astrid-config` 139 and `astrid-storage` 864 all pass. The kernel lib
passes 610 of 611 with `--test-threads=6` and umask 0022. The remaining
failure needs a copy-on-write workspace backend and also fails on
`main`.
- **Unit and behaviour tests:**
- host-call digest and fold known-answer values; every field changes the
digest;
  - a dropped, reordered or altered call fails `matches_calls`;
  - changing a signed count breaks the signature;
- call order across 6 producers, 3 principals, 8-entry batches and a 10
ms window (the same probe fails on `main`);
- a 40-call run whose fold, count and time range recompute from the
calls;
- queue overflow written as one loss entry that commits to the lost
calls;
- gap entries after an abandoned lane run, none after a clean restart,
and gap duties carried across a second unclean stop;
- a corrupt marker and a failing marker store, including the
unavailable-marker gap surviving a later marker write;
- a system-only gap recorded at start with no host call, and retried
while idle after a refused append;
- the marker namespace written and read back through a runtime principal
store;
- a batch the log refuses, retried and written in order once the cap is
raised;
  - a fail-closed write durable before `admit()` returns;
  - refusal within the first failed batch, and after shutdown;
  - config validation and workspace restriction;
  - `connect-tcp` and `bind-tcp` refusing without a connection attempt.
- **Lint:** `cargo fmt` and `cargo clippy --all-features --all-targets
-- -D warnings` on the touched crates.

## AI / Tool Assistance

Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation
and tests in all touched crates, and this description.
Assisted-by: Codex CLI. Review passes over the diff.

Current integration and verification assisted by Codex. Fresh CI
pending; no claim of merge readiness until those checks complete.

## Checklist

- [x] Linked to an issue
- [x] Changelog fragment added under `changes/{issue}.{kind}.md`
(docs/CI-only may skip; release PRs roll fragments into the version
section instead of adding one)
- [x] I understand every change in this PR and can explain its design,
risks, and validation.
- [x] I reviewed and tested any meaningful tool-generated output
included in this PR.
- [x] Every non-bot, non-merge commit has a matching `Signed-off-by`
trailer.

Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
Co-authored-by: Joshua J. Bouw <jjb@unicity-labs.com>
joshuajbouw added a commit that referenced this pull request Sep 29, 2026
… changes and capsule loads (#2004)

## Linked Issue

Closes #1998

Part of #1997. #1996, #2002, #2003, and #2008 are merged. Current head
`c0f4052305da424d9625c9f96332448667a9259b` integrates this PR with main
`e63f65cc`. Remaining order: this PR, then #2005 (entry format v2).

The integration retains #2003's ordered, loss-accounted writer and
fail-closed admission. Capsule identity is part of the run key, so
alternating capsules cannot fold together. HTTP/tool/approval coverage
records remain individual queue entries; overflow still records a
payload-bound loss summary. Existing unattributed call digests keep
their exact v1 encoding; attributed and new coverage calls use the
documented `astrid.audit.host-call.v2` digest domain. This is a
call-fold encoding distinction, not #2005's entry-format change.

Ordering boundary: queued records retain per-principal FIFO order.
Synchronous HTTP/approval commits still use the existing durable append
path, like direct administrative audit writes; no new global ordering
guarantee between that path and queued records is claimed.

## Summary

The audit log did not record the events that determine what an agent
did:

- LLM and other HTTP exchanges appeared only in `tracing::debug!`, so an
agent turn, including its model request, added no entry;
- capsule tool calls and approval decisions had no producer;
- capability grants and revocations were recorded as `AdminRequest` rows
at authorization time, before the change was applied and without its
result, and grant-on-use was not recorded at all;
- capsule install and load were not recorded, and install hooks ran
without an audit sink;
- host-call entries did not name the capsule, and `FileWrite` stored a
zero hash.

This PR records all of these on the signed log, without changing the v1
entry format. Content never enters the log, only commitments to it.
Provider credentials can now be injected by the host, so they never
enter guest memory.

## Changes

- **`astrid-audit`: entry kinds.**
  - `HttpRequest` (pre-commit) and `HttpResponse` (completion).
  - `CapabilityChanged`, `CapsuleInstalled` and `CapsuleLoaded`.
- A `CapsuleActor` (capsule id and wasm hash) on file, network, process,
tool-call and approval entries.
  - Linking fields on `CapsuleToolCall` and the approval entries.

New fields are optional and omitted when unset, and new variants are
appended after the existing ones. Entries written before this change
re-serialize to the same signed bytes.
- **HTTP exchanges** (`astrid-capsule` HTTP host, `astrid-kernel`).
- Each wire request, each redirect hop included, gets a durable
`HttpRequest` pre-commit before it is sent. The pre-commit holds the
method, host, port, BLAKE3 commitments to the path, headers and body,
the redirect hop, and a per-principal sequence number scoped to the
kernel run.
- An `HttpResponse` completion links to its pre-commit. It carries the
status, provider request ids, and a hash of the response body as it was
read (incrementally for streams).
  - Completions are appended durably.
- Refusals by the scheme check, egress, the security gate or the SSRF
airlock are recorded as denied requests and take a sequence number.
- Credential headers, credential query parameters and every secret value
the capsule received are redacted before hashing. The redaction rules
are documented so a verifier can recompute the commitments.
- Recording is best-effort, like the rest of the audit log since 0.9.0.
If the pre-commit cannot be written, the request is still sent, a
security event is logged, and the missing entry shows as a gap in the
run's sequence numbers. Approval prompts behave the same way. Making
these fail closed is an operator policy; it would fit as an `http` class
of `audit.host_fail_closed` from #2003 once both have landed.
- **Host-side credentials.**
- A header value can name a manifest-declared secret as
`{{secret:NAME}}`, and the host substitutes it on the wire.
- Undeclared names are refused and recorded. Placeholders are stripped
on a cross-origin redirect.
- Commitments cover the placeholder form, and the entry lists injected
secret names, never values.
  - `docs/models.md` describes how a provider capsule adopts this.
- **Tool calls and approvals.**
- One `CapsuleToolCall` entry per `tool.v1.execute.<tool>` invocation,
with the call id, argument and result hashes, and the capsule.
- Approval prompts are committed before they are published. Every
decision is linked to its prompt, with how it was reached: user, session
allowance, remembered consent, timeout and so on.
- Grant-on-use prompts are committed by the dispatcher before
`GrantRequired` is published. An approval is committed before
`GrantResult` is published, so the chain shows it before the grant it
caused.
- **Capability changes.** Applied changes are recorded on the affected
principal's chain after they are saved: token mint and revoke, caps
grant and revoke (only the patterns actually added), group and capsule
membership changes, distro self-grants, and grant-on-use.
- **Code identity.**
  - `CapsuleInstalled` binds the wasm and exact `Capsule.toml` hashes.
- `CapsuleLoaded` (on load or replace) binds the verified wasm hash, the
manifest hash and the engine profile.
- Install and upgrade hooks run by the daemon get the signed sink,
attributed to the hook's component.
- **Attribution.**
- Host-call entries name the capsule and wasm hash, stamped by the host
from the verified component; a guest cannot choose them.
  - `write-file` reports the hash of the bytes written.
- Changelog fragment.

## Verification

Current signed integration head: **111 audit tests passed / 2 ignored;
777 capsule tests passed; 643 kernel tests passed / 1 ignored** (1,531
passed in total). Commands: `cargo test --locked -p astrid-audit -p
astrid-capsule -p astrid-kernel --lib -- --quiet`; `cargo check --locked
--workspace`; `cargo clippy --locked -p astrid-audit -p astrid-capsule
-p astrid-kernel --lib --tests -- -D warnings`; formatting and diff
checks. All passed locally on macOS. Added integration regressions bind
capsule/wasm identity in folded digests and prove coverage events remain
individual while overflow binds the original payload. The attribution
test now asserts FIFO order across alternating capsules instead of
expecting the old unordered folding behavior. Fresh platform CI is still
required.

Prior author verification before integration:

- **Test suites:** `astrid-audit` 71, `astrid-capsule` 777,
`astrid-capsule-install` 106 and CLI 818 all pass. The kernel lib passes
601 of 602 with `--test-threads=6` and umask 0022. The remaining failure
needs a copy-on-write workspace backend and also fails on `main`.
- **Unit and integration tests:**
- legacy JSON encodings pinned, and a legacy entry re-verified after a
round trip;
  - pre-commits resolve before a loopback server sees the request;
- buffered and streamed responses hashed and linked, including streams
closed early, transport failures and gate denials;
  - redaction: short secrets, many secrets, overlapping secrets;
- credential injection end to end: the server receives the secret, the
guest never reads it, and the commitment covers the placeholder;
- placeholder rules: undeclared, malformed and unterminated placeholders
refused, also after a blank secret, and an omitted header names no
secret;
- tool results captured through the IPC host function, with error,
missing, mismatched and cancelled cases;
  - approval prompts committed before they are visible on the bus;
- capability grant, revoke and membership entries, with none for a no-op
re-grant, and a grant-on-use approval ahead of its grant;
- a daemon install producing `CapsuleInstalled` and `CapsuleLoaded` with
the expected hashes, and an upgrade recorded as load then replace.
- **Lint:** `cargo fmt` and `cargo clippy --all-features --all-targets
-- -D warnings` on the touched crates. `cargo check --target
wasm32-unknown-unknown` of `astrid-audit`, `astrid-capsule` and
`astrid-kernel`.

## AI / Tool Assistance

Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation
and tests in all touched crates, and this description.
Assisted-by: Codex CLI. Review passes over the diff.

Integration onto the landed recorder and associated regression tests
assisted by Codex. The earlier source review does not constitute
independent review of these integration edits.

## Checklist

- [x] Linked to an issue
- [x] Changelog fragment added under `changes/{issue}.{kind}.md`
(docs/CI-only may skip; release PRs roll fragments into the version
section instead of adding one)
- [x] I understand every change in this PR and can explain its design,
risks, and validation.
- [x] I reviewed and tested any meaningful tool-generated output
included in this PR.
- [x] Every non-bot, non-merge commit has a matching `Signed-off-by`
trailer.

---------

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
Co-authored-by: Joshua J. Bouw <jjb@unicity-labs.com>
joshuajbouw added a commit that referenced this pull request Sep 29, 2026
…d-key verification (#2005)

## Linked Issue

Closes #2000

Part of #1997. The current integration includes the landed #1996, #2002,
#2003 and #2004. Proposed merge order: #1996, #2002, #2003 (ordered
recording), #2004 (recording coverage), this PR.

v2 derives an entry's sections from the serde form of the action,
authorization and outcome. The new action variants in #2003 and #2004
therefore encode without any change to this format or its specification.

## Summary

Format v1 cannot be verified outside Astrid with confidence:

- its signed bytes mix binary fields with `serde_json` of the action and
authorization, and drop either one if serialization fails;
- it signs whole-second time, and only a success bit of the outcome;
- it has no sequence number;
- verification trusts the public key embedded in each entry, so anyone
with write access to the store can re-sign a rewritten chain under any
key and still pass;
- one runtime key signs audit entries, capability tokens and builds.

This PR adds format v2, which is off by default and enabled with
`[audit] entry_format = "v2"`:

- a deterministic CBOR body that signs every stored field;
- a per-chain sequence number;
- a dedicated audit key;
- verification against a cross-signed key registry instead of the
embedded key.

v1 history is kept as it is, and the v2 chain links to it.

## Changes

- **`astrid-crypto`:** `verify_strict` (RFC 8032 strict Ed25519) and a
`random_bytes` helper.
- **`astrid-audit`: `entry_v2`.** The module documentation is the
byte-level specification, with a known-answer test that an
implementation written from the specification alone reproduces.
  - The body (RFC 8949 §4.2.1 deterministic CBOR) carries:
    - the format tag;
- a chain id derived from the registry, session, principal UID and
alias;
    - a sequence number and the previous hash;
    - nanosecond time, entry and session ids, and the acting capsule;
- the action, authorization and outcome as `[tag, {field =>
commitment}]` sections derived from their serde form;
    - the signer key and its key epoch.
- Field values are salted commitments. Salts are HMAC-SHA256 of a
per-entry salt key, so one field can be disclosed without the others.
- The entry hash is SHA-256 of the body, and the audit key signs a
domain-separated wrapper of it.
- **`astrid-audit`: key registry.**
- A hash chain of records binding keys to roles: audit, capability,
build and audit-v1.
- The genesis is signed by every key it binds, and a rotation by both
the old and the new key. A key can never be registered twice.
- **`astrid-audit`: verification.** `ChainVerifier` accepts a v2 entry
only if all of these hold:
  - its signer holds the audit role at the entry's epoch;
  - its chain id derives from the registry;
  - sequence numbers run 1, 2, 3;
  - links hold;
  - epochs never decrease.

With a registry present, v1 entries and archive receipts must be signed
by registered keys. v2 receipts carry the signer's epoch.
- **`astrid-audit`: writing.**
  - `AuditLog::enable_entry_v2` writes the genesis on first use.
- The first v2 entry of each chain has sequence 1 and links to the
chain's last v1 entry.
  - After that, v1 appends are refused (`V1Closed`).
- Storage enforces the v1 closure and the current key epoch at commit
time, under the durable append lock, so another opener's switch or
rotation cannot be bypassed.
- The same check applies to a prune before anything is deleted: a new
deletion plan is accepted only if its receipt's signer is the registry's
current audit key (a receipt without an epoch only while no registry
exists). A plan accepted earlier still finishes after a rotation.
  - `rotate_audit_key` appends a cross-signed rotation.
- **`astrid-config`, `astrid-kernel`: `[audit] entry_format`.** It is
operator-only, and the default is `"v1"`.
- With `"v2"`, the kernel loads or creates `keys/audit.key`
(owner-only), enables v2, and on first use writes a genesis that
registers the runtime key for the capability, build and v1-audit roles.
  - A store with a registry stays on v2.
- Boot stops if the registry does not verify, if the audit key is
missing or different, or if the configuration cannot be read on a store
that is not yet on v2.
  - `keys/audit.key` is protected from admin filesystem edits.
  - It applies on every native host.
- **Docs and changelog.**
  - `docs/config.md` documents the switch and its one-way migration.
  - Rolling back to a release without v2 is not supported.
  - Changelog fragment.

With v2 off, v1 behaviour, verification results and archive keys are
unchanged.

## Verification

### Current stack integration

Head `0dd52fdd48fd25eab5c778353ecadd9701033ff8` reconciles landed #2004
(`3b3360fd`) with integration `c225dfc7`. Its tree is byte-identical to
`c225dfc7` (tree `c81645720ec1d81d2c1e447aecdf328f54b4e0a0`). All
Actions jobs, including both MUSL smokes, passed on that tree. Three
CodeQL hard-coded-value alerts were individually investigated and
dismissed as false positives: an HMAC output buffer fully overwritten
before return, a test-only known-answer key, and an OS-randomness buffer
that cannot return on RNG failure. No scanner rule or test was weakened.
Required checks on the ancestry-only commit must still complete before
merge.

- Retains both anchor-watermark and signing-key-epoch checks before
prune-plan acceptance, as well as ordered host recording and capsule
attribution.
- Keeps `entry_format`, retention settings and fail-closed settings in
the shared `AuditConfig`, with both operator-only restrictions intact.
- Adds regressions for v2 pruning at the anchor boundary and commitment
to host-stamped capsule identity.
- The forged-receipt test still requires signature rejection. It now
restores the authentic receipt before its subsequent legitimate-prune
scenario, because immutable receipt history correctly rejects replacing
that generation.
- Audit: 171 passed, 2 ignored. Config: 143 passed. Crypto: 28 passed.
Kernel: 648 passed, 1 ignored with `--test-threads=4`. The first
parallel kernel run hit two unchanged 2-second response timeouts; all
three tests in that group passed unchanged in an isolated rerun (0.71s),
then the complete four-thread kernel run passed (51.72s).
- Audit doctests: 2 passed. Workspace `cargo check --locked
--workspace`, formatting and focused audit/config/crypto/kernel Clippy
(`--lib --tests -- -D warnings`) passed.
- `cargo check --locked -p astrid-kernel --target
wasm32-unknown-unknown` passed, with unused/dead-code warnings; this is
build evidence, not a browser runtime test.
- The prior independent review applies to the earlier source head. These
integration checks do not claim a new independent review; completed CI
on the identical earlier tree is distinguished from checks on the
current commit.

### Original author validation (before integration)

- **Test suites:** `astrid-audit` 123 (+2 doc), `astrid-config` 140 and
`astrid-crypto` 31 all pass. The kernel lib passes 592 of 593 with
`--test-threads=6` and umask 0022. The remaining failure needs a
copy-on-write workspace backend and also fails on `main`.
- **Unit and integration tests:**
- the known-answer test, and CBOR against RFC 8949 vectors, with
non-canonical input rejected;
- every stored field covered by the signature: each JSON leaf of a
stored entry, and each field of every action, authorization and outcome
variant;
  - sequence gaps, including an entry deleted from the store;
- unregistered, foreign, retired and backdated keys rejected, v1 after
v2 rejected, and epoch regression across a chain change detected;
  - registry genesis, rotation and tamper rules;
- v1 to v2 migration with mixed chains, persistence and rotation across
restart, and principal UID binding;
  - concurrent single and batch appends producing gap-free sequences;
- commit-time refusal of v1 after another opener enables v2, and of a
stale epoch after another opener rotates;
- archive receipts bound to a registry epoch, and forged receipts
rejected;
- a stale handle's prune refused, with entries, receipts and
verification unchanged: one without v2 after another enabled it, and one
at epoch 0 after another rotated to epoch 1; a plan accepted before a
rotation finishes after it;
- kernel boot: v1 default, key creation, v2 persistence with the setting
back at v1, and a missing or replaced key or unreadable config stopping
boot.
- **Scratch daemon:** boot on v1, switch to v2, then restart with the
setting back at v1. The chain reads 8 v1 entries followed by v2 entries
1 to 10, bound to the principal UID, under one registry. It verifies
with registered v1 keys required.
- **Lint:** `cargo fmt` and `cargo clippy --all-features --all-targets
-- -D warnings` on the touched crates. `cargo check --target
wasm32-unknown-unknown` of `astrid-kernel`.

## AI / Tool Assistance

Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation
and tests in all touched crates, the format specification, and this
description.
Assisted-by: Codex CLI. Review passes over the diff.

Merge after #2004 and successful checks on the current integration.

## Checklist

- [x] Linked to an issue
- [x] Changelog fragment added under `changes/{issue}.{kind}.md`
(docs/CI-only may skip; release PRs roll fragments into the version
section instead of adding one)
- [x] I understand every change in this PR and can explain its design,
risks, and validation.
- [x] I reviewed and tested any meaningful tool-generated output
included in this PR.
- [x] Every non-bot, non-merge commit has a matching `Signed-off-by`
trailer.

---------

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
Co-authored-by: Joshua J. Bouw <jjb@unicity-labs.com>
joshuajbouw pushed a commit that referenced this pull request Sep 30, 2026
## Linked Issue

Closes #2011

Follow-up to #1996 and #2005.

## Summary

`audit.export` reported every entry's `content_hash_hex` as
`BLAKE3(signing_data)`. For a format-v1 entry that is its content hash.
For a format-v2 entry the signing data is the signature wrapper of the
entry hash, while chain links, chain heads and prune receipts use the
SHA-256 entry hash of the canonical body. On a chain switched to v2, an
exported entry's hash therefore matched neither the next entry's
`previous_hash_hex` nor the head in `audit.heads`, and a verifier
following the export could not link the chain.

The export now reports `AuditEntry::content_hash()`, which is unchanged
for v1 entries.

## Changes

- **`astrid-kernel`: `audit.export`.** `content_hash_hex` is the entry's
content hash in either format: `BLAKE3(signing_data)` for v1, the
SHA-256 entry hash for v2.
- **`astrid-core`: export wire types.** Their documentation states which
hash each format exports, that v2 signing data is the signature wrapper
of that hash, and that only a v1 signature covers whole seconds.
- Changelog fragment.

## Verification

- **New kernel test**
`export_reports_the_hash_that_links_a_chain_switched_to_v2`. It exports
a chain with two v1 entries followed by three v2 entries and checks:
  - each exported hash against the stored entry;
  - the signature over the exported signing data;
  - that each v2 entry's signing data wraps its entry hash;
  - the links across the switch;
  - the head in the export page and in `audit.heads`.

It fails on the previous hash computation (the v2 hashes differ) and
passes with this change.
- **Test suites:** `astrid-core` 391 pass. The kernel lib passes 647 of
648 with `--test-threads=6` and umask 0022. The remaining failure needs
a copy-on-write workspace backend and also fails on `main`.
- **Lint and changelog:** `cargo fmt`, `cargo clippy --all-features
--all-targets -- -D warnings` on `astrid-core` and `astrid-kernel`, and
`scripts/changelog.py check`.

## AI / Tool Assistance

Assisted-by: Claude Code:claude-opus-5-5. Covers the fix, the test, the
documentation and this description.
Assisted-by: Codex CLI. Review pass over the diff.

## Checklist

- [x] Linked to an issue
- [x] Changelog fragment added under `changes/{issue}.{kind}.md`
(docs/CI-only may skip; release PRs roll fragments into the version
section instead of adding one)
- [x] I understand every change in this PR and can explain its design,
risks, and validation.
- [x] I reviewed and tested any meaningful tool-generated output
included in this PR.
- [x] Every non-bot, non-merge commit has a matching `Signed-off-by`
trailer.

---------

Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(audit,cli): signed chain-head snapshot and raw chain export for external anchoring

3 participants