feat(audit): canonical, fully signed entry format (v2) with registered-key verification - #2005
Merged
joshuajbouw merged 34 commits intoSep 29, 2026
Merged
Conversation
Signature::verify uses ed25519-dalek's default verification, which accepts some signatures that other implementations reject (small-order public keys and R points). A format whose signatures must verify the same way in every language needs RFC 8032 strict verification. Signature::verify_strict and PublicKey::verify_strict provide it; existing callers keep verify. random_bytes::<N>() fills an array from the OS CSPRNG, for secret salts and nonces, so callers outside this crate do not need their own RNG dependency. Tests: strict verification accepts a valid signature and rejects a wrong message and the all-identity forgery under the identity (small-order) key; two random_bytes draws differ and are non-zero. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Format v1 cannot be verified outside Astrid with confidence. Its signed bytes mix binary fields with serde_json of the action and authorization, and drop either one when serialization fails. It signs whole-second time and one success bit instead of the outcome, and has no sequence number. Verification trusts the public key embedded in each entry, so anyone with write access to the store can re-sign a rewritten chain under any key and still pass. The runtime key signs audit entries, capability tokens and builds alike. Format v2 (astrid_audit::entry_v2) signs a deterministic CBOR body (RFC 8949 §4.2.1) that covers every stored field. The body carries: - the format tag; - a chain id derived from the key registry, session, principal UID and alias; - a per-chain sequence number and the previous entry hash; - nanosecond time, the entry and session ids, and the acting capsule; - the action, authorization and full outcome as [kind, field map] sections; - the signer key and the key epoch it was signed under. Text fields and hashes of caller content are salted commitments. Salts are HMAC-SHA256 of a random per-entry salt key, so one field can be disclosed without the others. The entry hash is SHA-256 of the body. The audit key signs a domain-separated wrapper of that hash, verified strictly. Keys come from a key registry stored with the log: a hash chain of records binding keys to roles (audit, capability, build, audit-v1). The genesis is signed by every key it binds. A rotation is signed by both the old and the new key. A new key can never have been registered before. ChainVerifier accepts a v2 entry only if its signer holds the audit role at the entry's key epoch and its chain id derives from the registry. It also requires sequence numbers to run 1, 2, 3 per chain, links to hold, and key epochs never to decrease. A chain rewritten under an unregistered key, a gap and a v1 entry after v2 are reported as ChainIssue variants; ChainIssue is now non_exhaustive. EntryV2Header and verify_entry_v2_body verify the bytes-only (redacted) form. AuditLog::enable_entry_v2 writes the genesis on first use, or checks that the supplied key is the active audit key. Appends then sign v2. The first v2 entry of each storage chain has sequence 1 and links to the chain's last v1 entry, so v1 history stays as it is and remains hash-linked. After that the log refuses v1 appends (AuditError::V1Closed), also for a later opener that has not enabled v2. rotate_audit_key appends a cross-signed rotation and switches keys. Archive receipts are signed by the audit key under v2. For a v2 first retained entry, a receipt counts only if the registry lists its key. append_with_actor records the acting capsule. With v2 off, v1 behaviour, verification results and archive keys are unchanged. The module documentation is the byte-level specification. It includes a known-answer test: fixed keys, registry genesis and one entry, with the expected body, hashes, signatures and field salts. The chain verification loops now share ChainVerifier. The test-only storage hooks moved to storage/test_hooks.rs to keep storage.rs under 1000 lines. The crate depends on hmac 0.13, already in the lockfile through other crates. Tests (44 new, all 65 existing pass): - the KAT, also reproduced independently from the specification text; - CBOR encoding against RFC 8949 vectors, and rejection of non-canonical input; - every stored field covered by the signature, checked by editing each JSON leaf of a stored entry and each field of every action, authorization and outcome variant; - sequence gaps, including an entry deleted from the store; - rejection of unregistered keys, foreign registries, retired and backdated keys, and v1 after v2; - registry genesis, rotation and tamper rules; - v1 to v2 migration with mixed chains; - restart persistence, v1 refusal after reopening, rotation across restart, principal UID binding, batch appends, actors, and forged prune receipts. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…format Format v2 is off unless the operator sets `[audit] entry_format = "v2"`, so maintainers decide when it rolls out. The default stays "v1". Only the operator's own configuration can set the key; a workspace config layer that touches it is reverted, like other operator-only settings. Enabling v2 is one-way and creates keys. The kernel applies the setting right after the runtime tree is admitted, before anything can append. With v2 it: - loads or creates keys/audit.key, owner-only, next to runtime.key; - enables v2 on the audit log with the principal directory for UID binding; - on first use, writes a registry genesis that binds the audit key and records the runtime key as the capability, build and v1-audit key. The runtime key keeps signing capability tokens and builds. Moving each of those roles to its own key later is a cross-signed registry rotation. Once the store has a registry, the kernel stays on v2 even if the setting goes back to "v1" and logs a warning, because the log refuses v1 appends there. It refuses to boot when the registry does not verify, when keys/audit.key is missing (a replacement cannot be registered without the old key, so none is generated), or when the key file holds a different key. keys/audit.key is protected from admin filesystem edits like runtime.key. docs/config.md documents the switch and the migration semantics, and changes/2000.added.md records the addition. Tests: - config: v1 is the default; only "v1" and "v2" parse; a workspace layer can neither enable v2 nor reset an operator's v2; - kernel: v1 leaves the log and the keys directory alone; v2 creates a separate private audit key and registers every role; v2 survives a reopen with the setting back at v1; a missing or replaced audit key stops boot and no replacement is generated. A scratch daemon was also run on a temporary home: boot on v1, switch to v2, then restart with the setting back at v1. The stored chain reads 8 v1 entries followed by v2 entries 1 to 10, bound to the principal UID, under one registry across restarts. It verifies with the log's verifier and with registered v1 keys required. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…Unix The kernel read audit.entry_format and set up the audit key and registry only under cfg(unix). A kernel composed on another native platform through with_resources would silently keep writing v1 entries with the setting at "v2". The platform file helpers and key storage used by that path work on every native target, so the setting is now applied wherever the kernel reads host configuration, which excludes only the browser profile (wasm32-unknown-unknown). That profile reads no host configuration; its host enables v2 on the audit log it injects, as docs/config.md now states. Verified with clippy on astrid-kernel (all targets), the audit_keys tests, and cargo check for wasm32-unknown-unknown. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…only releases A release without format v2 ignores the v2 data of stored entries, so it cannot verify them, and it has no v1-after-v2 guard, so it would append v1 entries after v2 ones. The entry_format documentation now says that going back to such a release after enabling v2 is not supported. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Sequence numbers are assigned while a chain's head is resolved. Single appends hold the chain lock, while batch appends rely on the storage compare-and-swap and re-sign on conflict, so the two paths can race on one chain. The new test runs eight tasks mixing single appends and two-entry batches on one principal chain. The chain must come out as sequence 1 to 48 without gaps and pass verify_chain. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
A v2 chain's first retained entry after a prune is anchored by an archive receipt. The verifier accepted a receipt signed by any key ever registered for the audit role. Once a retired audit key leaked, someone able to write the store could delete a prefix and plant a receipt signed by that key, and the chain would still verify. Receipts written under format v2 now carry the signer's key epoch, which the signature covers. For a v2 first entry, a receipt is accepted only if: - it names an epoch; - that epoch is no earlier than the entry's key epoch (a receipt is always written after the entries it keeps); - its key holds the audit role in that registry state. A retired key can therefore no longer anchor a suffix signed after its retirement. This is the same bound the entries themselves have. v1 receipts omit the field and keep their signed bytes, so existing receipts and their prior-receipt hashes are unchanged. Tests: after a rotation, a prune keeps a suffix signed at epoch 1. The honest receipt names epoch 1 and verifies. The retired key's self-consistent forgeries naming epoch 0 or 1 are rejected, and so is a new-key receipt with the epoch removed. The existing prune tests pass unchanged. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
The kernel read audit.entry_format with a fallback to the default. A config that failed to load or validate therefore selected v1 without notice, even where the operator had configured v2. Entries would then be written in the weaker format until someone noticed. The kernel now passes the load result through. When the configuration cannot be read, boot stops with the reason, unless the store already holds a key registry: such a store never leaves v2, so it continues on v2 with a warning. Tests: an unreadable configuration on a fresh store fails boot with the reason and creates no key. The same configuration on a store already on v2 keeps v2. The other audit_keys tests pass with the configured format passed as a result. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…time Two guarantees of format v2 were held only in the memory of one AuditLog: - "v1 is closed once a key registry exists": a log cached, on its first v1 append, that the store had no registry. If another opener enabled v2 later, this log kept appending v1 entries, including to chains that had not yet received a v2 entry. - "nothing is signed at a retired epoch": a log signed with the registry snapshot it held. After another opener rotated the audit key, or when an append picked up the old signer just before a rotation in the same log, the old key kept producing entries at the old epoch. Verification accepts those, because the key was active at the epoch they name. Storage now checks the registry when it commits, under the process-wide durable append lock, which registry record writes also take: - a v1 entry is refused (V1Closed) once registry record 0 exists; - a v2 entry is refused (StaleAuditKey) once a record newer than its key epoch exists. Both checks are point reads, and they cover single appends, atomic batches and the per-entry fallback. The log re-signs a refused single append when its own signing state has moved on (v2 enabled or key rotated in this log). The retry is safe because a single append commits all or nothing. Otherwise the refusal is returned, so a writer left behind by another opener's rotation fails closed. A refused batch fails as it does for other storage errors: the per-entry fallback may have committed part of it, so signing it again could duplicate entries. The per-log "v1 closed" cache is removed, and sign_entry is now synchronous. The specification states the writer-side rule. It also notes that bounding what a leaked retired key can sign outside Astrid needs an external anchor of the chain heads at rotation. Tests: - a second opener enabling v2 closes both an existing v1 chain and new chains to the first opener; - an entry signed before a rotation is refused at commit, and the same log's next append re-signs at epoch 1 with the new key; - a writer left behind by another opener's rotation gets StaleAuditKey and writes nothing; - the existing v1-refusal and rotation tests pass unchanged. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…try exists With a key registry present, the log still verified v1 entries against the key embedded in each one. After v2 was enabled, a dormant v1-only chain could therefore be replaced by a chain re-signed under any key and still verify. That is the weakness format v2 addresses, left open for v1 history, although the registry genesis records the key that signed that history (the runtime key, under the audit-v1 role). ChainVerifier, and with it AuditLog::verify_chain, now requires a v1 entry's key to hold the audit-v1 role whenever a registry is given. Signatures and links are checked as before. Without a registry nothing changes. require_registered_v1_keys(false) restores the embedded-key rule for callers that want it. v1 entries signed by an earlier runtime key, for example one replaced before v2 was enabled, are reported as UnregisteredKey. The specification, docs/config.md and the verifier documentation state this. Tests: - a store holding a v1 chain under another key reports every entry UnregisteredKey once v2 is enabled, where plain v1 verification accepted it; - with the check turned off, or without a registry, the same chain verifies; - v1 history signed by the registered key, followed by v2, verifies with the default verifier. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…overflowing sequences Three verifier gaps: - The key epoch was compared with the predecessor's only when both entries had the same chain id. After a rotation, the retired key could therefore append an entry at its old epoch to the current head, as long as a changed principal UID gave it a new chain id at sequence 1. A storage chain's entries are signed in order, each at the registry head of its time, so the epoch never legitimately decreases, whatever the chain id. The verifier now reports KeyEpochRegression whenever a v2 entry's epoch is below its v2 predecessor's. The writer applies the same check to any v2 head. - EntryV2Header::decode accepted sequence 0, so verify_entry_v2_body could pass a signed malformed body. It is now rejected. - A predecessor at u64::MAX saturated the expected sequence, so a duplicate terminal sequence passed. It is now reported as MalformedEntry. The specification states these rules. Tests: the retired key opening a "new" chain at epoch 0 after an epoch-1 head is reported; a detached chain with two entries at u64::MAX is reported as malformed; a signed body with sequence 0 fails decoding and verification. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…oads The audit log has no entry kinds for kernel-mediated HTTP, principal grant changes or capsule install/load, and host-call entries cannot say which capsule acted. Approval and tool-call entries have no fields to link a decision to its request or to commit to a tool result. Add, without changing the v1 entry format: - HttpRequest (pre-commit: sequence, method, host, port, path, header and body hashes, redirect hop, injected secret names) and HttpResponse (completion: sequence, request entry id, status, body hash and length, completeness, provider request ids); - CapabilityChanged for principal capability, capsule and group grants; - CapsuleInstalled and CapsuleLoaded binding the wasm hash, manifest hash and engine profile; - a CapsuleActor (capsule id + wasm hash) field on file, network, process, tool-call and approval entries; - call_id and result_hash on CapsuleToolCall, and request_id, request_entry_id and via on the approval entries. New fields on existing variants are optional and omitted when unset, and new variants are appended after the existing ones. An entry written before this change therefore re-serializes to the same signed bytes and still verifies; the new fields are covered by the signature because the action JSON is part of the signing data. The action descriptions and entry tests move into entry/ to keep entry.rs under the file-size cap. Verified with the astrid-audit tests, including new tests that pin the legacy JSON encoding of variants with unset fields, re-verify a legacy entry after a parse/serialize round trip, and show that changing an attributed capsule invalidates the signature; clippy -D warnings on astrid-audit; cargo check of the workspace. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
… writes Host-call audit entries (file, network, process) name the principal but not the capsule that made the call, and every FileWrite entry records the zero hash because the host never reported the written bytes. An auditor cannot tell which code acted or what it wrote. HostAuditSink gains `attributed(actor)`, which returns a sink that stamps a HostAuditActor (capsule id + BLAKE3 of the verified wasm component) on every record. The wasm engine binds the kernel sink this way at load, using the hash it just verified against the install metadata, and the lifecycle (install/upgrade hook) path binds it with the hook component's hash. The actor is therefore host-owned; a guest cannot choose it. The kernel sink maps it onto the new `actor` field of the audit action. `write-file` now reports the BLAKE3 of the bytes it wrote and the kernel records it as the FileWrite content hash. Directory creation and writes denied before any content was accepted keep the zero hash. The bounded writer folds allowed calls of one class per principal; the fold key now includes the actor, so one capsule's calls are never folded into a row attributed to another capsule. The writer is otherwise unchanged; the actor helpers live in new audit_sink/coverage.rs modules in both crates. Verified with new kernel tests (attributed handles stamp the capsule identity and the kernel's own handle does not; two capsules' reads in one window stay separate rows; write hashes are recorded), new capsule tests (the write-file host call reports the content hash; a sink without attribution support is returned unchanged), the existing audit_sink tests in both crates, and clippy -D warnings on astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
The v2 body encoded the action, authorization and outcome through a hand-written table: a numeric kind per variant and a numeric key, a public-or-committed choice and a CBOR type per field. Every new AuditAction variant then needed a table row and a specification change before an entry could be signed. The host-call recording variants planned in astrid-runtime#1999 are one example. A section is now derived from the enum's stored serde form, which is an internally tagged object: - the section is [tag value, {field name => salted commitment}]; - the tag is `type` for the action and the authorization, `status` for the outcome; - each field's committed value is its JSON value mapped to CBOR by the existing JSON rule. A new variant, or a new optional field skipped when absent, is encoded without any change to the encoder, the verifier or the specification. The encoding of existing entries does not change. The specification now says that the serde form of these enums is part of the format. Every field is committed; there are no public fields any more. The body shows which variant and which fields an entry has, never their values. Salts and commitments are keyed by the field name instead of a numeric key. Field disclosures carry the name and come out in body order. Building a body can now fail, if a section has no tagged object form. Such an entry is refused before signing. Its content hash is a fixed value that no signature or link can match, and the verifier reports it as MalformedEntry. AuditEntry::v2_body and v2_field_disclosures return a Result. The known-answer values for the body, entry hash, signature and field salts change. The new values were reproduced by an implementation written from the updated specification alone. Registry and chain-id values are unchanged. Tests: the KAT with the new values; a variant defined only in the test, with a nested struct and a skipped optional field, encodes by the same rule; a value without a tagged object form is refused; every field of every variant still changes the body; all 31 action variants have distinct kinds. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Every model call an agent makes goes through the capsule HTTP host, but the host only logged it with tracing::debug!. The signed audit log had no record of a request or its response, so an agent could re-ask a model without leaving any trace. Each wire request (each redirect hop included) now produces: - an HttpRequest pre-commit: method, host, port, BLAKE3 hashes of the path+query, the capsule-supplied headers and the body, the body length, the redirect hop, and a kernel-assigned per-principal sequence number. The host awaits a durable append (new HostAuditSink::commit) after the scheme, egress and security-gate checks and before the request is sent, so the entry precedes the request on the chain; - an HttpResponse completion carrying the same sequence and the pre-commit's entry id, the status, provider request ids (x-request-id, request-id and similar), and a BLAKE3 hash of the response body computed over the bytes as the host reads them — incrementally for streams, which complete at end of body, on a read error, or when the guest closes the stream. An exchange dropped before completion (cancelled host call, store teardown) still records a failed completion; - a denied HttpRequest when the principal egress check, the security gate or the SSRF airlock refuses the request. Every HttpRequest entry takes the next number of its principal's sequence, including denials and pre-commits whose append failed, so a missing entry shows as a gap. A failed pre-commit append does not fail the request; it is logged as a security event, matching the existing continue-and-alert behaviour. No content is stored. Commitments are computed after redaction: values of credential headers (authorization, cookie, x-api-key, ...) and credential query parameters (key, api_key, access_token, ...) become "[REDACTED]", as does every occurrence of a secret value get_config handed to the capsule instance. The host remembers those values (zeroized on drop) only for this purpose. The canonical header form and the redaction rules are documented in the http audit module so a verifier holding a request can recompute its commitments. The redirect decision moves into a redirect_step helper so each followed or failed hop can complete its exchange; behaviour is unchanged. The new record types and their kernel mapping live in the audit_sink/coverage.rs modules; the kernel's bounded writer is unchanged. Verified with new capsule tests against loopback servers (the pre-commit resolves before the server sees the request; buffered and streamed responses are hashed and linked to their pre-commit; a stream closed early completes incomplete; a transport failure completes without status; a gate denial is recorded and never sent), unit tests of the redaction and commitment forms (a secret never reaches the hashed bytes), new kernel tests (pre-commits are readable when commit returns and are numbered per principal; denials take a number; completions link to their request and bound provider ids), the existing HTTP host tests, and clippy -D warnings on astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Provider capsules read their API key with get_config and set the
Authorization header themselves (aos-ce capsule-openai-compat does this),
so the key lives in guest memory and passes through the request the audit
log commits to.
A capsule can now name a secret in a request header value with the
placeholder {{secret:NAME}}, for example
`Authorization: Bearer {{secret:api_key}}`. The HTTP host substitutes the
value when it builds the wire request:
- NAME must be a secret-typed [env] key declared in the capsule manifest;
any other name refuses the request with capability-denied and records a
denied HttpRequest entry. The value is resolved like get_config
(invoking principal first, then host-wide) and trimmed.
- An unset or blank secret omits the header, so keyless endpoints still
receive no credential header.
- Only header values are substituted; placeholders in the URL or body are
sent as written. Injected values are marked sensitive.
- On a cross-origin redirect every header carrying a placeholder is
dropped along with Authorization and Cookie, so a secret is only sent to
the origin the capsule addressed.
- The audit commitment is computed over the placeholder form, and the
HttpRequest entry lists the injected secret names, never values.
Capsules that keep reading secrets themselves are unaffected: their values
are still redacted from commitments. docs/models.md describes the provider
request entries and how a capsule adopts injection.
Verified with new capsule tests: substitution rules (multiple and repeated
placeholders, trimming, blank secret omits the header, malformed and
undeclared placeholders and CR/LF values refused, cross-origin strip); an
end-to-end request through the HTTP host where the loopback server
receives the secret, the guest never read it, and the pre-commit's header
hash equals the hash of the placeholder form and names the secret; and an
undeclared placeholder that is refused and recorded without anything being
sent. Existing HTTP host tests and clippy -D warnings on astrid-capsule
pass.
Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
…ned v1 entry Once a store has a key registry, v1 entries must be signed by the registered audit-v1 key. Archive receipts did not follow the same rule. When a prune left a v1 entry at the start of the retained suffix, the receipt was checked only under the key embedded in it. Someone able to write the store could plant a self-consistent receipt under their own key and make an arbitrary predecessor hash look anchored. With a registry, every receipt must now be signed by a key the registry lists: - a receipt without a key epoch (written before v2 was enabled) anchors only a v1 first entry, and must be signed by the registered audit-v1 key; - a receipt with an epoch must be signed by a key holding the audit role at that epoch, and for a v2 first entry the epoch must be no earlier than the entry's. Without a registry, receipts are checked as in v1. The specification states the rule. Tests: a v1 prefix pruned before v2 still verifies after v2 is enabled; the same receipt re-signed by an unregistered key fails to anchor; a prune under v2 that keeps a v1 entry first yields a receipt at epoch 0 that anchors it. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Tool invocations and approval decisions determine what an agent did, but neither reached the audit log: CapsuleToolCall and the Approval* actions had no producer, so a tool run or a user's "approve always" left no signed record. Tool calls: when the engine delivers a tool.v1.execute.<tool> request to a tool capsule as an interceptor invocation, it arms a ToolCallAudit. The IPC publish host fn captures the ToolExecuteResult the capsule publishes for that call (on the request topic + ".result" or on tool.v1.execute.result, matched by call id). When the invocation ends the engine records one CapsuleToolCall entry with the tool name taken from the topic, the caller's call id, BLAKE3 hashes of the JSON arguments and of the result content, and the capsule identity. The outcome is a failure when the tool reported an error, published no result, the guest call failed, or the invocation was cancelled (the guard records on drop). The capture is per invocation and cleared when the pooled instance is returned. Approvals: request_approval and the local-egress consent flow now record each check through an ApprovalAudit: - before a prompt is published, an ApprovalRequested entry (host-minted request id, action, resource) is committed and awaited; - the decision is an ApprovalGranted (scope once/session/always) or ApprovalDenied entry carrying the request id and the request entry's id, and how it was reached: user, session_allowance, session_grant, remembered_consent, no_request_owner, timeout, cancelled, or error; - a prompt that ends without a decision (an error path) records a denial when the guard is dropped, so every prompt on the log has a decision. The kernel maps these onto CapsuleToolCall, ApprovalRequested, ApprovalGranted and ApprovalDenied with bounded string fields (in audit_sink/coverage.rs), and stamps "kernel-dispatched tool invocation" or "approval gate" as the system authorization reason instead of the manifest-gate reason. Verified with new capsule tests (a result published through the IPC host fn is recorded with argument and result hashes; tool errors, missing or mismatched results, guest failures and cancellation are failures; non-tool invocations are not armed; each user answer is linked to its committed prompt; the prompt is committed before it is observable on the bus; the missing-owner denial and remembered consent are recorded; egress consent prompts and cached-grant decisions are recorded), new kernel mapping tests, the full astrid-capsule lib suite (771 passed), and clippy -D warnings on astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Capability tokens, principal grants and grant-on-use consent change what an agent may do, but none of them left an audit entry describing the change: admin requests were recorded as AdminRequest rows at authorization time (before the change is applied, and without its result), and grant-on-use was not recorded at all. Applied authority changes are now recorded on the chain of the principal whose authority changed, after the change is saved: - admin.caps.token.mint → CapabilityCreated (token id, resource, permissions, scope); - admin.caps.token.revoke → CapabilityRevoked, chained under the token's subject when the token is still known; - admin.caps.grant / admin.caps.revoke → CapabilityChanged (kind "capability") listing only the patterns actually added, so a repeated grant records nothing; - admin.agent.modify → CapabilityChanged for group and capsule membership differences; - admin.distro.self_grant → CapabilityChanged for the capsules granted; - grant-on-use → an ApprovalRequested entry for the dispatcher's prompt, ApprovalGranted (scope always) or ApprovalDenied (user, timeout) for the decision, linked by request id, and CapabilityChanged (via grant_on_use) for the applied capsule grant. The grant-on-use CapabilityChanged entry is appended after the GrantResult is published so the durable append does not delay the caller. A failed append is logged and does not undo the change, as for admin audit rows. The helpers live in the new grant_audit module. Verified with new kernel tests: caps grant/revoke and agent.modify group changes produce the expected CapabilityChanged entries (and none for a no-op re-grant); token mint and revoke produce CapabilityCreated and CapabilityRevoked under the subject; grant-on-use approve and deny produce linked request/decision entries and the grant entry only on approve. The existing grant_on_use tests pass unchanged, and clippy -D warnings on astrid-kernel passes. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Nothing on the audit log said which code a capsule runtime was running: install and load outcomes were not recorded, and an install or upgrade hook's host calls were not audited at all because the install path passed no audit sink to the lifecycle engine. - The daemon install path records CapsuleInstalled once the capsule is installed and activated: capsule id, version, target principal, the BLAKE3 of the installed wasm component and the BLAKE3 of the exact installed Capsule.toml bytes. - Publishing a runtime generation records CapsuleLoaded with trigger "load"; a live restart/upgrade that replaces a running generation records it with trigger "replace". The entry binds the wasm hash from the install metadata (the engine refuses to load bytes that do not match it), the manifest hash of the runtime directory, and the engine profile: "wasm:<compiled engine ABI>" (COMPILED_ENGINE_ABI, now public), plus "mcp-host" for host-process MCP servers, or "static". - InstallOptions gains audit_sink, threaded through the lifecycle entry points to LifecycleConfig. The daemon passes its signed sink, so hook host calls (file writes, network, HTTP) are recorded and attributed to the hook's code identity. The CLI (workspace installs) passes none. The audit appends are boxed inside the helpers so the request-handler future stays under clippy's large-future bound, and grant_audit is gated to native builds like its callers so the wasm32 portability check gains no warnings. Verified with a kernel test that installs a signed one-component capsule through the daemon install handler and finds CapsuleInstalled and CapsuleLoaded with the component's and manifest's BLAKE3 and the wasm engine profile; an assertion in the durable-upgrade test that the old and new generations are recorded as load and replace with their wasm hashes; a capsule test that the lifecycle host state binds the sink to the hook's capsule id and component hash; astrid-capsule-install tests; clippy -D warnings on astrid-capsule, astrid-capsule-install, astrid-kernel and the CLI; and cargo check for wasm32-unknown-unknown of astrid-audit, astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
Summarizes the audit coverage added for astrid-runtime#1998: HTTP pre-commit and completion entries, tool calls, linked approval requests and decisions, applied capability and grant changes, capsule install and load identity, capsule attribution and file-write hashes, audited install hooks, and host-side credential injection. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
The HTTP audit only remembered secret values of at least four bytes and at most 32 distinct values per host state. A shorter value, or any value past the cap, was left out of redaction, so if the capsule put it in a request the signed commitment hashed it in the clear, and a low-entropy secret could be recovered by hashing candidates. Every distinct non-empty value get_config hands out is now remembered and redacted. The set is not guest-growable: values come only from the operator's secret store for the manifest's declared secret keys. Redaction is now one left-to-right pass that replaces the longest secret starting at each position and never rescans replaced text, so the committed form is a deterministic function of the request and the secret set (the previous per-secret passes could rescan the marker), and positions whose byte starts no secret are skipped cheaply. Verified with new tests: a three-byte secret and 40 distinct secrets are all redacted, and overlapping secrets resolve to the longest match without rescanning; the existing HTTP audit and credential tests pass. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
HttpRequest sequence numbers are kept in memory and restart at 1 when the daemon restarts, while the daemon's audit chains are keyed by a session id that stays the same across restarts (the nil session by default). After a restart the same principal chain therefore held two runs of numbers, and a gap could not be told apart from a restart. KernelAuditSink now draws a random run id when it is created and stamps it on every HttpRequest and HttpResponse entry (new run_id field on both variants, which were added in this series). The sequence is defined per principal and run, so a gap within a run means an entry is missing. docs/models.md says so. Verified with the kernel HTTP audit tests (pre-commits carry a non-empty run id and all entries of one kernel share it) and the astrid-audit entry tests; clippy -D warnings on astrid-audit, astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
send_one_hop returned the scheme check's error before any audit record was
built, so a request refused because the caller asked for https-only (or
used a non-HTTP scheme) left no HttpRequest entry and took no sequence
number, unlike requests refused by the egress or security gate.
The scheme refusal is now recorded as a denied HttpRequest ("scheme
denied") whenever the URL parses; the returned error is unchanged.
Verified with a new test that an http:// request with https-only set is
refused with scheme-denied, recorded as denied, and never sent, plus the
existing HTTP host tests and clippy -D warnings on astrid-capsule.
Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
HTTP completion records and approval decisions went through the host-audit queue, which refuses a new unique record once its pending map is full. Under an HTTP burst a durably pre-committed HttpRequest could therefore lose its HttpResponse (status, response hash, provider ids), and an approval prompt committed before it was shown could lose its decision. HostAuditSink::commit now takes the record's outcome, so any record can be appended durably. The HTTP host writes every completion through it: in the async request, redirect, body and stream-read paths it waits for the append; where the exchange ends in a synchronous context (the guest closes or drops a stream, or an unfinished exchange is dropped) it spawns the append on the current runtime instead of queueing it. The decision for an approval whose prompt was committed is appended durably before the host call returns. Decisions made without a prompt (an existing grant), and the denial recorded when an unanswered check is dropped, still use the queue. Verified with the HTTP audit tests (buffered, streamed, early-dropped and failed exchanges complete through the durable path), a new kernel test that a committed completion is readable when commit returns and keeps its failure outcome and run id, the approval and egress-consent audit tests (answered prompts are durable, prompt-less decisions are queued), the full astrid-capsule lib suite, and clippy -D warnings on astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
The grant-on-use prompt (GrantRequired) was recorded by the kernel's approval observer after the dispatcher had already published it, through the droppable host-audit queue. The prompt could be shown and answered before its ApprovalRequested entry existed, and queue pressure could drop it. The dispatcher now commits the ApprovalRequested entry (action "capsule-grant", resource = capsule, the prompt's request id) through HostAuditSink::commit and waits for it before publishing GrantRequired. The sink is carried by CapsuleAccessResolver, which the kernel builds with its audit sink, so the dispatcher's signature is unchanged; the grant signals of one dispatch pass are published after the registry lock is released rather than while it is held. The kernel observer no longer records the prompt and keeps recording the decision and applied grant, linked by the same request id. Verified with a new capsule test that the prompt's entry is committed before the prompt is visible on the bus, the dispatcher and access tests, the grant-on-use kernel tests (decision and grant entries unchanged), and clippy -D warnings on astrid-capsule and astrid-kernel. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
A header value's placeholders were substituted left to right, and the
first one whose secret was unset or blank ended the value: the header was
omitted and the rest of the value was never parsed. An undeclared or
malformed placeholder after it therefore went unchecked, and
`{{secret:unset}} {{secret:undeclared}}` sent the request without the
header instead of refusing it with capability-denied as documented. The
names substituted before the blank one also stayed in the entry's list of
injected secrets although the header was not sent.
Every placeholder in a value is now parsed and resolved before the value
is kept or omitted, so a malformed or undeclared name refuses the request
wherever it appears, and names are recorded only for headers that are
sent.
A test puts an undeclared, a malformed and an unterminated placeholder
after a blank secret, each refused, and checks that an omitted header
names no secret.
Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
The provider-request section said the host waits for the http_request entry before the request leaves, but not what happens when that append fails. As with the rest of the audit log, recording is best-effort: the request is still sent, the failure is logged as a security event, and the missing entry leaves a gap in the run's sequence numbers. Approval prompts behave the same way. The section now says so. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
This was referenced Sep 28, 2026
An approved grant-on-use prompt recorded its ApprovalGranted decision through the host-audit queue, and appended the applied CapabilityChanged entry directly once the grant result was published. The queue is written later, so the chain could show the capsule grant before the approval that caused it. The decision is now committed durably before the grant result is published, as answered approval prompts already are, and the grant change is appended after it as before. A shutdown right after the result therefore cannot lose the decision either. Denials and timeouts add no grant entry and still use the queue. The approve test now checks that the decision precedes the grant on the chain; on the previous code it fails. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
joshuajbouw
requested changes
Sep 29, 2026
Member
There was a problem hiding this comment.
This is a good direction for Astrid: canonical audit records, separate signing keys, and registered-key verification are generic runtime responsibilities, without bringing AOS policy into the kernel.
One blocker on 6313fd30: a stale log handle can prune valid history and leave a receipt that fails verification. I reproduced this after both a v1→v2 switch and an audit-key rotation. Details and the requested regression coverage are inline; no architecture rework needed.
Validation: 294 tests/doctests passed, 2 ignored, plus focused Clippy. Both additional stale-handle pruning probes reproduced the failure. This was focused validation, not a full workspace or platform run.
Two non-blocking notes:
- Numeric JSON parameters can change on storage round-trip and invalidate signatures. Reproduced in the unchanged v1 path too, so this belongs in a separate follow-up.
- External verifiers still need an independently trusted registry identity/checkpoint; cross-signatures alone cannot authenticate a replacement genesis. Keep that trust policy outside Astrid.
…tion Deriving v2 sections from the serde form rewrote the known-answer body in the specification, but left everything after the new body in place: the formulas and field notes of "Entry body" followed it, and then a second copy of the specification from "Chain id" on, with the previous body value and the receipt rule as it stood before receipts that anchor a retained v1 entry required a registered key. A reader following the module documentation met two specifications that disagree. The second copy is removed and the known-answer section again lists the body, entry hash, signature and field salts together; they match the values the known-answer test pins. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
A prune signed its receipt with the signer its log handle held in memory, and storage accepted the deletion plan without consulting the key registry. Appends are checked against the stored registry when they commit; prunes were not. A handle that had not seen another opener enable v2, or rotate the audit key, could still prune: the prune returned Ok and deleted entries, and the chain then failed verification, because the receipt was signed by the runtime key after the registry existed, or by a retired audit key. Storage now checks the receipt's signer before it accepts a new deletion plan, under the durable append lock that registry writes take. A receipt without a key epoch is refused (V1Closed) once any registry record exists, and a receipt must name the latest registry epoch (StaleAuditKey otherwise). A refused prune deletes nothing and writes no receipt. A plan accepted earlier is resumed without the check and finishes under the receipt it was accepted with, so a rotation cannot strand a half-applied prune. The specification states the rule next to the append rule. Signing the receipt is split from persisting it (sign_prune_receipt), so a test can accept a plan before a rotation and finish it after. Tests: a v1 handle pruning after another handle enabled v2, and an epoch-0 handle pruning after another rotated to epoch 1, are refused, and the entries, the absence of a receipt and chain verification are unchanged; the up-to-date handle then prunes and the chain verifies. A plan accepted before a rotation finishes after it and the chain verifies. On the previous code both stale prunes succeed and delete four entries. Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
MastaP
marked this pull request as ready for review
September 29, 2026 13:14
joshuajbouw
previously approved these changes
Sep 29, 2026
joshuajbouw
left a comment
joshuajbouw
left a comment
Member
There was a problem hiding this comment.
Re-reviewed e149501. The stale-signer pruning finding is fixed; approving.
I reran both original independent reproductions unchanged. The stale v1 handle now gets V1Closed, and the retired epoch-0 handle gets StaleAuditKey; neither damages chain verification. The new regressions additionally check that entries and receipts remain unchanged, that a current handle can prune successfully, and that a plan accepted before rotation still finishes afterward.
The check is at the right boundary: accepting a new deletion plan under the same durable lock used for registry writes, before deletion. Resuming an accepted plan remains separate, and the verifier has not been weakened. The documentation cleanup also removes the conflicting duplicate specification.
Validation on this head: audit library 123 passed / 2 ignored; original stale-opener probes 2 passed; audit-library Clippy passed with warnings denied. This closes my previous requested change, not a claim that the full platform CI or the combined PR stack has passed. CI is still pending.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
5 tasks done
Member
|
I checked the failed CodeQL annotations. The reported hard-coded values are the output buffer/fallback in field_salt, a fixed known-answer-test input, and the random_bytes buffer that is filled by SysRng (failure panics). These locations do not establish use of a hard-coded production signing key or salt. Please handle these as narrowly documented false positives, preserving the cryptographic checks rather than changing their semantics to satisfy the scanner. Separately, #1996 has landed with Wasmtime 48.0.3. This branch still conflicts with main and needs the planned integration after #2002–#2004, which will also bring in that dependency fix. My approval of the stale-signer fix remains the source-review evidence; this is not yet merge-ready. |
Contributor
Author
|
An interaction with #1996 that this PR needs once it is on a
Fix: unicity-aos@217a0d8 (branch This PR's branch is unchanged. I can rebase it onto |
joshuajbouw
added a commit
that referenced
this pull request
Sep 29, 2026
## Linked Issue Closes #1999 Part of #1997. #1996, #2002, and the mount cleanup fix #2008 are merged. Current head `c92562e65087fe47e313e4c029781e89a69927d4` is based on main `26a500a6`. Remaining merge order: this PR, #2004 (recording coverage), #2005 (entry format v2). The integration preserves Pavel's ordered host-audit implementation. The overlapping configuration is resolved in the existing `astrid-config::audit` module, retaining both retention validation and fail-closed host-call settings. Kernel startup loads both settings from the same audit configuration. This PR replaces the host-audit writer. #2004 changes the old writer's fold key to include the capsule. Whichever of the two lands second is rebased onto the other, and the capsule then becomes part of the run key. ## Summary The host-audit sink could lose calls and reorder them without a trace: - It coalesced allowed calls per class over `host_coalesce_ms`, and appended `repeats=N` to the outcome details. The entry signature covers the outcome only as a success bit, so that count was not signed. - It drained pending work per key, so a chain's entries did not follow call order. - A call that met the full queue (`host_queue_capacity`, default 4,096 keys) was dropped, and only a counter moved. - A failed batch was dropped, and calls still queued when the daemon died vanished. As a result, even an externally anchored log could not show that it was complete. This PR records host calls in call order, and makes every loss visible as a signed entry on the chain. It also lets an operator make chosen host-call classes fail closed. ## Changes - **`astrid-audit`: signed host-call records.** These actions are covered by the entry signature: - `HostCallRun`: consecutive calls of one principal. It holds the call count, the first and last call times, a tally per class and outcome, and a fold over every call. - `HostCallLoss`: calls that were accepted but not recorded individually. - `HostCallGap`: an earlier run of the lane stopped without draining. - `HostCallAdmitted`: the write-ahead entry of a fail-closed call. `astrid_audit::host_call` specifies the per-call digest and the fold, so a verifier can recompute them without the kernel. Known-answer tests pin them. - **`astrid-kernel`: ordered lane** (`audit_sink/lane.rs`, `writer.rs`). - Each principal chain has a FIFO of pending slots, so FIFO order is call order. - The writer takes slots from the front and appends each batch atomically with `append_batch_with_principal`. - A failed batch is retried as is, with backoff up to 5 s, before anything behind it is taken. - Signing happens at the durable commit, because only the commit fixes a chain position. The kernel router appends to the same chains directly. - **`astrid-kernel`: lossless coalescing.** - Consecutive allowed and failed calls share one run, and consecutive identical denials share one run. - A one-call run is written as the plain action, as before. - **`astrid-kernel`: loss and gap entries.** - Queue capacity bounds slots, not calls. Overflow folds into one loss slot per chain, which is written as `HostCallLoss` at its place in the chain. - A lane marker in `system:control:audit-lane` records the run and the chains it wrote. After an unclean stop, the next start writes `HostCallGap` into each of those chains and into the session's system chain. - `astrid-storage`: the state-owner resolver admits `system:control:audit-lane` as a system control projection, like `audit`, `invites` and `pair-tokens`. - An unreadable marker, or a marker store that fails, also produces a gap entry. - If the writer thread dies, the lane closes and health reports it. - **`audit.host_fail_closed`** (`astrid-config`, `astrid-capsule`, `astrid-kernel`). - It lists host-call classes whose effect runs only after a write-ahead entry is durable: `file_read`, `file_write`, `file_delete`, `net_connect`, `net_bind`, `process_spawn`. The default is empty. - `HostAuditSink::admit()` is called by the fs, net and process host functions after the security gate and before the effect. - A refusal fails the call with `unknown("audit unavailable")` and is recorded as a denial. - A workspace layer can add classes but not remove them. - **Health:** `accepted` and `persisted` now count calls, and `lost`, `gaps_recorded` and `dropped_after_shutdown` are new. The admin `AuditHealth` wire type is unchanged. - Changelog fragment. **Consequences.** - A denial between allowed calls closes the run. - Order is guaranteed among a chain's host-call entries. Admin rows are still appended when they run. - Calls reported after shutdown are counted in health, not written. **Overhead** (release build, in-memory log, per producer thread; `audit_sink::tests::bench`, ignored by default): | Workload | Before | After | |---|---|---| | 1 thread, 100k mixed allowed calls | 1.61–1.65 µs/call | 0.84–0.92 µs/call | | 8 threads × 8 principals, 200k calls | 15.1–17.9 µs/call | 8.9–12.8 µs/call | | 4 threads, reads with 2% denials | 6.7–7.8 µs/call | 4.2–4.8 µs/call | | 20k distinct denials, 1 thread | 1.35–1.58 µs/call; 15,904 calls dropped | 1.82–1.91 µs/call; none dropped, overflow in 5–7 loss entries | ## Verification Current integrated tree: full kernel library **628 passed, 1 ignored**; capsule audit tests **45 passed**. Earlier validation of the same recorder/retention integration: audit 106 passed / 2 ignored, config 140 passed, kernel audit filter 51 passed / 1 ignored; audit/config/kernel library Clippy passed with warnings denied. The published tree is byte-identical to the tested combined tree; CI must pass on this new head before merge. Prior author verification (before this integration): - **Test suites:** `astrid-audit` 74, `astrid-capsule` 748, `astrid-config` 139 and `astrid-storage` 864 all pass. The kernel lib passes 610 of 611 with `--test-threads=6` and umask 0022. The remaining failure needs a copy-on-write workspace backend and also fails on `main`. - **Unit and behaviour tests:** - host-call digest and fold known-answer values; every field changes the digest; - a dropped, reordered or altered call fails `matches_calls`; - changing a signed count breaks the signature; - call order across 6 producers, 3 principals, 8-entry batches and a 10 ms window (the same probe fails on `main`); - a 40-call run whose fold, count and time range recompute from the calls; - queue overflow written as one loss entry that commits to the lost calls; - gap entries after an abandoned lane run, none after a clean restart, and gap duties carried across a second unclean stop; - a corrupt marker and a failing marker store, including the unavailable-marker gap surviving a later marker write; - a system-only gap recorded at start with no host call, and retried while idle after a refused append; - the marker namespace written and read back through a runtime principal store; - a batch the log refuses, retried and written in order once the cap is raised; - a fail-closed write durable before `admit()` returns; - refusal within the first failed batch, and after shutdown; - config validation and workspace restriction; - `connect-tcp` and `bind-tcp` refusing without a connection attempt. - **Lint:** `cargo fmt` and `cargo clippy --all-features --all-targets -- -D warnings` on the touched crates. ## AI / Tool Assistance Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation and tests in all touched crates, and this description. Assisted-by: Codex CLI. Review passes over the diff. Current integration and verification assisted by Codex. Fresh CI pending; no claim of merge readiness until those checks complete. ## Checklist - [x] Linked to an issue - [x] Changelog fragment added under `changes/{issue}.{kind}.md` (docs/CI-only may skip; release PRs roll fragments into the version section instead of adding one) - [x] I understand every change in this PR and can explain its design, risks, and validation. - [x] I reviewed and tested any meaningful tool-generated output included in this PR. - [x] Every non-bot, non-merge commit has a matching `Signed-off-by` trailer. Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com> Co-authored-by: Joshua J. Bouw <jjb@unicity-labs.com>
Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
…rage Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
joshuajbouw
added a commit
that referenced
this pull request
Sep 29, 2026
… changes and capsule loads (#2004) ## Linked Issue Closes #1998 Part of #1997. #1996, #2002, #2003, and #2008 are merged. Current head `c0f4052305da424d9625c9f96332448667a9259b` integrates this PR with main `e63f65cc`. Remaining order: this PR, then #2005 (entry format v2). The integration retains #2003's ordered, loss-accounted writer and fail-closed admission. Capsule identity is part of the run key, so alternating capsules cannot fold together. HTTP/tool/approval coverage records remain individual queue entries; overflow still records a payload-bound loss summary. Existing unattributed call digests keep their exact v1 encoding; attributed and new coverage calls use the documented `astrid.audit.host-call.v2` digest domain. This is a call-fold encoding distinction, not #2005's entry-format change. Ordering boundary: queued records retain per-principal FIFO order. Synchronous HTTP/approval commits still use the existing durable append path, like direct administrative audit writes; no new global ordering guarantee between that path and queued records is claimed. ## Summary The audit log did not record the events that determine what an agent did: - LLM and other HTTP exchanges appeared only in `tracing::debug!`, so an agent turn, including its model request, added no entry; - capsule tool calls and approval decisions had no producer; - capability grants and revocations were recorded as `AdminRequest` rows at authorization time, before the change was applied and without its result, and grant-on-use was not recorded at all; - capsule install and load were not recorded, and install hooks ran without an audit sink; - host-call entries did not name the capsule, and `FileWrite` stored a zero hash. This PR records all of these on the signed log, without changing the v1 entry format. Content never enters the log, only commitments to it. Provider credentials can now be injected by the host, so they never enter guest memory. ## Changes - **`astrid-audit`: entry kinds.** - `HttpRequest` (pre-commit) and `HttpResponse` (completion). - `CapabilityChanged`, `CapsuleInstalled` and `CapsuleLoaded`. - A `CapsuleActor` (capsule id and wasm hash) on file, network, process, tool-call and approval entries. - Linking fields on `CapsuleToolCall` and the approval entries. New fields are optional and omitted when unset, and new variants are appended after the existing ones. Entries written before this change re-serialize to the same signed bytes. - **HTTP exchanges** (`astrid-capsule` HTTP host, `astrid-kernel`). - Each wire request, each redirect hop included, gets a durable `HttpRequest` pre-commit before it is sent. The pre-commit holds the method, host, port, BLAKE3 commitments to the path, headers and body, the redirect hop, and a per-principal sequence number scoped to the kernel run. - An `HttpResponse` completion links to its pre-commit. It carries the status, provider request ids, and a hash of the response body as it was read (incrementally for streams). - Completions are appended durably. - Refusals by the scheme check, egress, the security gate or the SSRF airlock are recorded as denied requests and take a sequence number. - Credential headers, credential query parameters and every secret value the capsule received are redacted before hashing. The redaction rules are documented so a verifier can recompute the commitments. - Recording is best-effort, like the rest of the audit log since 0.9.0. If the pre-commit cannot be written, the request is still sent, a security event is logged, and the missing entry shows as a gap in the run's sequence numbers. Approval prompts behave the same way. Making these fail closed is an operator policy; it would fit as an `http` class of `audit.host_fail_closed` from #2003 once both have landed. - **Host-side credentials.** - A header value can name a manifest-declared secret as `{{secret:NAME}}`, and the host substitutes it on the wire. - Undeclared names are refused and recorded. Placeholders are stripped on a cross-origin redirect. - Commitments cover the placeholder form, and the entry lists injected secret names, never values. - `docs/models.md` describes how a provider capsule adopts this. - **Tool calls and approvals.** - One `CapsuleToolCall` entry per `tool.v1.execute.<tool>` invocation, with the call id, argument and result hashes, and the capsule. - Approval prompts are committed before they are published. Every decision is linked to its prompt, with how it was reached: user, session allowance, remembered consent, timeout and so on. - Grant-on-use prompts are committed by the dispatcher before `GrantRequired` is published. An approval is committed before `GrantResult` is published, so the chain shows it before the grant it caused. - **Capability changes.** Applied changes are recorded on the affected principal's chain after they are saved: token mint and revoke, caps grant and revoke (only the patterns actually added), group and capsule membership changes, distro self-grants, and grant-on-use. - **Code identity.** - `CapsuleInstalled` binds the wasm and exact `Capsule.toml` hashes. - `CapsuleLoaded` (on load or replace) binds the verified wasm hash, the manifest hash and the engine profile. - Install and upgrade hooks run by the daemon get the signed sink, attributed to the hook's component. - **Attribution.** - Host-call entries name the capsule and wasm hash, stamped by the host from the verified component; a guest cannot choose them. - `write-file` reports the hash of the bytes written. - Changelog fragment. ## Verification Current signed integration head: **111 audit tests passed / 2 ignored; 777 capsule tests passed; 643 kernel tests passed / 1 ignored** (1,531 passed in total). Commands: `cargo test --locked -p astrid-audit -p astrid-capsule -p astrid-kernel --lib -- --quiet`; `cargo check --locked --workspace`; `cargo clippy --locked -p astrid-audit -p astrid-capsule -p astrid-kernel --lib --tests -- -D warnings`; formatting and diff checks. All passed locally on macOS. Added integration regressions bind capsule/wasm identity in folded digests and prove coverage events remain individual while overflow binds the original payload. The attribution test now asserts FIFO order across alternating capsules instead of expecting the old unordered folding behavior. Fresh platform CI is still required. Prior author verification before integration: - **Test suites:** `astrid-audit` 71, `astrid-capsule` 777, `astrid-capsule-install` 106 and CLI 818 all pass. The kernel lib passes 601 of 602 with `--test-threads=6` and umask 0022. The remaining failure needs a copy-on-write workspace backend and also fails on `main`. - **Unit and integration tests:** - legacy JSON encodings pinned, and a legacy entry re-verified after a round trip; - pre-commits resolve before a loopback server sees the request; - buffered and streamed responses hashed and linked, including streams closed early, transport failures and gate denials; - redaction: short secrets, many secrets, overlapping secrets; - credential injection end to end: the server receives the secret, the guest never reads it, and the commitment covers the placeholder; - placeholder rules: undeclared, malformed and unterminated placeholders refused, also after a blank secret, and an omitted header names no secret; - tool results captured through the IPC host function, with error, missing, mismatched and cancelled cases; - approval prompts committed before they are visible on the bus; - capability grant, revoke and membership entries, with none for a no-op re-grant, and a grant-on-use approval ahead of its grant; - a daemon install producing `CapsuleInstalled` and `CapsuleLoaded` with the expected hashes, and an upgrade recorded as load then replace. - **Lint:** `cargo fmt` and `cargo clippy --all-features --all-targets -- -D warnings` on the touched crates. `cargo check --target wasm32-unknown-unknown` of `astrid-audit`, `astrid-capsule` and `astrid-kernel`. ## AI / Tool Assistance Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation and tests in all touched crates, and this description. Assisted-by: Codex CLI. Review passes over the diff. Integration onto the landed recorder and associated regression tests assisted by Codex. The earlier source review does not constitute independent review of these integration edits. ## Checklist - [x] Linked to an issue - [x] Changelog fragment added under `changes/{issue}.{kind}.md` (docs/CI-only may skip; release PRs roll fragments into the version section instead of adding one) - [x] I understand every change in this PR and can explain its design, risks, and validation. - [x] I reviewed and tested any meaningful tool-generated output included in this PR. - [x] Every non-bot, non-merge commit has a matching `Signed-off-by` trailer. --------- Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com> Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com> Co-authored-by: Joshua J. Bouw <jjb@unicity-labs.com>
Member
|
The three CodeQL hard-coded-key/salt alerts are false positives; I verified and dismissed each with a source-specific explanation. The production salt buffer is completely overwritten by the fixed-length HMAC result; the random-byte buffer is completely filled by the OS CSPRNG or fails without returning; the remaining constant is a test-only known-answer fixture. No code, test, or scanning policy was weakened. #2004 is now merged, and its squash tree is identical to the integration parent used here. The final MUSL check is still pending. |
Preserve the tested entry-v2 tree unchanged after the coverage squash merge. Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
This was referenced Sep 30, 2026
joshuajbouw
pushed a commit
that referenced
this pull request
Sep 30, 2026
## Linked Issue Closes #2011 Follow-up to #1996 and #2005. ## Summary `audit.export` reported every entry's `content_hash_hex` as `BLAKE3(signing_data)`. For a format-v1 entry that is its content hash. For a format-v2 entry the signing data is the signature wrapper of the entry hash, while chain links, chain heads and prune receipts use the SHA-256 entry hash of the canonical body. On a chain switched to v2, an exported entry's hash therefore matched neither the next entry's `previous_hash_hex` nor the head in `audit.heads`, and a verifier following the export could not link the chain. The export now reports `AuditEntry::content_hash()`, which is unchanged for v1 entries. ## Changes - **`astrid-kernel`: `audit.export`.** `content_hash_hex` is the entry's content hash in either format: `BLAKE3(signing_data)` for v1, the SHA-256 entry hash for v2. - **`astrid-core`: export wire types.** Their documentation states which hash each format exports, that v2 signing data is the signature wrapper of that hash, and that only a v1 signature covers whole seconds. - Changelog fragment. ## Verification - **New kernel test** `export_reports_the_hash_that_links_a_chain_switched_to_v2`. It exports a chain with two v1 entries followed by three v2 entries and checks: - each exported hash against the stored entry; - the signature over the exported signing data; - that each v2 entry's signing data wraps its entry hash; - the links across the switch; - the head in the export page and in `audit.heads`. It fails on the previous hash computation (the v2 hashes differ) and passes with this change. - **Test suites:** `astrid-core` 391 pass. The kernel lib passes 647 of 648 with `--test-threads=6` and umask 0022. The remaining failure needs a copy-on-write workspace backend and also fails on `main`. - **Lint and changelog:** `cargo fmt`, `cargo clippy --all-features --all-targets -- -D warnings` on `astrid-core` and `astrid-kernel`, and `scripts/changelog.py check`. ## AI / Tool Assistance Assisted-by: Claude Code:claude-opus-5-5. Covers the fix, the test, the documentation and this description. Assisted-by: Codex CLI. Review pass over the diff. ## Checklist - [x] Linked to an issue - [x] Changelog fragment added under `changes/{issue}.{kind}.md` (docs/CI-only may skip; release PRs roll fragments into the version section instead of adding one) - [x] I understand every change in this PR and can explain its design, risks, and validation. - [x] I reviewed and tested any meaningful tool-generated output included in this PR. - [x] Every non-bot, non-merge commit has a matching `Signed-off-by` trailer. --------- Signed-off-by: Pavel Grigorenko <pavel@unicity-labs.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked Issue
Closes #2000
Part of #1997. The current integration includes the landed #1996, #2002, #2003 and #2004. Proposed merge order: #1996, #2002, #2003 (ordered recording), #2004 (recording coverage), this PR.
v2 derives an entry's sections from the serde form of the action, authorization and outcome. The new action variants in #2003 and #2004 therefore encode without any change to this format or its specification.
Summary
Format v1 cannot be verified outside Astrid with confidence:
serde_jsonof the action and authorization, and drop either one if serialization fails;This PR adds format v2, which is off by default and enabled with
[audit] entry_format = "v2":v1 history is kept as it is, and the v2 chain links to it.
Changes
astrid-crypto:verify_strict(RFC 8032 strict Ed25519) and arandom_byteshelper.astrid-audit:entry_v2. The module documentation is the byte-level specification, with a known-answer test that an implementation written from the specification alone reproduces.[tag, {field => commitment}]sections derived from their serde form;astrid-audit: key registry.astrid-audit: verification.ChainVerifieraccepts a v2 entry only if all of these hold:With a registry present, v1 entries and archive receipts must be signed by registered keys. v2 receipts carry the signer's epoch.
astrid-audit: writing.AuditLog::enable_entry_v2writes the genesis on first use.V1Closed).rotate_audit_keyappends a cross-signed rotation.astrid-config,astrid-kernel:[audit] entry_format. It is operator-only, and the default is"v1"."v2", the kernel loads or createskeys/audit.key(owner-only), enables v2, and on first use writes a genesis that registers the runtime key for the capability, build and v1-audit roles.keys/audit.keyis protected from admin filesystem edits.Docs and changelog.
docs/config.mddocuments the switch and its one-way migration.With v2 off, v1 behaviour, verification results and archive keys are unchanged.
Verification
Current stack integration
Head
0dd52fdd48fd25eab5c778353ecadd9701033ff8reconciles landed #2004 (3b3360fd) with integrationc225dfc7. Its tree is byte-identical toc225dfc7(treec81645720ec1d81d2c1e447aecdf328f54b4e0a0). All Actions jobs, including both MUSL smokes, passed on that tree. Three CodeQL hard-coded-value alerts were individually investigated and dismissed as false positives: an HMAC output buffer fully overwritten before return, a test-only known-answer key, and an OS-randomness buffer that cannot return on RNG failure. No scanner rule or test was weakened. Required checks on the ancestry-only commit must still complete before merge.entry_format, retention settings and fail-closed settings in the sharedAuditConfig, with both operator-only restrictions intact.--test-threads=4. The first parallel kernel run hit two unchanged 2-second response timeouts; all three tests in that group passed unchanged in an isolated rerun (0.71s), then the complete four-thread kernel run passed (51.72s).cargo check --locked --workspace, formatting and focused audit/config/crypto/kernel Clippy (--lib --tests -- -D warnings) passed.cargo check --locked -p astrid-kernel --target wasm32-unknown-unknownpassed, with unused/dead-code warnings; this is build evidence, not a browser runtime test.Original author validation (before integration)
astrid-audit123 (+2 doc),astrid-config140 andastrid-crypto31 all pass. The kernel lib passes 592 of 593 with--test-threads=6and umask 0022. The remaining failure needs a copy-on-write workspace backend and also fails onmain.cargo fmtandcargo clippy --all-features --all-targets -- -D warningson the touched crates.cargo check --target wasm32-unknown-unknownofastrid-kernel.AI / Tool Assistance
Assisted-by: Claude Code:claude-opus-5-5. Covers design, implementation and tests in all touched crates, the format specification, and this description.
Assisted-by: Codex CLI. Review passes over the diff.
Merge after #2004 and successful checks on the current integration.
Checklist
changes/{issue}.{kind}.md(docs/CI-only may skip; release PRs roll fragments into the version section instead of adding one)Signed-off-bytrailer.