diff --git a/CHANGELOG.md b/CHANGELOG.md index 76dcd12..f52005d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,14 @@ compatible, or editorial. ## Unreleased +- Hardened the unreleased Execution Provenance candidate after PR #9 review: + enforce identity policy, portable domains, status/finish relationships, + structured processing identity and timestamp formats; add 64 conformance cases. + +- Added Execution Provenance v1.0.0 (new candidate contract): metadata-only + records, canonical identity, classification-aware redaction, observer failure + isolation, RFC 0005, and synthetic conformance fixtures for issue #69. + - Added the Structured Output v1.0.0 candidate (new contract): RFC 0004, validated result/failure schema, raw/normalized identity, exact schema verification, retention and provider-constraint provenance, explicit failure diff --git a/CONSTITUTION_CONFORMANCE.md b/CONSTITUTION_CONFORMANCE.md index 2552915..17353d6 100644 --- a/CONSTITUTION_CONFORMANCE.md +++ b/CONSTITUTION_CONFORMANCE.md @@ -1,5 +1,28 @@ # Constitution conformance record +## Issue #69 assessment — 2026-09-11 + +PR #9 review follow-up: all five schema findings addressed with 64 passing +schema conformance cases. Policy suppression, numeric domains, outcome pairing, +processing identity and date-time format assertion are enforced. Platform's +pinned consumer validator is refreshed in its coordinated existing PR #12. + +This assessment extends and supersedes the prior issue #68 scope below for the +current candidate. Constitution v1.0.0 at +`a9cc8a503aa30e17820edc62ac95f7cbe10e0564` remains the authority. +Accountable owner: @andrewperis; status: Conforming candidate pending owner merge. +Base revision: `7fe034e508fdda2c70eb111c0517e1fe115e818b`. + +Scope: Execution Provenance v1 candidate schema, RFC 0005, specification, redaction policy and synthetic fixtures. +Evidence: Coordinated Platform suite: 74 passing tests, including public/non-public context projection, identity tampering, structured outcomes and observer failure isolation. +No new constitutional exceptions. Public fixtures are synthetic. Body content, +authorization and diagnostic strings are omitted; non-public content identities +are suppressed. Sinks cannot change the provider result or trigger retries. +Caller-owned opaque identifiers, production sink retention/access controls and +immutable contract publication (#72) remain outside this candidate. +The schema defines a new independent candidate contract; existing contracts are unchanged. +Next review: before release or any security-boundary/dependency change. + ## Constitutional alignment - Constitution: [Definitely Secure Studio Constitution v1.0.0](https://github.com/DefinitelySecureStudio/studio/tree/constitution/v1.0.0) diff --git a/README.md b/README.md index d0bec47..40d7441 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,9 @@ Stable specifications and contracts for Definitely Secure Studio systems. +The [Execution Provenance v1 candidate](specs/execution-provenance/README.md) +defines metadata-only execution evidence and observer policy. + > [!NOTE] > This repository defines contracts. It does not implement production software > and it does not own creative canon or private lore. diff --git a/fixtures/README.md b/fixtures/README.md index 5a56c91..37dc608 100644 --- a/fixtures/README.md +++ b/fixtures/README.md @@ -1,5 +1,22 @@ # Conformance fixtures +Execution Provenance v1: `valid/execution-provenance-v1.json` is the synthetic +Platform observer example. `invalid/execution-provenance-v1.json` adds a +forbidden body field; closed-schema validation must reject it (its digest also +no longer matches). Digest tampering is separately tested by Platform. + +`execution-provenance-v1-cases.json` contains schema-only overlays on the valid +fixture: `set` uses dotted record paths and `remove` names record fields. These +are not standalone envelopes; digest recomputation is deliberately excluded to +isolate schema rejection. The runner checks policy, parameter boundaries, every +status/finish pair, processing ids and RFC 3339 formats (including invalid dates). +With Platform's locked Ajv 8.20.0 and ajv-formats 3.0.1 installed in a sibling +checkout, run from Codex: + +```sh +NODE_PATH=../platform/node_modules node --test tests/execution-provenance.test.cjs +``` + Synthetic examples that demonstrate contract conformance belong here. - [`valid/`](valid/) contains inputs that must pass validation. diff --git a/fixtures/execution-provenance-v1-cases.json b/fixtures/execution-provenance-v1-cases.json new file mode 100644 index 0000000..3fa9c29 --- /dev/null +++ b/fixtures/execution-provenance-v1-cases.json @@ -0,0 +1,800 @@ +[ + { + "name": "public baseline", + "valid": true, + "set": {}, + "remove": [] + }, + { + "name": "omit with no identities", + "valid": true, + "set": { + "policy.content_identities": "omit" + }, + "remove": [ + "rendered", + "output" + ] + }, + { + "name": "omit rejects rendered", + "valid": false, + "set": { + "policy.content_identities": "omit" + }, + "remove": [ + "output" + ] + }, + { + "name": "omit rejects output", + "valid": false, + "set": { + "policy.content_identities": "omit" + }, + "remove": [ + "rendered" + ] + }, + { + "name": "public context", + "valid": true, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "public", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "public" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "omit rejects context identities", + "valid": false, + "set": { + "policy.content_identities": "omit", + "contexts": [ + { + "slot": "approved_notes", + "classification": "public", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "public" + } + ] + } + ] + }, + "remove": [ + "rendered", + "output" + ] + }, + { + "name": "reject context internal", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "internal", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "public" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject source internal", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "public", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "internal" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject context confidential", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "confidential", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "public" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject source confidential", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "public", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "confidential" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject context restricted", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "restricted", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "public" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject source restricted", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "public", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "restricted" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject context unknown", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "unknown", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "public" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject source unknown", + "valid": false, + "set": { + "contexts": [ + { + "slot": "approved_notes", + "classification": "public", + "package": { + "id": "context.synthetic", + "version": "1.0.0", + "instance_id": "ctx_1", + "manifest_sha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + }, + "sources": [ + { + "source_id": "source.synthetic", + "version": "1.0.0", + "kind": "synthetic", + "classification": "unknown" + } + ] + } + ] + }, + "remove": [] + }, + { + "name": "reject max_output_tokens -1.5", + "valid": false, + "set": { + "parameters.max_output_tokens": -1.5 + }, + "remove": [] + }, + { + "name": "reject max_output_tokens 0", + "valid": false, + "set": { + "parameters.max_output_tokens": 0 + }, + "remove": [] + }, + { + "name": "reject max_output_tokens 1.5", + "valid": false, + "set": { + "parameters.max_output_tokens": 1.5 + }, + "remove": [] + }, + { + "name": "reject temperature -0.1", + "valid": false, + "set": { + "parameters.temperature": -0.1 + }, + "remove": [] + }, + { + "name": "reject temperature 2.1", + "valid": false, + "set": { + "parameters.temperature": 2.1 + }, + "remove": [] + }, + { + "name": "reject top_p 0", + "valid": false, + "set": { + "parameters.top_p": 0 + }, + "remove": [] + }, + { + "name": "reject top_p -1", + "valid": false, + "set": { + "parameters.top_p": -1 + }, + "remove": [] + }, + { + "name": "reject top_p 1.1", + "valid": false, + "set": { + "parameters.top_p": 1.1 + }, + "remove": [] + }, + { + "name": "reject seed -1", + "valid": false, + "set": { + "parameters.seed": -1 + }, + "remove": [] + }, + { + "name": "reject seed 0.5", + "valid": false, + "set": { + "parameters.seed": 0.5 + }, + "remove": [] + }, + { + "name": "reject seed 9007199254740992", + "valid": false, + "set": { + "parameters.seed": 9007199254740992 + }, + "remove": [] + }, + { + "name": "parameter lower bounds", + "valid": true, + "set": { + "parameters": { + "max_output_tokens": 1, + "temperature": 0, + "top_p": 0.01, + "seed": 0 + } + }, + "remove": [] + }, + { + "name": "parameter upper bounds", + "valid": true, + "set": { + "parameters": { + "max_output_tokens": 256, + "temperature": 2, + "top_p": 1, + "seed": 9007199254740991 + } + }, + "remove": [] + }, + { + "name": "succeeded / stop", + "valid": true, + "set": { + "status": "succeeded", + "finish_reason": "stop" + }, + "remove": [] + }, + { + "name": "succeeded / length", + "valid": true, + "set": { + "status": "succeeded", + "finish_reason": "length" + }, + "remove": [] + }, + { + "name": "succeeded / content-filter", + "valid": true, + "set": { + "status": "succeeded", + "finish_reason": "content-filter" + }, + "remove": [] + }, + { + "name": "succeeded / unknown", + "valid": true, + "set": { + "status": "succeeded", + "finish_reason": "unknown" + }, + "remove": [] + }, + { + "name": "succeeded / error", + "valid": false, + "set": { + "status": "succeeded", + "finish_reason": "error" + }, + "remove": [] + }, + { + "name": "succeeded / cancelled", + "valid": false, + "set": { + "status": "succeeded", + "finish_reason": "cancelled" + }, + "remove": [] + }, + { + "name": "failed / stop", + "valid": false, + "set": { + "status": "failed", + "finish_reason": "stop" + }, + "remove": [ + "output" + ] + }, + { + "name": "failed / length", + "valid": false, + "set": { + "status": "failed", + "finish_reason": "length" + }, + "remove": [ + "output" + ] + }, + { + "name": "failed / content-filter", + "valid": false, + "set": { + "status": "failed", + "finish_reason": "content-filter" + }, + "remove": [ + "output" + ] + }, + { + "name": "failed / unknown", + "valid": false, + "set": { + "status": "failed", + "finish_reason": "unknown" + }, + "remove": [ + "output" + ] + }, + { + "name": "failed / error", + "valid": true, + "set": { + "status": "failed", + "finish_reason": "error" + }, + "remove": [ + "output" + ] + }, + { + "name": "failed / cancelled", + "valid": false, + "set": { + "status": "failed", + "finish_reason": "cancelled" + }, + "remove": [ + "output" + ] + }, + { + "name": "cancelled / stop", + "valid": false, + "set": { + "status": "cancelled", + "finish_reason": "stop" + }, + "remove": [ + "output" + ] + }, + { + "name": "cancelled / length", + "valid": false, + "set": { + "status": "cancelled", + "finish_reason": "length" + }, + "remove": [ + "output" + ] + }, + { + "name": "cancelled / content-filter", + "valid": false, + "set": { + "status": "cancelled", + "finish_reason": "content-filter" + }, + "remove": [ + "output" + ] + }, + { + "name": "cancelled / unknown", + "valid": false, + "set": { + "status": "cancelled", + "finish_reason": "unknown" + }, + "remove": [ + "output" + ] + }, + { + "name": "cancelled / error", + "valid": false, + "set": { + "status": "cancelled", + "finish_reason": "error" + }, + "remove": [ + "output" + ] + }, + { + "name": "cancelled / cancelled", + "valid": true, + "set": { + "status": "cancelled", + "finish_reason": "cancelled" + }, + "remove": [ + "output" + ] + }, + { + "name": "timed-out / stop", + "valid": false, + "set": { + "status": "timed-out", + "finish_reason": "stop" + }, + "remove": [ + "output" + ] + }, + { + "name": "timed-out / length", + "valid": false, + "set": { + "status": "timed-out", + "finish_reason": "length" + }, + "remove": [ + "output" + ] + }, + { + "name": "timed-out / content-filter", + "valid": false, + "set": { + "status": "timed-out", + "finish_reason": "content-filter" + }, + "remove": [ + "output" + ] + }, + { + "name": "timed-out / unknown", + "valid": false, + "set": { + "status": "timed-out", + "finish_reason": "unknown" + }, + "remove": [ + "output" + ] + }, + { + "name": "timed-out / error", + "valid": true, + "set": { + "status": "timed-out", + "finish_reason": "error" + }, + "remove": [ + "output" + ] + }, + { + "name": "timed-out / cancelled", + "valid": false, + "set": { + "status": "timed-out", + "finish_reason": "cancelled" + }, + "remove": [ + "output" + ] + }, + { + "name": "validated processing id", + "valid": true, + "set": { + "structured_output": { + "status": "validated", + "processing_id": "processing_1" + } + }, + "remove": [] + }, + { + "name": "validated missing processing id", + "valid": false, + "set": { + "structured_output": { + "status": "validated" + } + }, + "remove": [] + }, + { + "name": "failed processing id", + "valid": true, + "set": { + "structured_output": { + "status": "failed", + "processing_id": "processing_1" + } + }, + "remove": [] + }, + { + "name": "failed missing processing id", + "valid": false, + "set": { + "structured_output": { + "status": "failed" + } + }, + "remove": [] + }, + { + "name": "not-run rejects processing id", + "valid": false, + "set": { + "structured_output": { + "status": "not-run", + "processing_id": "processing_1" + } + }, + "remove": [] + }, + { + "name": "reject empty processing id", + "valid": false, + "set": { + "structured_output": { + "status": "validated", + "processing_id": "" + } + }, + "remove": [] + }, + { + "name": "reject started_at later", + "valid": false, + "set": { + "timing.started_at": "later" + }, + "remove": [] + }, + { + "name": "reject started_at 2026-02-30T00:00:00Z", + "valid": false, + "set": { + "timing.started_at": "2026-02-30T00:00:00Z" + }, + "remove": [] + }, + { + "name": "reject started_at 2026-09-11", + "valid": false, + "set": { + "timing.started_at": "2026-09-11" + }, + "remove": [] + }, + { + "name": "reject completed_at later", + "valid": false, + "set": { + "timing.completed_at": "later" + }, + "remove": [] + }, + { + "name": "reject completed_at 2026-02-30T00:00:00Z", + "valid": false, + "set": { + "timing.completed_at": "2026-02-30T00:00:00Z" + }, + "remove": [] + }, + { + "name": "reject completed_at 2026-09-11", + "valid": false, + "set": { + "timing.completed_at": "2026-09-11" + }, + "remove": [] + }, + { + "name": "RFC3339 offset", + "valid": true, + "set": { + "timing.started_at": "2026-09-11T08:00:00-04:00", + "timing.completed_at": "2026-09-11T08:00:00-04:00" + }, + "remove": [] + } +] diff --git a/fixtures/invalid/execution-provenance-v1.json b/fixtures/invalid/execution-provenance-v1.json new file mode 100644 index 0000000..b89ad2e --- /dev/null +++ b/fixtures/invalid/execution-provenance-v1.json @@ -0,0 +1,72 @@ +{ + "spec_version": "1.0.0", + "kind": "execution-provenance", + "record": { + "execution_id": "exec_provenance_example", + "correlation_id": "build_example", + "prompt": { + "id": "prompt.example.describe-public-item", + "version": "1.0.0", + "spec_version": "1.0.0" + }, + "contexts": [], + "redacted_context_count": 0, + "target": { + "adapter_id": "studio.mock.text", + "provider_id": "studio-mock", + "model_id": "mock-text-v1" + }, + "identity": { + "adapter_id": "studio.mock.text", + "adapter_version": "1.0.0", + "provider_id": "studio-mock", + "model_id": "mock-text-v1", + "model_revision": "synthetic-1" + }, + "parameters": { + "max_output_tokens": 256, + "temperature": 1, + "top_p": 1 + }, + "parameters_status": "resolved", + "stop_sequence_count": 0, + "timing": { + "started_at": "1970-01-01T00:00:00.000Z", + "completed_at": "1970-01-01T00:00:00.000Z", + "duration_ms": 0 + }, + "status": "succeeded", + "finish_reason": "stop", + "validation": { + "status": "not-run", + "errors": 0, + "warnings": 0 + }, + "structured_output": { + "status": "not-run" + }, + "policy": { + "content_identities": "public-only", + "bodies": "omitted" + }, + "usage": { + "provider_reported": true, + "input_tokens": 18, + "output_tokens": 4, + "total_tokens": 22 + }, + "rendered": { + "byte_size": 872, + "sha256": "sha256:a646b6b7f701371c838fbe061fa7438a0440a4b09516fc3d1c167bf041fb3705" + }, + "output": { + "byte_size": 24, + "sha256": "sha256:ad56d367cf578d1295ce4a8b14d7ebb4357a1311f4fcd5e3d672a5d8f10f5d4a" + }, + "content": "Forbidden body" + }, + "identity": { + "byte_size": 1211, + "sha256": "sha256:b951d0e1ebf50509bd9cd0cbb10f6dd184cf28db8c13f68e4648d647dc0c572f" + } +} diff --git a/fixtures/valid/execution-provenance-v1.json b/fixtures/valid/execution-provenance-v1.json new file mode 100644 index 0000000..8309408 --- /dev/null +++ b/fixtures/valid/execution-provenance-v1.json @@ -0,0 +1,71 @@ +{ + "spec_version": "1.0.0", + "kind": "execution-provenance", + "record": { + "execution_id": "exec_provenance_example", + "correlation_id": "build_example", + "prompt": { + "id": "prompt.example.describe-public-item", + "version": "1.0.0", + "spec_version": "1.0.0" + }, + "contexts": [], + "redacted_context_count": 0, + "target": { + "adapter_id": "studio.mock.text", + "provider_id": "studio-mock", + "model_id": "mock-text-v1" + }, + "identity": { + "adapter_id": "studio.mock.text", + "adapter_version": "1.0.0", + "provider_id": "studio-mock", + "model_id": "mock-text-v1", + "model_revision": "synthetic-1" + }, + "parameters": { + "max_output_tokens": 256, + "temperature": 1, + "top_p": 1 + }, + "parameters_status": "resolved", + "stop_sequence_count": 0, + "timing": { + "started_at": "1970-01-01T00:00:00.000Z", + "completed_at": "1970-01-01T00:00:00.000Z", + "duration_ms": 0 + }, + "status": "succeeded", + "finish_reason": "stop", + "validation": { + "status": "not-run", + "errors": 0, + "warnings": 0 + }, + "structured_output": { + "status": "not-run" + }, + "policy": { + "content_identities": "public-only", + "bodies": "omitted" + }, + "usage": { + "provider_reported": true, + "input_tokens": 18, + "output_tokens": 4, + "total_tokens": 22 + }, + "rendered": { + "byte_size": 872, + "sha256": "sha256:a646b6b7f701371c838fbe061fa7438a0440a4b09516fc3d1c167bf041fb3705" + }, + "output": { + "byte_size": 24, + "sha256": "sha256:ad56d367cf578d1295ce4a8b14d7ebb4357a1311f4fcd5e3d672a5d8f10f5d4a" + } + }, + "identity": { + "byte_size": 1211, + "sha256": "sha256:b951d0e1ebf50509bd9cd0cbb10f6dd184cf28db8c13f68e4648d647dc0c572f" + } +} diff --git a/rfcs/0005-execution-provenance.md b/rfcs/0005-execution-provenance.md new file mode 100644 index 0000000..05f8a37 --- /dev/null +++ b/rfcs/0005-execution-provenance.md @@ -0,0 +1,19 @@ +# RFC 0005: Execution provenance and observers + +Status: Candidate, accepted by owner merge. Owner: @andrewperis. +Issue: https://github.com/DefinitelySecureStudio/studio/issues/69 + +Adopt Execution Provenance v1 as a closed metadata projection with canonical +identity. Separate observer delivery from provider execution and preserve +explicit validation states. Use public-only content identities by default, +with an omit policy for more restrictive callers. Numeric effective parameters +remain traceable while free-form strings and bodies are excluded. + +The consumer is a vendor-neutral observer interface and bounded local/test +implementation. Downstream build/manifest tooling can link execution and +correlation ids and exact record identities. No telemetry vendor or remote +delivery is mandated. Automatic retries on observer failure are excluded. + +Synthetic consumer tests verify completeness, redaction, outcome coverage, +canonical identity, isolation, and bounded delivery. This contract does not +approve publication, provider output, or a production sink. diff --git a/schemas/json/execution-provenance/v1/execution-provenance.schema.json b/schemas/json/execution-provenance/v1/execution-provenance.schema.json new file mode 100644 index 0000000..e006b1b --- /dev/null +++ b/schemas/json/execution-provenance/v1/execution-provenance.schema.json @@ -0,0 +1,555 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "urn:definitely-secure:contract:execution-provenance:1.0.0:record", + "type": "object", + "additionalProperties": false, + "required": [ + "spec_version", + "kind", + "record", + "identity" + ], + "properties": { + "spec_version": { + "const": "1.0.0" + }, + "kind": { + "const": "execution-provenance" + }, + "record": { + "type": "object", + "additionalProperties": false, + "required": [ + "execution_id", + "prompt", + "contexts", + "redacted_context_count", + "target", + "identity", + "parameters", + "parameters_status", + "stop_sequence_count", + "timing", + "status", + "finish_reason", + "validation", + "structured_output", + "policy" + ], + "properties": { + "execution_id": { + "type": "string" + }, + "correlation_id": { + "type": "string" + }, + "prompt": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "version", + "spec_version" + ], + "properties": { + "id": { + "type": "string" + }, + "version": { + "type": "string" + }, + "spec_version": { + "type": "string" + } + } + }, + "rendered": { + "type": "object", + "additionalProperties": false, + "required": [ + "byte_size", + "sha256" + ], + "properties": { + "byte_size": { + "type": "integer", + "minimum": 0 + }, + "sha256": { + "type": "string", + "pattern": "^sha256:[0-9a-f]{64}$" + } + } + }, + "contexts": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "slot", + "classification" + ], + "properties": { + "slot": { + "type": "string" + }, + "classification": { + "const": "public" + }, + "package": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "version", + "instance_id", + "manifest_sha256" + ], + "properties": { + "id": { + "type": "string" + }, + "version": { + "type": "string" + }, + "instance_id": { + "type": "string" + }, + "manifest_sha256": { + "type": "string" + } + } + }, + "sources": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "source_id", + "version", + "kind", + "classification" + ], + "properties": { + "source_id": { + "type": "string" + }, + "version": { + "type": "string" + }, + "kind": { + "type": "string" + }, + "classification": { + "const": "public" + } + } + } + } + } + } + }, + "redacted_context_count": { + "type": "integer", + "minimum": 0 + }, + "target": { + "type": "object", + "additionalProperties": false, + "required": [ + "adapter_id", + "provider_id", + "model_id" + ], + "properties": { + "adapter_id": { + "type": "string" + }, + "provider_id": { + "type": "string" + }, + "model_id": { + "type": "string" + } + } + }, + "identity": { + "type": "object", + "additionalProperties": false, + "required": [ + "adapter_id", + "adapter_version", + "provider_id", + "model_id" + ], + "properties": { + "adapter_id": { + "type": "string" + }, + "adapter_version": { + "type": "string" + }, + "provider_id": { + "type": "string" + }, + "model_id": { + "type": "string" + }, + "model_revision": { + "type": "string" + } + } + }, + "parameters": { + "type": "object", + "additionalProperties": false, + "required": [], + "properties": { + "max_output_tokens": { + "type": "integer", + "minimum": 1 + }, + "temperature": { + "type": "number", + "minimum": 0, + "maximum": 2 + }, + "top_p": { + "type": "number", + "exclusiveMinimum": 0, + "maximum": 1 + }, + "seed": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + } + } + }, + "parameters_status": { + "enum": [ + "resolved", + "not-resolved" + ] + }, + "stop_sequence_count": { + "type": "integer", + "minimum": 0 + }, + "timing": { + "type": "object", + "additionalProperties": false, + "required": [ + "started_at", + "completed_at", + "duration_ms" + ], + "properties": { + "started_at": { + "type": "string", + "format": "date-time" + }, + "completed_at": { + "type": "string", + "format": "date-time" + }, + "duration_ms": { + "type": "integer", + "minimum": 0 + } + } + }, + "status": { + "enum": [ + "succeeded", + "failed", + "cancelled", + "timed-out" + ] + }, + "finish_reason": { + "enum": [ + "stop", + "length", + "content-filter", + "cancelled", + "error", + "unknown" + ] + }, + "usage": { + "type": "object", + "additionalProperties": false, + "required": [ + "provider_reported" + ], + "properties": { + "provider_reported": { + "type": "boolean" + }, + "input_tokens": { + "type": "integer", + "minimum": 0 + }, + "output_tokens": { + "type": "integer", + "minimum": 0 + }, + "total_tokens": { + "type": "integer", + "minimum": 0 + } + } + }, + "output": { + "type": "object", + "additionalProperties": false, + "required": [ + "byte_size", + "sha256" + ], + "properties": { + "byte_size": { + "type": "integer", + "minimum": 0 + }, + "sha256": { + "type": "string", + "pattern": "^sha256:[0-9a-f]{64}$" + } + } + }, + "validation": { + "type": "object", + "additionalProperties": false, + "required": [ + "status", + "errors", + "warnings" + ], + "properties": { + "status": { + "enum": [ + "not-run", + "passed", + "failed" + ] + }, + "errors": { + "type": "integer", + "minimum": 0 + }, + "warnings": { + "type": "integer", + "minimum": 0 + } + } + }, + "structured_output": { + "type": "object", + "additionalProperties": false, + "required": [ + "status" + ], + "properties": { + "status": { + "enum": [ + "not-run", + "validated", + "failed" + ] + }, + "processing_id": { + "type": "string", + "pattern": "^[A-Za-z0-9][A-Za-z0-9._:/-]{0,127}$" + } + }, + "allOf": [ + { + "if": { + "properties": { + "status": { + "const": "not-run" + } + }, + "required": [ + "status" + ] + }, + "then": { + "not": { + "required": [ + "processing_id" + ] + } + }, + "else": { + "required": [ + "processing_id" + ] + } + } + ] + }, + "policy": { + "type": "object", + "additionalProperties": false, + "required": [ + "content_identities", + "bodies" + ], + "properties": { + "content_identities": { + "enum": [ + "public-only", + "omit" + ] + }, + "bodies": { + "const": "omitted" + } + } + } + }, + "allOf": [ + { + "if": { + "properties": { + "policy": { + "type": "object", + "properties": { + "content_identities": { + "const": "omit" + } + }, + "required": [ + "content_identities" + ] + } + }, + "required": [ + "policy" + ] + }, + "then": { + "properties": { + "contexts": { + "type": "array", + "maxItems": 0 + }, + "rendered": false, + "output": false + } + } + }, + { + "if": { + "properties": { + "status": { + "const": "succeeded" + } + }, + "required": [ + "status" + ] + }, + "then": { + "properties": { + "finish_reason": { + "enum": [ + "stop", + "length", + "content-filter", + "unknown" + ] + } + } + } + }, + { + "if": { + "properties": { + "status": { + "const": "failed" + } + }, + "required": [ + "status" + ] + }, + "then": { + "properties": { + "finish_reason": { + "enum": [ + "error" + ] + } + } + } + }, + { + "if": { + "properties": { + "status": { + "const": "cancelled" + } + }, + "required": [ + "status" + ] + }, + "then": { + "properties": { + "finish_reason": { + "enum": [ + "cancelled" + ] + } + } + } + }, + { + "if": { + "properties": { + "status": { + "const": "timed-out" + } + }, + "required": [ + "status" + ] + }, + "then": { + "properties": { + "finish_reason": { + "enum": [ + "error" + ] + } + } + } + } + ] + }, + "identity": { + "type": "object", + "additionalProperties": false, + "required": [ + "byte_size", + "sha256" + ], + "properties": { + "byte_size": { + "type": "integer", + "minimum": 0 + }, + "sha256": { + "type": "string", + "pattern": "^sha256:[0-9a-f]{64}$" + } + } + } + } +} diff --git a/specs/execution-provenance/README.md b/specs/execution-provenance/README.md new file mode 100644 index 0000000..099a37e --- /dev/null +++ b/specs/execution-provenance/README.md @@ -0,0 +1,61 @@ +# Execution Provenance v1 + +Candidate for Studio #69; owner: @andrewperis. RFC 0005 defines this new +contract. Constitution v1.0.0, commit a9cc8a503aa30e17820edc62ac95f7cbe10e0564, +governs review. Owner merge accepts the candidate; issue #72 remains the +immutable-release gate. + +The closed JSON schema is ../../schemas/json/execution-provenance/v1/execution-provenance.schema.json. +An envelope contains `record` and its Studio canonical JSON v1 UTF-8 byte size +and SHA-256 `identity`. Equal explicit inputs produce equal records. A later +structured-output outcome produces a new record identity for the same execution. +Consumers must preserve both revisions when audit history is required. + +Records MUST identify the prompt/specification, execution/correlation, requested +target, observed adapter/model, final status, finish reason, timing and reported +usage. Parameters are the numeric portable values after adapter defaults, when +negotiation completed; `not-resolved` MUST distinguish preflight failure. +Stop sequence bodies are omitted; only their count is recorded. + +Copied portable parameters retain Provider Execution v1 domains: positive integer +`max_output_tokens`, temperature 0–2, top_p greater than 0 and at most 1, and +integer seed 0–9007199254740991. Timestamps MUST be RFC 3339 date-time values; +consumers MUST enable format assertion when validating this schema. +Success permits stop, length, content-filter or unknown finish reasons. Failed +and timed-out outcomes require error; cancelled requires cancelled. + +Validation outcomes contain status and counts only. Missing evidence is +`not-run`, never a pass. Structured-output evidence must match the execution +and expectation before its processing id/status can be copied. + +Completed structured-output states require a nonempty processing id; not-run +forbids one. The schema enforces these state-dependent fields. + +`public-only` permits rendered/output byte identities and context slot/package/ +source identities only when the associated classification is public. Context +identities also require the overall rendered prompt to be public. `omit` +suppresses all such identities. Non-public contexts contribute only to the +redacted count. Identity fields must contain approved opaque identifiers; +classification and syntactic validation cannot detect a secret disguised as an id. + +The schema forbids rendered/output identities and all context entries under +`omit`. Every retained context and source classification MUST be public. +Original input classifications are not carried in this projection: the producer +must additionally check source evidence before emitting hashes or identities. + +Bodies, raw output, parsed values, references/URIs, authorization evidence, +provider request ids, idempotency keys, stop strings, extension configuration, +error/warning text, diagnostic paths and arbitrary metadata MUST NOT be emitted. +Even body capture permission in an execution request does not change this rule. +Records are operational evidence, not automatically public release manifests. + +Observers accept one immutable metadata record asynchronously. Sink errors and +timeouts MUST remain visible as a static delivery warning without changing the +provider outcome or retrying execution. Default local/test storage is bounded +memory, with snapshot isolation and explicit clearing. Invalid requests that +never begin execution retain the existing validation exception behavior. + +This is a new independent contract. Unknown fields/versions fail. Breaking +identity, redaction, or outcome semantics require a new major version; optional +additions require a negotiated minor version. Production implementations and +sink retention/access control belong in Platform and its callers. diff --git a/tests/execution-provenance.test.cjs b/tests/execution-provenance.test.cjs new file mode 100644 index 0000000..d657b10 --- /dev/null +++ b/tests/execution-provenance.test.cjs @@ -0,0 +1,25 @@ +const { readFileSync } = require('node:fs'); +const { resolve } = require('node:path'); +const assert = require('node:assert/strict'); +const test = require('node:test'); +const Ajv = require('ajv/dist/2020'); +const addFormats = require('ajv-formats'); +const read = path => JSON.parse(readFileSync(resolve(__dirname, '..', path))); +const ajv = new Ajv({ strict: true, strictRequired: false, allErrors: true }); +addFormats(ajv); +const validate = ajv.compile(read('schemas/json/execution-provenance/v1/execution-provenance.schema.json')); +const base = read('fixtures/valid/execution-provenance-v1.json'); +// These overlays exercise schema constraints alone, not canonical digest checks. +for (const fixture of read('fixtures/execution-provenance-v1-cases.json')) { + test(fixture.name, () => { + const document = structuredClone(base); + for (const [path, value] of Object.entries(fixture.set)) { + const keys = path.split('.'); + const field = keys.pop(); + const parent = keys.reduce((obj, key) => obj[key], document.record); + parent[field] = structuredClone(value); + } + for (const field of fixture.remove) delete document.record[field]; + assert.equal(validate(document), fixture.valid, JSON.stringify(validate.errors)); + }); +}