diff --git a/.codex/skills/release-oliphaunt/SKILL.md b/.codex/skills/release-oliphaunt/SKILL.md index 20ab923b..49233cf6 100644 --- a/.codex/skills/release-oliphaunt/SKILL.md +++ b/.codex/skills/release-oliphaunt/SKILL.md @@ -82,13 +82,13 @@ another repository mutation. - Recover: inventory external state first. For ordinary single-identity recovery, resume idempotently from the publication ledger. For an authorized same-version control recovery, keep the original commit/tree, pinned complete - payload CI inventory, approved lock/capsule, and terminal ledger as the - publication source; require fresh full CI and approved control equivalence on - the later controller; require lock replay to be byte-identical including - `source` and `lockDigest`; keep tags/releases/assets source-bound; disable - bootstrap and continuations; and rerun root `publish` to reconcile missing - state. Never delete, overwrite, or republish a matching immutable public - version. + payload CI inventory, approved lock/capsule, recorded immutable boundary, and + any required terminal ledger as the publication source; require fresh + recovery-control CI and approved control equivalence on the later controller; + require lock replay to be byte-identical including `source` and `lockDigest`; + keep tags/releases/assets source-bound; disable bootstrap and continuations; + and rerun root `publish` to reconcile missing state. Never delete, overwrite, + or republish a matching immutable public version. - History repair: use only before any affected product tag/package is public. Follow `references/recovery.md` and require explicit maintainer authorization for protection changes or force-push. ## Local gates diff --git a/.codex/skills/release-oliphaunt/references/invariants.md b/.codex/skills/release-oliphaunt/references/invariants.md index 2647ef80..1e9b4fe3 100644 --- a/.codex/skills/release-oliphaunt/references/invariants.md +++ b/.codex/skills/release-oliphaunt/references/invariants.md @@ -15,10 +15,17 @@ qualified workflow controller. Never rewrite the lock source or relabel product evidence as controller output. - Same-version recovery selects the complete original payload CI inventory, - approved lock/capsule, and terminal bootstrap ledger by exact committed - run/artifact ID, digest, and size. Its replayed publication lock must be - byte-identical to the approved original, including `source` and `lockDigest`. - The current first-release recovery requires all 73 recorded CI artifacts. + approved lock/capsule, immutable recovery boundary, and any required terminal + bootstrap ledger by exact committed run/job/artifact ID, digest, and size. + The boundary is either a nonempty exact public-registry prefix or the complete + exact-source GitHub staged set backed by the failed run's recovery artifact. + Its replayed publication lock must be byte-identical to the approved original, + including `source` and `lockDigest`. The current first-release recovery + requires all 73 recorded CI artifacts. +- A recovery controller receives a fresh, exact-main recovery-control CI record + for its zero-owner control delta; it does not rebuild unchanged platform or + package payloads. The separately reverified, committed complete source CI + inventory remains the sole payload authority. - Product tags/releases/assets, Swift source publication, registry receipts, and consumer-facing provenance remain publication-source-bound. Workflow code, the transport tag, OIDC claims, request journals, and pacing are @@ -47,7 +54,8 @@ one byte of drift fails closed and requires a new version. - Same-version recovery cannot run bootstrap or any continuation. Resume only through an idempotent root `publish` rerun that verifies the original - terminal ledger and reconciles every exact immutable identity before writes. + terminal ledger when one was required, proves the recorded immutable recovery + boundary, and reconciles every exact immutable identity before writes. - With a clean release state, a pure zero-owner control-plane, workflow, validator, registry-transport, test, or documentation change creates no release PR and performs no publication. Semantic ownership, not a `ci:` diff --git a/.codex/skills/release-oliphaunt/references/recovery.md b/.codex/skills/release-oliphaunt/references/recovery.md index 68ee6038..a71607d4 100644 --- a/.codex/skills/release-oliphaunt/references/recovery.md +++ b/.codex/skills/release-oliphaunt/references/recovery.md @@ -15,9 +15,11 @@ ### Same-version control recovery after partial publication -Use this path only when at least one immutable carrier is already public, no -product tag/release has been promoted, and the required repository fix has no -release-semantic product owner. +Use this path only after the original source crossed a recorded immutable +publication boundary: either at least one exact registry carrier is public, or +the complete selected GitHub tag/release/asset set was staged at the original +source. No product tag/release may have been moved or replaced, and the +required repository fix must have no release-semantic product owner. 1. Keep the original release-bump commit and public history immutable. Every recovery commit must be a linear descendant with subject @@ -34,31 +36,41 @@ release-semantic product owner. change. 3. Name the identities explicitly. The trailer target is the immutable **publication source**: it owns the original commit/tree, product bytes, - versions, approved publication lock/capsule, terminal bootstrap ledger, + versions, approved publication lock/capsule, any required terminal bootstrap ledger, product tags/releases/assets, Swift source tag, registry receipts, and consumer-facing provenance. The later current-main recovery head is only the **controller**: it owns workflow code, its fresh CI/run identity, the release transport tag, OIDC claims, request journals, and pacing. -4. Run fresh complete CI on the controller. Then run `publish-dry-run` on that +4. Run fresh recovery-control CI on the controller. It must execute the + controller delta's checks/tests/policy plus exact-main `Required` and + `Qualified`, but must not rebuild payload or platform matrices. The complete + successful original payload CI inventory remains independently pinned and + reverified as the sole payload authority. Then run `publish-dry-run` on the controller to produce and approve recovery-control equivalence evidence. This dry-run must not upload a replacement publication lock or bootstrap capsule. 5. Resolve the committed immutable recovery record. Select the original source - SHA/tree, complete payload CI run, approved dry-run lock/capsule, and terminal - bootstrap ledger only by the exact recorded workflow run and artifact - ID/digest/size. For the current first-release recovery, compare the complete - observed CI inventory with all 73 recorded artifacts. Do not select “latest,” - fall back to artifact name alone, or accept a merely same-SHA run. + SHA/tree, complete payload CI run, approved dry-run lock/capsule, immutable + recovery boundary, and any required terminal bootstrap ledger only by the + exact recorded workflow run/job and artifact ID/digest/size. For the current + first-release recovery, compare the complete observed CI inventory with all + 73 recorded artifacts and prove the failed staging run's recovery artifact + against the approved lock. Do not select “latest,” fall back to artifact + name alone, or accept a merely same-SHA run. 6. Reassemble only from those pinned original payload artifacts. Replay publication-lock construction at the original source and require the resulting file to be byte-identical to the approved original lock, including the `source` object and `lockDigest`. Preserve the controller/source equivalence receipt. A lock rebound to the controller is a provenance mismatch even when every package-envelope byte is equal. -7. Verify the pinned terminal source-bound bootstrap ledger against the - original lock. Recovery bootstrap is disabled: do not create a new - controller-bound ledger, request bootstrap credentials, or invoke - `publish-bootstrap`. +7. If the pinned record contains a terminal source-bound bootstrap ledger, + verify it against the original lock. If the record explicitly contains no + ledger, require the live Cargo/npm bootstrap-state classification to prove + that no bootstrap ledger is needed, and independently require the exact + source product tags for a GitHub-staged boundary; never invent or select an + unrelated ledger. Recovery + bootstrap is disabled: do not create a new controller-bound ledger, request + bootstrap credentials, or invoke `publish-bootstrap`. 8. Derive the exhaustive Cargo/npm/Maven/JSR inventory from the original frozen lock so generated payload-part carriers cannot disappear behind the static catalog. A matching public identity is a read-only recovery skip and must diff --git a/.github/scripts/check-release-intent.sh b/.github/scripts/check-release-intent.sh index 0879f613..72a7571a 100755 --- a/.github/scripts/check-release-intent.sh +++ b/.github/scripts/check-release-intent.sh @@ -418,11 +418,46 @@ fi # A same-version partial-publication recovery is not a product release, but it # must fail before expensive planning unless its exact original release, # linear trailer chain, zero-product impact, and unchanged metadata all verify. +qualification_mode="full-payload" +recovery_release_sha="" +recovery_controller_sha="" if git show -s --format=%B "${head_ref}^{commit}" | grep -qi "^Oliphaunt-Release-Recovery-Of:"; then - tools/dev/bun.sh tools/release/verify-publication-candidate.mjs \ + recovery_candidate_output="$(mktemp "${TMPDIR:-/tmp}/oliphaunt-recovery-candidate.XXXXXX")" + if ! tools/dev/bun.sh tools/release/verify-publication-candidate.mjs \ --derive-products \ - --head-ref "${head_ref}" + --head-ref "${head_ref}" \ + --github-output "${recovery_candidate_output}"; then + rm -f "${recovery_candidate_output}" + exit 1 + fi + verified_recovery_mode="$(sed -n 's/^mode=//p' "${recovery_candidate_output}")" + recovery_release_sha="$(sed -n 's/^release_sha=//p' "${recovery_candidate_output}")" + recovery_controller_sha="$(sed -n 's/^publication_sha=//p' "${recovery_candidate_output}")" + recovery_mode_count="$(grep -c '^mode=' "${recovery_candidate_output}" || true)" + recovery_release_count="$(grep -c '^release_sha=' "${recovery_candidate_output}" || true)" + recovery_controller_count="$(grep -c '^publication_sha=' "${recovery_candidate_output}" || true)" + rm -f "${recovery_candidate_output}" + if [[ "${recovery_mode_count}" != "1" ]] || + [[ "${recovery_release_count}" != "1" ]] || + [[ "${recovery_controller_count}" != "1" ]] || + [[ "${verified_recovery_mode}" != "release-recovery" ]] || + [[ ! "${recovery_release_sha}" =~ ^[0-9a-f]{40}$ ]] || + [[ ! "${recovery_controller_sha}" =~ ^[0-9a-f]{40}$ ]] || + [[ "${recovery_controller_sha}" != "$(git rev-parse "${head_ref}^{commit}")" ]] || + [[ "${recovery_release_sha}" == "${recovery_controller_sha}" ]]; then + echo "verified recovery lineage did not emit one exact release/controller binding" >&2 + exit 1 + fi + qualification_mode="recovery-control" +fi + +if [[ -n "${GITHUB_OUTPUT:-}" ]]; then + { + echo "qualification_mode=${qualification_mode}" + echo "recovery_release_sha=${recovery_release_sha}" + echo "recovery_controller_sha=${recovery_controller_sha}" + } >> "${GITHUB_OUTPUT}" fi release_plan="$(tools/dev/bun.sh tools/release/release_plan.mjs --base-ref "${base_ref}" --head-ref "${head_ref}" --format json)" diff --git a/.github/scripts/manage-release-drafts.mjs b/.github/scripts/manage-release-drafts.mjs index b184fc94..8e466cbf 100644 --- a/.github/scripts/manage-release-drafts.mjs +++ b/.github/scripts/manage-release-drafts.mjs @@ -1,5 +1,5 @@ #!/usr/bin/env bun -import { existsSync, readFileSync } from "node:fs"; +import { appendFileSync, existsSync, readFileSync } from "node:fs"; import path from "node:path"; import process from "node:process"; @@ -63,7 +63,7 @@ function error(message, options = {}) { function usageError() { return error( - "usage: manage-release-drafts.mjs " + "usage: manage-release-drafts.mjs " + "--products-json JSON --head-ref SHA [--state draft|public|staged]", ); } @@ -89,7 +89,11 @@ function selectedPublicationLock(command, products, headRef, environment) { ?? DEFAULT_PUBLICATION_LOCK, ); if (!existsSync(file)) { - if (command === "preflight" || command === "recovery-preflight") return null; + if ( + command === "preflight" + || command === "recovery-preflight" + || command === "recovery-staged-preflight" + ) return null; throw error(`${command} requires the frozen publication lock: ${file}`); } const lock = loadPublicationLock(file); @@ -731,7 +735,7 @@ export function reconcileSelectedReleasesSync( }); let releasesByTag; - if (command === "verify") { + if (command === "verify" || command === "recovery-staged-preflight") { releasesByTag = requiredReleaseMap(expectedState); } else if (command === "promote") { // No mutation has happened yet. A missing/stale precondition can fail and @@ -750,10 +754,27 @@ export function reconcileSelectedReleasesSync( return; } if (command === "recovery-preflight") { + const exactTagCount = selected.filter(({ tag }) => tagsByName.get(tag) !== null).length; + const exactReleaseCount = selected.filter(({ tag }) => releasesByTag.has(tag)).length; console.log( `${selected.length} selected product tag/release names are absent or exact-SHA resumable for same-version recovery`, ); - return; + return { + exactReleaseCount, + exactTagCount, + selectedCount: selected.length, + }; + } + if (command === "recovery-staged-preflight") { + requireExactTagSnapshot(selected, tagsByName, headRef); + console.log( + `${selected.length} selected product tags and releases are exact-SHA staged for same-version recovery`, + ); + return { + exactReleaseCount: selected.filter(({ tag }) => releasesByTag.has(tag)).length, + exactTagCount: selected.filter(({ tag }) => tagsByName.get(tag) !== null).length, + selectedCount: selected.length, + }; } if (command === "stage") { @@ -880,8 +901,17 @@ export function createReleaseDraftOperationBudget( export function main(argv, { environment = process.env, now = Date.now } = {}) { const { command, values } = parseArgs([...argv]); - if (!["preflight", "recovery-preflight", "stage", "verify", "promote"].includes(command)) { - throw error("command must be preflight, recovery-preflight, stage, verify, or promote"); + if (![ + "preflight", + "recovery-preflight", + "recovery-staged-preflight", + "stage", + "verify", + "promote", + ].includes(command)) { + throw error( + "command must be preflight, recovery-preflight, recovery-staged-preflight, stage, verify, or promote", + ); } const repo = environment.GITHUB_REPOSITORY?.trim(); if (!repo || !environment.GH_TOKEN) { @@ -907,14 +937,15 @@ export function main(argv, { environment = process.env, now = Date.now } = {}) { if (!headRef || !FULL_SHA.test(headRef)) { throw error("--head-ref must be a full lowercase commit SHA"); } - const expectedState = values.get("state") ?? "draft"; + const expectedState = values.get("state") + ?? (command === "recovery-staged-preflight" ? "staged" : "draft"); if (!new Set(["draft", "public", "staged"]).has(expectedState)) { throw error("--state must be draft, public, or staged"); } const selected = selectedReleases(command, products, headRef, environment); const budget = createReleaseDraftOperationBudget(command, { environment, now }); - reconcileSelectedReleasesSync({ + const result = reconcileSelectedReleasesSync({ budget, command, environment: budget.environment, @@ -923,6 +954,21 @@ export function main(argv, { environment = process.env, now = Date.now } = {}) { repo, selected, }); + if ( + (command === "recovery-preflight" || command === "recovery-staged-preflight") + && environment.GITHUB_OUTPUT + ) { + appendFileSync( + environment.GITHUB_OUTPUT, + [ + `exact_release_count=${result.exactReleaseCount}`, + `exact_tag_count=${result.exactTagCount}`, + `selected_count=${result.selectedCount}`, + "", + ].join("\n"), + "utf8", + ); + } } if (import.meta.main) { diff --git a/.github/scripts/release-candidate-lib.mjs b/.github/scripts/release-candidate-lib.mjs index 1a6c6f2c..78a97e9d 100644 --- a/.github/scripts/release-candidate-lib.mjs +++ b/.github/scripts/release-candidate-lib.mjs @@ -65,6 +65,45 @@ function uniqueStrings(value, context) { return value; } +export const FULL_PAYLOAD_QUALIFICATION_MODE = "full-payload"; +export const RECOVERY_CONTROL_QUALIFICATION_MODE = "recovery-control"; +const QUALIFICATION_MODES = new Set([ + FULL_PAYLOAD_QUALIFICATION_MODE, + RECOVERY_CONTROL_QUALIFICATION_MODE, +]); +const FULL_SHA = /^[0-9a-f]{40}$/u; + +function qualificationBinding(plan, jobs) { + const fields = [ + "qualification_mode", + "qualification_base_sha", + "qualification_head_sha", + ]; + const present = fields.map((field) => Object.hasOwn(plan, field)); + if (!present.some(Boolean)) return undefined; + assert(present.every(Boolean), "affected CI plan qualification binding is incomplete"); + const mode = plan.qualification_mode; + const baseSha = plan.qualification_base_sha; + const headSha = plan.qualification_head_sha; + assert(QUALIFICATION_MODES.has(mode), `affected CI plan qualification mode is invalid: ${mode}`); + if (mode === FULL_PAYLOAD_QUALIFICATION_MODE) { + assert(baseSha === null && headSha === null, "full-payload CI plan must not carry a recovery affected range"); + } else { + assert(FULL_SHA.test(baseSha), "recovery-control CI plan base SHA must be lowercase and full"); + assert(FULL_SHA.test(headSha), "recovery-control CI plan head SHA must be lowercase and full"); + assert(baseSha !== headSha, "recovery-control CI plan source and controller must be distinct"); + assert( + JSON.stringify(jobs) === JSON.stringify(["affected"]), + "recovery-control CI plan must not select builder or E2E payload jobs", + ); + } + return { mode, baseSha, headSha }; +} + +export function candidateQualificationMode(candidate) { + return candidate?.affectedPlan?.qualification?.mode ?? FULL_PAYLOAD_QUALIFICATION_MODE; +} + export function affectedPlanBinding(planPath, wasixReleaseRegressionRequired) { assert(typeof wasixReleaseRegressionRequired === "boolean", "WASIX release regression requirement must be boolean"); const { value: plan } = strictJson(planPath, "affected CI plan"); @@ -79,6 +118,7 @@ export function affectedPlanBinding(planPath, wasixReleaseRegressionRequired) { wasixReleaseRegressionRequired === expectedRequirement, `affected CI plan WASIX requirement mismatch: jobs imply ${expectedRequirement}, workflow reported ${wasixReleaseRegressionRequired}`, ); + const qualification = qualificationBinding(plan, jobs); const canonical = JSON.stringify(canonicalValue(plan)); return { digest: sha256(canonical), @@ -86,6 +126,7 @@ export function affectedPlanBinding(planPath, wasixReleaseRegressionRequired) { projects, extensionPackageProducts, wasixReleaseRegressionRequired, + ...(qualification === undefined ? {} : { qualification }), }; } @@ -226,6 +267,38 @@ export function assertCandidateBindingShape(candidate) { candidate.affectedPlan.wasixReleaseRegressionRequired === jobs.includes("liboliphaunt-wasix-runtime"), "release candidate affectedPlan WASIX requirement is inconsistent with selected jobs", ); + const qualification = candidate.affectedPlan.qualification; + if (qualification !== undefined) { + assert( + qualification !== null && !Array.isArray(qualification) && typeof qualification === "object", + "release candidate affectedPlan qualification is invalid", + ); + assert( + JSON.stringify(Object.keys(qualification).sort()) + === JSON.stringify(["baseSha", "headSha", "mode"]), + "release candidate affectedPlan qualification fields are invalid", + ); + assert(QUALIFICATION_MODES.has(qualification.mode), "release candidate qualification mode is invalid"); + if (qualification.mode === FULL_PAYLOAD_QUALIFICATION_MODE) { + assert( + qualification.baseSha === null && qualification.headSha === null, + "full-payload release candidate must not carry a recovery affected range", + ); + } else { + assert(FULL_SHA.test(qualification.baseSha), "recovery-control candidate base SHA must be lowercase and full"); + assert(FULL_SHA.test(qualification.headSha), "recovery-control candidate head SHA must be lowercase and full"); + assert(qualification.baseSha !== qualification.headSha, "recovery-control candidate source and controller must be distinct"); + assert(candidate.sha === qualification.headSha, "recovery-control candidate controller does not match candidate SHA"); + assert( + JSON.stringify(jobs) === JSON.stringify(["affected"]), + "recovery-control candidate must not select builder or E2E payload jobs", + ); + assert( + candidate.affectedPlan.extensionPackageProducts.length === 0, + "recovery-control candidate must not select extension payload products", + ); + } + } const requirements = candidate.evidenceRequirements; assert(requirements !== null && typeof requirements === "object", "release candidate evidenceRequirements is missing"); assert( diff --git a/.github/scripts/verify-release-candidate.mjs b/.github/scripts/verify-release-candidate.mjs index d3e337c8..bafcb720 100644 --- a/.github/scripts/verify-release-candidate.mjs +++ b/.github/scripts/verify-release-candidate.mjs @@ -8,6 +8,9 @@ import { affectedPlanBinding, assertBindingMatches, assertCandidateBindingShape, + candidateQualificationMode, + FULL_PAYLOAD_QUALIFICATION_MODE, + RECOVERY_CONTROL_QUALIFICATION_MODE, wasixEvidenceBinding, } from "./release-candidate-lib.mjs"; @@ -39,7 +42,12 @@ function parseArgs(argv) { const values = new Map(); for (let index = 1; index < argv.length; index += 1) { const name = argv[index]; - if (!["--plan", "--wasix-evidence-required", "--wasix-evidence-root"].includes(name)) { + if (![ + "--plan", + "--qualification-mode", + "--wasix-evidence-required", + "--wasix-evidence-root", + ].includes(name)) { fail(`unknown argument: ${name}`); } if (index + 1 >= argv.length) { @@ -51,7 +59,8 @@ function parseArgs(argv) { if (!candidatePath || !values.has("plan") || !values.has("wasix-evidence-required")) { fail( "usage: verify-release-candidate.mjs --plan " - + "--wasix-evidence-required true|false [--wasix-evidence-root ]", + + "--wasix-evidence-required true|false [--qualification-mode full-payload|recovery-control] " + + "[--wasix-evidence-root ]", ); } const required = values.get("wasix-evidence-required"); @@ -61,11 +70,16 @@ function parseArgs(argv) { if (required === "true" && !values.has("wasix-evidence-root")) { fail("--wasix-evidence-root is required when WASIX evidence is required"); } + const qualificationMode = values.get("qualification-mode") ?? FULL_PAYLOAD_QUALIFICATION_MODE; + if (![FULL_PAYLOAD_QUALIFICATION_MODE, RECOVERY_CONTROL_QUALIFICATION_MODE].includes(qualificationMode)) { + fail("--qualification-mode must be full-payload or recovery-control"); + } return { candidatePath, planPath: values.get("plan"), wasixEvidenceRequired: required === "true", wasixEvidenceRoot: values.get("wasix-evidence-root"), + qualificationMode, }; } @@ -83,6 +97,12 @@ try { } catch (error) { fail(error.message); } +if (candidateQualificationMode(candidate) !== args.qualificationMode) { + fail( + `release candidate qualification mode mismatch: expected ${args.qualificationMode}, ` + + `got ${candidateQualificationMode(candidate)}`, + ); +} const expected = { repository: requiredEnv("GITHUB_REPOSITORY"), diff --git a/.github/scripts/write-release-candidate.mjs b/.github/scripts/write-release-candidate.mjs index 1a029b7f..e9e53bab 100644 --- a/.github/scripts/write-release-candidate.mjs +++ b/.github/scripts/write-release-candidate.mjs @@ -8,6 +8,9 @@ import { captureCommandOutput } from "../../tools/dev/capture-command-output.mjs import { affectedPlanBinding, assertCandidateBindingShape, + candidateQualificationMode, + FULL_PAYLOAD_QUALIFICATION_MODE, + RECOVERY_CONTROL_QUALIFICATION_MODE, wasixEvidenceBinding, } from "./release-candidate-lib.mjs"; @@ -57,6 +60,15 @@ try { } catch (error) { fail(error.message); } +const expectedQualificationMode = requiredEnv("CI_QUALIFICATION_MODE"); +if (![FULL_PAYLOAD_QUALIFICATION_MODE, RECOVERY_CONTROL_QUALIFICATION_MODE].includes(expectedQualificationMode)) { + fail(`CI_QUALIFICATION_MODE is invalid: ${expectedQualificationMode}`); +} +if (candidateQualificationMode({ affectedPlan }) !== expectedQualificationMode) { + fail( + `affected CI plan qualification mode does not match workflow mode ${expectedQualificationMode}`, + ); +} const runAttempt = Number.parseInt(requiredEnv("GITHUB_RUN_ATTEMPT"), 10); if (!Number.isSafeInteger(runAttempt) || runAttempt < 1) { diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 62cd2855..15432c75 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -126,6 +126,9 @@ jobs: test_jobs: ${{ steps.target-matrices.outputs.test_jobs }} test_matrix: ${{ steps.target-matrices.outputs.test_matrix }} reason: ${{ steps.plan.outputs.reason }} + qualification_mode: ${{ steps.plan.outputs.qualification_mode }} + qualification_base_sha: ${{ steps.plan.outputs.qualification_base_sha }} + qualification_head_sha: ${{ steps.plan.outputs.qualification_head_sha }} wasix_release_regression_required: ${{ contains(fromJson(steps.plan.outputs.jobs), 'liboliphaunt-wasix-runtime') && (github.event_name != 'workflow_dispatch' || inputs.wasm_target == 'all' || inputs.wasm_target == 'linux-x64-gnu') }} steps: - name: Checkout repository @@ -143,8 +146,9 @@ jobs: - name: Plan artifact builder jobs id: plan env: + CI_QUALIFICATION_MODE: ${{ needs.release-intent.outputs.qualification_mode }} GITHUB_EVENT_NAME: ${{ github.event_name }} - MOON_BASE: ${{ github.event.pull_request.base.sha }} + MOON_BASE: ${{ needs.release-intent.outputs.qualification_mode == 'recovery-control' && needs.release-intent.outputs.recovery_release_sha || github.event.pull_request.base.sha }} MOON_HEAD: ${{ github.event.pull_request.head.sha || github.sha }} WASM_TARGET: ${{ github.event_name == 'workflow_dispatch' && inputs.wasm_target || 'all' }} NATIVE_TARGET: ${{ github.event_name == 'workflow_dispatch' && inputs.native_target || 'all' }} @@ -153,6 +157,9 @@ jobs: - name: Plan check and test jobs id: target-matrices + env: + MOON_BASE: ${{ needs.release-intent.outputs.qualification_mode == 'recovery-control' && needs.release-intent.outputs.recovery_release_sha || github.event.pull_request.base.sha }} + MOON_HEAD: ${{ github.event.pull_request.head.sha || github.sha }} run: node .github/scripts/write-affected-moon-target-matrices.mjs check test - name: Upload build plan @@ -170,6 +177,10 @@ jobs: contents: read runs-on: ubuntu-24.04 timeout-minutes: 10 + outputs: + qualification_mode: ${{ steps.release_intent.outputs.qualification_mode }} + recovery_release_sha: ${{ steps.release_intent.outputs.recovery_release_sha }} + recovery_controller_sha: ${{ steps.release_intent.outputs.recovery_controller_sha }} steps: - name: Checkout repository uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd @@ -3229,13 +3240,16 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 5 steps: - - name: Require full CI success + - name: Require qualified CI success id: require_full_ci env: CI_EVENT_NAME: ${{ github.event_name }} CI_MOBILE_TARGET: ${{ inputs.mobile_target }} CI_NATIVE_TARGET: ${{ inputs.native_target }} + CI_QUALIFICATION_MODE: ${{ needs.affected.outputs.qualification_mode }} CI_WASM_TARGET: ${{ inputs.wasm_target }} + SELECTED_BUILDER_JOBS: ${{ needs.affected.outputs.builder_jobs }} + SELECTED_E2E_JOBS: ${{ needs.affected.outputs.e2e_jobs }} run: | if [[ '${{ needs.affected.result }}' != success || '${{ needs.required.result }}' != success ]]; then echo 'release qualification requires successful Plan and Required jobs' >&2 @@ -3246,6 +3260,11 @@ jobs: echo 'focused workflow_dispatch runs are diagnostic only and cannot qualify a release candidate' >&2 exit 1 fi + if [[ "$CI_QUALIFICATION_MODE" == recovery-control ]] && \ + [[ "$SELECTED_BUILDER_JOBS" != '[]' || "$SELECTED_E2E_JOBS" != '[]' ]]; then + echo 'recovery-control qualification must not select builder or E2E payload jobs' >&2 + exit 1 + fi - name: Checkout repository uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd @@ -3276,6 +3295,7 @@ jobs: id: qualification_record env: CI_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + CI_QUALIFICATION_MODE: ${{ needs.affected.outputs.qualification_mode }} CI_PLAN_PATH: target/qualification/affected-plan/ci-plan.json WASIX_RELEASE_REGRESSION_REQUIRED: ${{ needs.affected.outputs.wasix_release_regression_required }} WASIX_EVIDENCE_ROOT: target/qualification/wasix-release-regression-evidence diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 94777f50..9e778a43 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -455,15 +455,23 @@ jobs: - name: Prove same-version recovery GitHub state is absent or exact-SHA resumable id: verify_release_recovery_github_state - if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && inputs.operation == 'publish-dry-run' && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }} + if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && (inputs.operation == 'publish-dry-run' || inputs.operation == 'publish') && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }} timeout-minutes: 5 env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} PRODUCTS_JSON: ${{ steps.release_plan.outputs.products_json }} run: | - bun .github/scripts/manage-release-drafts.mjs recovery-preflight \ - --products-json "$PRODUCTS_JSON" \ - --head-ref "$RELEASE_SOURCE_SHA" + if [[ "${{ steps.recovery_source.outputs.recovery_boundary_kind }}" == github-staged ]]; then + bun .github/scripts/manage-release-drafts.mjs recovery-staged-preflight \ + --products-json "$PRODUCTS_JSON" \ + --head-ref "$RELEASE_SOURCE_SHA" \ + --state staged + else + bun .github/scripts/manage-release-drafts.mjs recovery-preflight \ + --products-json "$PRODUCTS_JSON" \ + --head-ref "$RELEASE_SOURCE_SHA" \ + --state staged + fi - name: Prove Release Please PR can complete after publication id: assert_release_please_markable @@ -529,6 +537,7 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} GH_REPO: ${{ github.repository }} + CANDIDATE_MODE: ${{ steps.verify_publication_candidate.outputs.mode }} RELEASE_OPERATION: ${{ inputs.operation }} REQUIRES_WASIX_EVIDENCE: ${{ steps.release_plan.outputs.requires_wasix_release_regression_evidence }} run: | @@ -548,11 +557,13 @@ jobs: --artifact artifact-build-plan --artifact oliphaunt-release-candidate ) - if [[ "$REQUIRES_WASIX_EVIDENCE" == true ]]; then - qualification_args+=(--artifact wasix-release-regression-evidence) - fi - if [[ "${{ steps.release_plan.outputs.has_extension_products }}" == true ]]; then - qualification_args+=(--artifact oliphaunt-extension-package-artifacts) + if [[ "$CANDIDATE_MODE" != release-recovery ]]; then + if [[ "$REQUIRES_WASIX_EVIDENCE" == true ]]; then + qualification_args+=(--artifact wasix-release-regression-evidence) + fi + if [[ "${{ steps.release_plan.outputs.has_extension_products }}" == true ]]; then + qualification_args+=(--artifact oliphaunt-extension-package-artifacts) + fi fi bash .github/scripts/require-workflow-success.sh "${qualification_args[@]}" @@ -671,7 +682,7 @@ jobs: --artifact artifact-build-plan - name: Download required exact-SHA WASIX evidence - if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && steps.release_plan.outputs.requires_wasix_release_regression_evidence == 'true' }} + if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && steps.verify_publication_candidate.outputs.mode != 'release-recovery' && steps.release_plan.outputs.requires_wasix_release_regression_evidence == 'true' }} env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} GH_REPO: ${{ github.repository }} @@ -689,13 +700,21 @@ jobs: id: verify_qualification if: ${{ steps.release_plan.outputs.has_release_changes == 'true' }} env: + CANDIDATE_MODE: ${{ steps.verify_publication_candidate.outputs.mode }} CI_RUN_ID: ${{ steps.ci_qualification.outputs.run_id }} WASIX_EVIDENCE_REQUIRED: ${{ steps.release_plan.outputs.requires_wasix_release_regression_evidence }} run: | + qualification_mode=full-payload + wasix_evidence_required="$WASIX_EVIDENCE_REQUIRED" + if [[ "$CANDIDATE_MODE" == release-recovery ]]; then + qualification_mode=recovery-control + wasix_evidence_required=false + fi node .github/scripts/verify-release-candidate.mjs \ target/release-candidate/oliphaunt-release-candidate.json \ --plan target/release-candidate/affected-plan/ci-plan.json \ - --wasix-evidence-required "$WASIX_EVIDENCE_REQUIRED" \ + --qualification-mode "$qualification_mode" \ + --wasix-evidence-required "$wasix_evidence_required" \ --wasix-evidence-root target/release-candidate/wasix-evidence - name: Download frozen-payload qualification record @@ -756,6 +775,7 @@ jobs: node .github/scripts/verify-release-candidate.mjs \ target/recovery-payload-candidate/oliphaunt-release-candidate.json \ --plan target/recovery-payload-candidate/affected-plan/ci-plan.json \ + --qualification-mode full-payload \ --wasix-evidence-required "$WASIX_EVIDENCE_REQUIRED" \ --wasix-evidence-root target/recovery-payload-candidate/wasix-evidence @@ -821,6 +841,20 @@ jobs: --artifact-metadata-json "$RECOVERY_LOCK_ARTIFACT_METADATA_JSON" \ --artifact oliphaunt-publication-lock + - name: Verify pinned GitHub-staged recovery boundary + id: verify_github_staged_recovery_boundary + if: ${{ steps.download_recovery_original_publication_lock.outcome == 'success' && steps.recovery_source.outputs.recovery_boundary_kind == 'github-staged' }} + timeout-minutes: 5 + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + RECOVERY_PROVENANCE_RECORD_JSON: ${{ steps.recovery_source.outputs.record_json }} + run: | + printf '%s\n' "$RECOVERY_PROVENANCE_RECORD_JSON" > "$RUNNER_TEMP/recovery-boundary.json" + tools/dev/bun.sh tools/release/verify-github-staged-recovery-boundary.mjs \ + --boundary "$RUNNER_TEMP/recovery-boundary.json" \ + --approved-lock "$RUNNER_TEMP/recovery-original-publication-lock/publication-lock.json" \ + --repo "$GITHUB_REPOSITORY" + - name: Require one approved dry-run lock and capsule id: approved_publication_lock if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && inputs.operation == 'publish' }} @@ -1210,7 +1244,8 @@ jobs: id: validate_product_dry_runs if: ${{ steps.release_plan.outputs.has_release_changes == 'true' }} env: - CI_RUN_ID: ${{ steps.ci_qualification.outputs.run_id }} + CANDIDATE_MODE: ${{ steps.verify_publication_candidate.outputs.mode }} + CI_RUN_ID: ${{ steps.release_artifact_source.outputs.ci_run_id }} OLIPHAUNT_BROKER_RELEASE_ASSET_INPUT_DIRS: ${{ github.workspace }}/target/oliphaunt-broker/release-assets OLIPHAUNT_NODE_ADDON_ASSET_INPUT_DIRS: ${{ github.workspace }}/target/oliphaunt-node-direct/release-assets OLIPHAUNT_VERIFIED_NODE_EXECUTABLE: ${{ steps.setup_github_stage_npm.outputs.node-executable }} @@ -1386,7 +1421,9 @@ jobs: tools/dev/bun.sh tools/release/verify-release-recovery-publication.mjs \ --lock "$PUBLICATION_LOCK_PATH" \ --inventory target/release/recovery-registry-inventory.json \ + --immutable-github-tag-count "${{ steps.verify_release_recovery_github_state.outputs.exact_tag_count }}" \ --products-json "$PRODUCTS_JSON" \ + --recovery-boundary-kind "${{ steps.recovery_source.outputs.recovery_boundary_kind }}" \ --output target/release/recovery-evidence/publication-state.json \ --github-output "$GITHUB_OUTPUT" @@ -1492,6 +1529,17 @@ jobs: RELEASE_HEAD_SHA: ${{ steps.release_identity.outputs.source_sha }} run: bun .github/scripts/registry-bootstrap-ledger-state.mjs + - name: Require pinned bootstrap evidence when recovery still needs it + id: require_recovery_bootstrap_ledger + if: ${{ steps.bootstrap_ledger_state.outputs.needs_ledger == 'true' && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }} + env: + PINNED_BOOTSTRAP_LEDGER_REQUIRED: ${{ steps.recovery_source.outputs.bootstrap_ledger_required }} + run: | + if [[ "$PINNED_BOOTSTRAP_LEDGER_REQUIRED" != true ]]; then + echo 'Recovery requires bootstrap evidence, but its immutable source record explicitly has no bootstrap ledger.' >&2 + exit 1 + fi + - name: Download immutable registry bootstrap ledger if: ${{ steps.bootstrap_ledger_state.outputs.needs_ledger == 'true' }} env: diff --git a/docs/maintainers/release-setup.md b/docs/maintainers/release-setup.md index 84a869e1..fc47ad4e 100644 --- a/docs/maintainers/release-setup.md +++ b/docs/maintainers/release-setup.md @@ -504,10 +504,12 @@ Publishing is resumable but not cross-registry atomic. On failure, preserve and A zero-owner release-control/test fix after partial immutable publication may instead use the documented same-version recovery. It keeps the original release commit/tree, exact pinned payload CI inventory, approved lock/capsule, -and terminal ledger as the publication source. A later current-main controller -receives fresh full CI and an approved control-equivalence dry-run, but supplies -only workflow/transport/OIDC/pacing code. Lock replay must be byte-identical to -the original, including source and lock digest; tags/releases/assets remain +recorded immutable boundary, and any required terminal ledger as the +publication source. A later current-main controller receives fresh +recovery-control CI and an approved control-equivalence dry-run, but supplies +only workflow/transport/OIDC/pacing code. The original complete payload CI is +still reverified independently. Lock replay must be byte-identical to the +original, including source and lock digest; tags/releases/assets remain source-bound. Recovery bootstrap and continuations are disabled, and an interruption is resumed through an idempotent root `publish` rerun. No matching public immutable identity is uploaded again. Use diff --git a/docs/maintainers/release.md b/docs/maintainers/release.md index 89d1959d..3ba3acef 100644 --- a/docs/maintainers/release.md +++ b/docs/maintainers/release.md @@ -195,11 +195,14 @@ Root publication admission accepts only a current-main candidate with one non-ca Normal publication has one identity: that qualified commit is both the workflow controller and the immutable publication source. The narrowly scoped same-version control recovery described below has two identities instead. Its -current-main controller must receive fresh complete CI, while the publication -source remains the original release-bump commit and tree. Controller -qualification proves the repaired workflow, policy, transport, OIDC, and -pacing implementation; it does not replace or relabel the qualified product -payload. +current-main controller must receive fresh recovery-control CI, while the +publication source remains the original release-bump commit and tree. That +controller CI qualifies the affected checks/tests/policy and exact-main +Required/Qualified gates without rebuilding unchanged payload matrices. The +separately pinned complete original CI inventory remains the payload authority. +Controller qualification proves the repaired workflow, policy, transport, +OIDC, and pacing implementation; it does not replace or relabel the qualified +product payload. The `macos-26` publication runner is ARM64, but its current runner-image contract exposes the installed Java 17 path as `JAVA_HOME_17_arm64` (including @@ -223,12 +226,15 @@ Missing and extra identities both fail. Publish commands reverify the lock immed A same-version control recovery does not perform that assembly from newly built controller artifacts. It resolves the committed immutable recovery record and selects the original payload CI run, approved dry-run lock and -capsule, and terminal bootstrap ledger by their exact run and artifact -IDs/digests/sizes. For the current first-release recovery, the record enumerates -all 73 original CI artifacts; any missing, extra, expired, or metadata-mismatched -artifact fails closed. The controller replays publication-lock construction -from those frozen inputs, and the result must be byte-identical to the approved -original lock, including its original `source` object and `lockDigest`. +capsule, immutable recovery boundary, and any required terminal bootstrap +ledger by their exact run/job and artifact IDs/digests/sizes. The boundary is +either a nonempty exact public-registry prefix or the complete exact-source +GitHub staged set plus its failed-run recovery artifact. For the current +first-release recovery, the record enumerates all 73 original CI artifacts; +any missing, extra, expired, or metadata-mismatched artifact fails closed. The +controller replays publication-lock construction from those frozen inputs, and +the result must be byte-identical to the approved original lock, including its +original `source` object and `lockDigest`. ## Operations @@ -267,10 +273,13 @@ The `Release` workflow has four operations: Same-version control recovery disables `publish-bootstrap` and every bootstrap or normal-publish continuation. It consumes the exact terminal original -bootstrap ledger and must be driven by a root `publish` dispatch on the current -controller. If a recovery run is interrupted, rerun that same root operation; -the frozen source lock and byte-verifying registry inventory make the rerun -idempotent. +bootstrap ledger only when the original source actually crossed the bootstrap +boundary. A source that reached complete immutable GitHub staging without +needing bootstrap instead binds the exact failed Release run, staging job, and +recovery artifact. Recovery must be driven by a root `publish` dispatch on the +current controller. If a recovery run is interrupted, rerun that same root +operation; the frozen source lock and byte-verifying registry inventory make +the rerun idempotent. On the normal path, only a successful `publish-dry-run` uploads the canonical `oliphaunt-publication-lock` and `oliphaunt-bootstrap-capsule` approval @@ -287,7 +296,12 @@ A same-version recovery dry-run is an approval of the current controller, not a new product-payload approval. It selects the original approved lock and capsule by exact recorded metadata, verifies the byte-identical replay, and uploads only the recovery-control equivalence evidence. It never emits a -replacement lock or capsule with the controller SHA. +replacement lock or capsule with the controller SHA. CI likewise qualifies +only the controller delta and release-policy surface; the committed recovery +record and its complete exact artifact inventory keep the successful original +payload CI as the sole payload authority. This avoids rebuilding an unchanged +multi-platform payload while still requiring `Required` and `Qualified` on the +exact controller commit. `.github/workflows/release.yml` is the one directly dispatched release workflow. Its operation jobs declare their own least-privilege permissions and @@ -805,9 +819,11 @@ possible values but never creates extension support by default. On a failed publish, preserve the candidate SHA, run id, lock, complete checkpoint chain, draft releases, and registry responses. Inventory every selected identity as absent, matching, or conflicting; restore and validate the exact-SHA chain, then resume only missing phases. Product-semantic repository changes require a new version and candidate. -If immutable packages are already public but the failure requires only a -zero-owner release-control/test fix, use the explicit same-version control -recovery instead of manufacturing a duplicate release. Keep the original +If the original release crossed an immutable publication boundary and the +failure requires only a zero-owner release-control/test fix, use the explicit +same-version control recovery instead of manufacturing a duplicate release. +The boundary may be a matching public registry carrier or the complete staged +GitHub tag/release/asset set at the original source. Keep the original release-bump commit immutable. The later linear `fix(release):` commit carries exactly one `Oliphaunt-Release-Recovery-Of: ` trailer. The publication-candidate verifier requires the authoritative base/head release @@ -817,28 +833,34 @@ owner, and proves versions and release metadata are unchanged. Treat the two identities as distinct and immutable: - the **publication source** is the original release-bump commit/tree. It owns - the product bytes, versions, approved publication lock/capsule, terminal - bootstrap ledger, product tags/releases/assets, Swift source tag, registry - receipts, and consumer-facing provenance; + the product bytes, versions, approved publication lock/capsule, immutable + recovery boundary, any required terminal bootstrap ledger, product + tags/releases/assets, Swift source tag, registry receipts, and + consumer-facing provenance; - the **controller** is the later current-main recovery commit. It owns only - the executing workflow, fresh complete control CI, immutable release + the executing workflow, fresh recovery-control CI, immutable release transport, OIDC claims, request journals, pacing, and the recovery run identity. -The controller must pass a fresh full `Qualified` CI run and a separate -successful recovery dry-run whose control-equivalence evidence is approved -before mutation. The recovery record selects the original source commit/tree, -the complete original 73-artifact CI payload, original approved lock/capsule, -and terminal bootstrap ledger by exact workflow run and artifact -ID/digest/size. Selection by “latest,” name alone, or merely matching SHA is -forbidden. Replaying the lock from those frozen inputs must produce a file -byte-identical to the approved lock, including `source` and `lockDigest`; a -rebound controller-source lock is not equivalent. +The controller must pass a fresh recovery-control `Qualified` CI run and a +separate successful recovery dry-run whose control-equivalence evidence is +approved before mutation. Recovery-control CI runs affected controller +checks/tests/policy but no payload builders or E2E; the recovery record pins and +reverifies the original complete payload CI separately. The record selects the +original source commit/tree, complete original 73-artifact CI payload, original +approved lock/capsule, immutable recovery boundary, and any required terminal +bootstrap ledger by exact workflow run/job and artifact ID/digest/size. +Selection by “latest,” name alone, or merely matching SHA is forbidden. +Replaying the lock from those frozen inputs must produce a file byte-identical +to the approved lock, including `source` and `lockDigest`; a rebound +controller-source lock is not equivalent. Recovery never invokes `publish-bootstrap`, never creates a controller-bound replacement ledger, and never uses an automatic continuation. It verifies the -terminal source-bound ledger, inventories the exhaustive original lock -(including generated payload parts), and runs only the root `publish` +terminal source-bound ledger when the record requires one; otherwise it proves +the complete exact-source staged GitHub boundary and refuses any live state +that would require an unrecorded ledger. It inventories the exhaustive original +lock (including generated payload parts) and runs only the root `publish` operation. Existing exact registry identities are skipped only after their bytes match the lock. An absent identity may be published once from the frozen source payload; any conflict fails closed and requires a new product version. diff --git a/tools/graph/ci_plan.mjs b/tools/graph/ci_plan.mjs index fcaa0bad..bba537b8 100644 --- a/tools/graph/ci_plan.mjs +++ b/tools/graph/ci_plan.mjs @@ -29,12 +29,15 @@ import { exactExtensionProducts, extensionSqlNames, } from "../release/release-artifact-targets.mjs"; +import { verifyPublicationRecoveryCandidate } from "../release/verify-publication-candidate.mjs"; const ROOT = path.resolve(import.meta.dir, "../.."); const PREFIX = "ci_plan.mjs"; export const BASE_JOBS = new Set(["affected"]); export const ALWAYS_JOBS = new Set(BASE_JOBS); +export const FULL_PAYLOAD_QUALIFICATION_MODE = "full-payload"; +export const RECOVERY_CONTROL_QUALIFICATION_MODE = "recovery-control"; export const BUILDER_JOBS = new Set([ "broker-release-assets", "broker-runtime", @@ -621,6 +624,47 @@ export function planForPullRequest() { return { jobs, projects, tasks: directTasks, reason, selectedTargets: selectedNativeTargets }; } +export function recoveryControlPlanForAffected( + { directProjects, projects, directTasks }, + { releaseSha, controllerSha }, +) { + return { + jobs: new Set(BASE_JOBS), + projects: new Set(projects), + tasks: new Set(directTasks), + reason: + `same-version recovery control changes from ${releaseSha} to ${controllerSha}; ` + + `direct affected projects: ${sorted(directProjects).join(", ") || "(none)"}; ` + + `downstream affected projects: ${sorted(projects).join(", ") || "(none)"}; ` + + `direct affected tasks: ${sorted(directTasks).join(", ") || "(none)"}`, + selectedTargets: null, + qualificationMode: RECOVERY_CONTROL_QUALIFICATION_MODE, + qualificationBaseSha: releaseSha, + qualificationHeadSha: controllerSha, + }; +} + +export function planForRecoveryControl() { + const base = process.env.MOON_BASE?.trim().toLowerCase(); + const head = process.env.MOON_HEAD?.trim().toLowerCase(); + if (!/^[0-9a-f]{40}$/u.test(base ?? "") || !/^[0-9a-f]{40}$/u.test(head ?? "")) { + throw new Error("recovery-control planning requires exact MOON_BASE and MOON_HEAD commit SHAs"); + } + const recovery = verifyPublicationRecoveryCandidate({ headRef: head }); + if (recovery === null) { + throw new Error("recovery-control planning requires a fully verified same-version recovery lineage"); + } + if (recovery.releaseSha !== base || recovery.publicationSha !== head) { + throw new Error( + "recovery-control affected range does not match the verified release source and controller", + ); + } + return recoveryControlPlanForAffected(affectedProjectsAndTasks(), { + releaseSha: recovery.releaseSha, + controllerSha: recovery.publicationSha, + }); +} + export function selectedExtensionProductsForPlan(directProjects, tasks, jobs) { const extensionJobs = new Set([ "extension-artifacts-native", @@ -822,7 +866,16 @@ function targetsForJobs(jobs) { } function renderPlan( - { jobs, projects, tasks, reason, selectedTargets }, + { + jobs, + projects, + tasks, + reason, + selectedTargets, + qualificationMode = FULL_PAYLOAD_QUALIFICATION_MODE, + qualificationBaseSha = null, + qualificationHeadSha = null, + }, { nativeTarget = process.env.NATIVE_TARGET || "all", wasmTarget = process.env.WASM_TARGET || "all", @@ -838,6 +891,9 @@ function renderPlan( selectedExtensionProducts, nativeTarget, wasmTarget, + qualificationMode, + qualificationBaseSha, + qualificationHeadSha, }); } @@ -953,7 +1009,27 @@ export function renderPlanWithSelection({ selectedExtensionProducts, nativeTarget = process.env.NATIVE_TARGET || "all", wasmTarget = process.env.WASM_TARGET || "all", + qualificationMode = FULL_PAYLOAD_QUALIFICATION_MODE, + qualificationBaseSha = null, + qualificationHeadSha = null, }) { + if (![FULL_PAYLOAD_QUALIFICATION_MODE, RECOVERY_CONTROL_QUALIFICATION_MODE].includes(qualificationMode)) { + throw new Error(`unknown CI qualification mode ${qualificationMode}`); + } + if (qualificationMode === RECOVERY_CONTROL_QUALIFICATION_MODE) { + if ( + !/^[0-9a-f]{40}$/u.test(qualificationBaseSha ?? "") + || !/^[0-9a-f]{40}$/u.test(qualificationHeadSha ?? "") + || qualificationBaseSha === qualificationHeadSha + ) { + throw new Error("recovery-control plan requires distinct exact source and controller SHAs"); + } + if (JSON.stringify(sorted(jobs)) !== JSON.stringify(sorted(BASE_JOBS))) { + throw new Error("recovery-control plan must not select builder or E2E payload jobs"); + } + } else if (qualificationBaseSha !== null || qualificationHeadSha !== null) { + throw new Error("full-payload plan must not carry a recovery affected range"); + } const extensionProducts = sorted(selectedExtensionProducts ?? new Set()); const extensionSqlNames = extensionSqlNamesForProducts(extensionProducts); const nativeLifecycleProducts = jobs.has(NATIVE_EXTENSION_LIFECYCLE_JOB) @@ -964,6 +1040,9 @@ export function renderPlanWithSelection({ const nativeLifecycleSqlNames = extensionSqlNamesForProducts(nativeLifecycleProducts); const nativeLifecycleShards = nativeExtensionLifecycleShardPlan(nativeLifecycleProducts); const plan = { + qualification_mode: qualificationMode, + qualification_base_sha: qualificationBaseSha, + qualification_head_sha: qualificationHeadSha, jobs: sorted(jobs), builder_jobs: sorted(new Set([...jobs].filter((job) => BUILDER_JOBS.has(job)))), e2e_jobs: mobileE2eJobsForPlan(jobs), @@ -1113,7 +1192,13 @@ function writePlanArtifact(plan) { export function emitGithubOutputs() { let planned; try { - if (process.env.GITHUB_EVENT_NAME === "pull_request") { + const qualificationMode = process.env.CI_QUALIFICATION_MODE || FULL_PAYLOAD_QUALIFICATION_MODE; + if (![FULL_PAYLOAD_QUALIFICATION_MODE, RECOVERY_CONTROL_QUALIFICATION_MODE].includes(qualificationMode)) { + throw new Error(`unknown CI qualification mode ${qualificationMode}`); + } + if (qualificationMode === RECOVERY_CONTROL_QUALIFICATION_MODE) { + planned = renderPlan(planForRecoveryControl()); + } else if (process.env.GITHUB_EVENT_NAME === "pull_request") { const pullRequestPlan = planForPullRequest(); let directProjects = new Set(); try { diff --git a/tools/policy/assertions/workflow-semantics.mjs b/tools/policy/assertions/workflow-semantics.mjs index aa453820..76c44e74 100644 --- a/tools/policy/assertions/workflow-semantics.mjs +++ b/tools/policy/assertions/workflow-semantics.mjs @@ -2199,8 +2199,10 @@ function assertNormalStageConditions(workflow) { '--head-ref "$RELEASE_HEAD_SHA"', ) && productDryRun.env?.CI_RUN_ID - === "${{ steps.ci_qualification.outputs.run_id }}", - `${jobId} product dry-run registry identity checks must use the verified release source while qualified replay remains controller-bound`, + === "${{ steps.release_artifact_source.outputs.ci_run_id }}" + && productDryRun.env?.CANDIDATE_MODE + === "${{ steps.verify_publication_candidate.outputs.mode }}", + `${jobId} product dry-run registry identity checks and qualified replay must use the verified payload source while controller qualification remains separate`, ); for (const id of [ "verify_oidc_identity", @@ -3701,7 +3703,7 @@ function assertDryRunEvidence(workflow) { const recoveryModeCondition = "${{ steps.verify_publication_candidate.outputs.mode == 'release-recovery' }}"; const recoveryGithubStateCondition = - "${{ steps.release_plan.outputs.has_release_changes == 'true' && inputs.operation == 'publish-dry-run' && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }}"; + "${{ steps.release_plan.outputs.has_release_changes == 'true' && (inputs.operation == 'publish-dry-run' || inputs.operation == 'publish') && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }}"; const recoveryDryRunCondition = "${{ inputs.operation == 'publish-dry-run' && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }}"; const recoveryPublishCondition = @@ -3847,8 +3849,10 @@ function assertDryRunEvidence(workflow) { ); assertActiveTokens(recoveryGithubState, [ "recovery-preflight", + "recovery-staged-preflight", '--products-json "$PRODUCTS_JSON"', '--head-ref "$RELEASE_SOURCE_SHA"', + "--state staged", ], "same-version recovery resumable GitHub-state proof"); const controllerQualification = stepById(workflow, "publish", "ci_qualification"); @@ -3987,6 +3991,34 @@ function assertDryRunEvidence(workflow) { "--artifact oliphaunt-publication-lock", ], "original same-version recovery lock download"); + const stagedBoundary = stepById( + workflow, + "publish", + "verify_github_staged_recovery_boundary", + ); + invariant( + normalized(stagedBoundary.step.if) + === "${{ steps.download_recovery_original_publication_lock.outcome == 'success' && steps.recovery_source.outputs.recovery_boundary_kind == 'github-staged' }}" + && stagedBoundary.step["continue-on-error"] === undefined + && stagedBoundary.step["timeout-minutes"] === 5 + && sameSet(Object.keys(stagedBoundary.step.env ?? {}), [ + "GH_TOKEN", + "RECOVERY_PROVENANCE_RECORD_JSON", + ]) + && stagedBoundary.step.env?.GH_TOKEN === "${{ secrets.GITHUB_TOKEN }}" + && stagedBoundary.step.env?.RECOVERY_PROVENANCE_RECORD_JSON + === "${{ steps.recovery_source.outputs.record_json }}" + && recoveryDownload.index < stagedBoundary.index, + "GitHub-staged recovery must reverify its exact pinned failed-run boundary against the approved lock", + ); + assertActiveTokens(stagedBoundary, [ + 'printf \'%s\\n\' "$RECOVERY_PROVENANCE_RECORD_JSON"', + "tools/release/verify-github-staged-recovery-boundary.mjs", + '--boundary "$RUNNER_TEMP/recovery-boundary.json"', + '--approved-lock "$RUNNER_TEMP/recovery-original-publication-lock/publication-lock.json"', + '--repo "$GITHUB_REPOSITORY"', + ], "pinned GitHub-staged recovery boundary"); + const controlApproval = stepById( workflow, "publish", @@ -4123,12 +4155,14 @@ function assertDryRunEvidence(workflow) { && sameSet(Object.keys(recoveryPublication.step.env ?? {}), ["PRODUCTS_JSON"]) && recoveryPublication.step.env?.PRODUCTS_JSON === "${{ steps.release_plan.outputs.products_json }}", - "same-version recovery must prove a nonempty exact public registry prefix", + "same-version recovery must prove its exact immutable GitHub-staged or public-registry boundary", ); assertActiveTokens(recoveryPublication, [ '--lock "$PUBLICATION_LOCK_PATH"', "--inventory target/release/recovery-registry-inventory.json", + '--immutable-github-tag-count "${{ steps.verify_release_recovery_github_state.outputs.exact_tag_count }}"', '--products-json "$PRODUCTS_JSON"', + '--recovery-boundary-kind "${{ steps.recovery_source.outputs.recovery_boundary_kind }}"', "--output target/release/recovery-evidence/publication-state.json", '--github-output "$GITHUB_OUTPUT"', ], "same-version recovery partial-publication proof"); @@ -4165,6 +4199,29 @@ function assertDryRunEvidence(workflow) { "same-version recovery must have one immutable bootstrap-ledger selector", ); const [ledger] = ledgerDownload; + const ledgerRequirement = stepById( + workflow, + "publish", + "require_recovery_bootstrap_ledger", + ); + invariant( + normalized(ledgerRequirement.step.if) + === "${{ steps.bootstrap_ledger_state.outputs.needs_ledger == 'true' && steps.verify_publication_candidate.outputs.mode == 'release-recovery' }}" + && ledgerRequirement.step["continue-on-error"] === undefined + && sameSet(Object.keys(ledgerRequirement.step.env ?? {}), [ + "PINNED_BOOTSTRAP_LEDGER_REQUIRED", + ]) + && ledgerRequirement.step.env?.PINNED_BOOTSTRAP_LEDGER_REQUIRED + === "${{ steps.recovery_source.outputs.bootstrap_ledger_required }}" + && stagedBoundary.index < ledgerRequirement.index + && ledgerRequirement.index < ledger.index, + "recovery must reject live bootstrap state that lacks an exact pinned source ledger", + ); + assertActiveTokens(ledgerRequirement, [ + '[[ "$PINNED_BOOTSTRAP_LEDGER_REQUIRED" != true ]]', + "Recovery requires bootstrap evidence", + "exit 1", + ], "conditional pinned bootstrap ledger requirement"); invariant( normalized(ledger.step.if) === "${{ steps.bootstrap_ledger_state.outputs.needs_ledger == 'true' }}" diff --git a/tools/policy/assertions/workflow-semantics.test.mjs b/tools/policy/assertions/workflow-semantics.test.mjs index c6061c4e..a1650086 100644 --- a/tools/policy/assertions/workflow-semantics.test.mjs +++ b/tools/policy/assertions/workflow-semantics.test.mjs @@ -1818,7 +1818,7 @@ test("product dry-run registry identities remain bound to the release source", ( ).run.replace("$RELEASE_SOURCE_SHA", "$RELEASE_HEAD_SHA"); assert.throws( () => assertReleaseOperationWorkflow(controllerBound), - /product dry-run registry identity checks must use the verified release source/u, + /product dry-run registry identity checks and qualified replay must use the verified payload source/u, ); const duplicateHead = candidate(); @@ -1829,7 +1829,7 @@ test("product dry-run registry identities remain bound to the release source", ( ).run += ' --head-ref "$UNVERIFIED_REF"'; assert.throws( () => assertReleaseOperationWorkflow(duplicateHead), - /product dry-run registry identity checks must use the verified release source/u, + /product dry-run registry identity checks and qualified replay must use the verified payload source/u, ); const unqualifiedRun = candidate(); @@ -1840,7 +1840,7 @@ test("product dry-run registry identities remain bound to the release source", ( ).env.CI_RUN_ID = "${{ github.run_id }}"; assert.throws( () => assertReleaseOperationWorkflow(unqualifiedRun), - /product dry-run registry identity checks must use the verified release source/u, + /product dry-run registry identity checks and qualified replay must use the verified payload source/u, ); } }); diff --git a/tools/release/manage-release-drafts.test.mjs b/tools/release/manage-release-drafts.test.mjs index 2d24c414..3aae09b4 100644 --- a/tools/release/manage-release-drafts.test.mjs +++ b/tools/release/manage-release-drafts.test.mjs @@ -1052,27 +1052,27 @@ test("same-version recovery preflight accepts only absent or exact-SHA resumable repo: "o/r", selected, }; - assert.doesNotThrow(() => reconcileSelectedReleasesSync(context, { + assert.deepEqual(reconcileSelectedReleasesSync(context, { readReleaseMap: () => new Map(), readTagMap: () => new Map(absentTags), - })); + }), { exactReleaseCount: 0, exactTagCount: 0, selectedCount: 2 }); const exactTags = tagState(selected, headRef); const exactReleases = new Map(selected.map(({ metadata, tag }, index) => [ tag, { ...metadata, draft: index === 0, id: index + 1 }, ])); - assert.doesNotThrow(() => reconcileSelectedReleasesSync(context, { + assert.deepEqual(reconcileSelectedReleasesSync(context, { readReleaseMap: () => exactReleases, readTagMap: () => exactTags, - })); + }), { exactReleaseCount: 2, exactTagCount: 2, selectedCount: 2 }); const partialTags = new Map(absentTags); partialTags.set(selected[0].tag, exactTags.get(selected[0].tag)); - assert.doesNotThrow(() => reconcileSelectedReleasesSync(context, { + assert.deepEqual(reconcileSelectedReleasesSync(context, { readReleaseMap: () => new Map([[selected[0].tag, exactReleases.get(selected[0].tag)]]), readTagMap: () => partialTags, - })); + }), { exactReleaseCount: 1, exactTagCount: 1, selectedCount: 2 }); const conflictingTag = new Map(exactTags); conflictingTag.set(selected[0].tag, { @@ -1097,6 +1097,47 @@ test("same-version recovery preflight accepts only absent or exact-SHA resumable ); }); +test("GitHub-staged recovery preflight requires the complete exact source-bound set", () => { + const selected = selection(2); + const headRef = "d".repeat(40); + const exactTags = tagState(selected, headRef); + const exactReleases = new Map(selected.map(({ metadata, tag }, index) => [ + tag, + { ...metadata, draft: true, id: index + 1 }, + ])); + const context = { + budget: budget(), + command: "recovery-staged-preflight", + environment: {}, + expectedState: "staged", + headRef, + repo: "o/r", + selected, + }; + const noSleep = { releaseSnapshotSleep: () => {} }; + assert.deepEqual(reconcileSelectedReleasesSync(context, { + ...noSleep, + readReleaseMap: () => exactReleases, + readTagMap: () => exactTags, + }), { exactReleaseCount: 2, exactTagCount: 2, selectedCount: 2 }); + + const partialTags = new Map(exactTags); + partialTags.set(selected[0].tag, null); + assert.throws(() => reconcileSelectedReleasesSync(context, { + ...noSleep, + readReleaseMap: () => exactReleases, + readTagMap: () => partialTags, + }), /tag .* does not exist/u); + + const partialReleases = new Map(exactReleases); + partialReleases.delete(selected[0].tag); + assert.throws(() => reconcileSelectedReleasesSync(context, { + ...noSleep, + readReleaseMap: () => partialReleases, + readTagMap: () => exactTags, + }), /did not converge to staged state/u); +}); + test("batch staging reconciles ambiguous responses once and exact reruns issue no mutations", () => { const selected = selection(1); const headRef = "d".repeat(40); diff --git a/tools/release/qualified-release-replay.mjs b/tools/release/qualified-release-replay.mjs index 1cc4118f..968b01a5 100644 --- a/tools/release/qualified-release-replay.mjs +++ b/tools/release/qualified-release-replay.mjs @@ -1,6 +1,42 @@ import { captureCommandOutput } from "../dev/capture-command-output.mjs"; const EXACT_SHA = /^[0-9a-f]{40}$/u; +const POSITIVE_INTEGER = /^[1-9][0-9]*$/u; + +export function qualifiedReplayCandidateBinding({ + candidateMode, + controllerSha, + releaseSourceSha, + runId, +}) { + const controller = String(controllerSha ?? "").toLowerCase(); + const source = String(releaseSourceSha ?? "").toLowerCase(); + const normalizedRunId = String(runId ?? ""); + if (!EXACT_SHA.test(controller) || !EXACT_SHA.test(source)) { + throw new Error("qualified release replay candidate binding requires exact controller and source SHAs"); + } + if (!POSITIVE_INTEGER.test(normalizedRunId)) { + throw new Error("qualified release replay candidate binding requires a positive CI run ID"); + } + if (!new Set(["release-bump", "release-recovery"]).has(candidateMode)) { + throw new Error(`qualified release replay candidate mode is invalid: ${candidateMode}`); + } + const recovery = candidateMode === "release-recovery"; + if (recovery && controller === source) { + throw new Error("qualified recovery replay requires distinct controller and release source SHAs"); + } + if (!recovery && controller !== source) { + throw new Error("ordinary qualified replay requires identical controller and release source SHAs"); + } + return Object.freeze({ + candidateRoot: recovery + ? "target/recovery-payload-candidate" + : "target/release-candidate", + candidateSha: recovery ? source : controller, + qualificationMode: "full-payload", + runId: normalizedRunId, + }); +} function git(repo, args, { allowEmptyOutput = false, stdoutTerminator = undefined } = {}) { const result = captureCommandOutput("git", args, { diff --git a/tools/release/qualified-release-replay.test.mjs b/tools/release/qualified-release-replay.test.mjs index bc58e723..5f04b7f9 100644 --- a/tools/release/qualified-release-replay.test.mjs +++ b/tools/release/qualified-release-replay.test.mjs @@ -5,7 +5,10 @@ import { tmpdir } from "node:os"; import path from "node:path"; import { test } from "node:test"; -import { assertQualifiedReplaySourceState } from "./qualified-release-replay.mjs"; +import { + assertQualifiedReplaySourceState, + qualifiedReplayCandidateBinding, +} from "./qualified-release-replay.mjs"; function git(repo, ...args) { return execFileSync("git", args, { cwd: repo, encoding: "utf8" }).trim(); @@ -102,6 +105,67 @@ test("qualified replay separately binds a recovery controller and release source } }); +test("qualified replay selects controller evidence normally and frozen source evidence for recovery", () => { + const source = "1".repeat(40); + const controller = "2".repeat(40); + assert.deepEqual( + qualifiedReplayCandidateBinding({ + candidateMode: "release-bump", + controllerSha: source, + releaseSourceSha: source, + runId: "123", + }), + { + candidateRoot: "target/release-candidate", + candidateSha: source, + qualificationMode: "full-payload", + runId: "123", + }, + ); + assert.deepEqual( + qualifiedReplayCandidateBinding({ + candidateMode: "release-recovery", + controllerSha: controller, + releaseSourceSha: source, + runId: "456", + }), + { + candidateRoot: "target/recovery-payload-candidate", + candidateSha: source, + qualificationMode: "full-payload", + runId: "456", + }, + ); + for (const fixture of [ + { + candidateMode: "release-recovery", + controllerSha: source, + releaseSourceSha: source, + runId: "456", + }, + { + candidateMode: "release-bump", + controllerSha: controller, + releaseSourceSha: source, + runId: "456", + }, + { + candidateMode: "unknown", + controllerSha: controller, + releaseSourceSha: source, + runId: "456", + }, + { + candidateMode: "release-recovery", + controllerSha: controller, + releaseSourceSha: source, + runId: "0", + }, + ]) { + assert.throws(() => qualifiedReplayCandidateBinding(fixture), /replay/u); + } +}); + test("qualified replay rejects tracked, staged, and untracked source changes", () => { for (const mutate of [ (repo) => writeFileSync(path.join(repo, "tracked.txt"), "modified\n"), diff --git a/tools/release/recovery-control-ci.test.mjs b/tools/release/recovery-control-ci.test.mjs new file mode 100644 index 00000000..69fbc6db --- /dev/null +++ b/tools/release/recovery-control-ci.test.mjs @@ -0,0 +1,205 @@ +#!/usr/bin/env bun + +import assert from "node:assert/strict"; +import { + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import test from "node:test"; + +import { execFileSync, spawnSync } from "../test/fd-backed-spawn-sync.mjs"; + +import { affectedPlanBinding } from "../../.github/scripts/release-candidate-lib.mjs"; +import { + BASE_JOBS, + BUILDER_JOBS, + recoveryControlPlanForAffected, + renderPlanForFullRun, + renderPlanWithSelection, +} from "../graph/ci_plan.mjs"; + +const ROOT = path.resolve(import.meta.dir, "../.."); + +test("recovery-control plans bind the exact affected range and omit every payload lane", () => { + const releaseSha = "1".repeat(40); + const controllerSha = "2".repeat(40); + const selected = recoveryControlPlanForAffected({ + directProjects: new Set(["ci-workflows", "release-tools"]), + projects: new Set(["ci-workflows", "release-tools"]), + directTasks: new Set(["release-tools:check", "release-tools:test"]), + }, { releaseSha, controllerSha }); + const plan = renderPlanWithSelection({ + ...selected, + selectedExtensionProducts: null, + }); + + assert.deepEqual(plan.jobs, [...BASE_JOBS].sort()); + assert.deepEqual(plan.builder_jobs, []); + assert.deepEqual(plan.e2e_jobs, []); + assert.equal([...BUILDER_JOBS].some((job) => plan.jobs.includes(job)), false); + assert.equal(plan.qualification_mode, "recovery-control"); + assert.equal(plan.qualification_base_sha, releaseSha); + assert.equal(plan.qualification_head_sha, controllerSha); + assert.match(plan.reason, new RegExp(`${releaseSha} to ${controllerSha}`, "u")); + for (const matrix of [ + "broker_runtime_matrix", + "extension_artifacts_native_matrix", + "extension_artifacts_wasix_matrix", + "js_exact_candidate_consumer_matrix", + "liboliphaunt_native_android_runtime_matrix", + "liboliphaunt_native_desktop_runtime_matrix", + "liboliphaunt_native_ios_runtime_matrix", + "liboliphaunt_wasix_aot_runtime_matrix", + "node_direct_runtime_matrix", + "react_native_android_mobile_app_matrix", + ]) { + assert.deepEqual(plan[matrix], { include: [] }, `${matrix} must be empty`); + } +}); + +test("CI obtains recovery-control mode only after release-intent lineage verification", () => { + const workflow = Bun.YAML.parse( + readFileSync(path.join(ROOT, ".github/workflows/ci.yml"), "utf8"), + ); + const intent = readFileSync( + path.join(ROOT, ".github/scripts/check-release-intent.sh"), + "utf8", + ); + const verifier = intent.indexOf("tools/release/verify-publication-candidate.mjs"); + const modeOutput = intent.indexOf('qualification_mode="recovery-control"'); + assert(verifier >= 0 && verifier < modeOutput); + + const releaseIntent = workflow.jobs["release-intent"]; + assert.equal( + releaseIntent.outputs.qualification_mode, + "${{ steps.release_intent.outputs.qualification_mode }}", + ); + const affected = workflow.jobs.affected; + const plan = affected.steps.find((step) => step.id === "plan"); + const matrices = affected.steps.find((step) => step.id === "target-matrices"); + for (const step of [plan, matrices]) { + assert.match(step.env.MOON_BASE, /recovery_release_sha/u); + assert.match(step.env.MOON_HEAD, /github[.]sha/u); + } + assert.match(plan.env.CI_QUALIFICATION_MODE, /release-intent/u); + + const macMetadata = workflow.jobs["release-metadata-portability"]; + assert.equal(macMetadata["runs-on"], "macos-26"); + assert.equal(macMetadata.if, undefined); + assert.deepEqual(workflow.jobs.required.needs, [ + "affected", + "release-intent", + "checks", + "tests", + "builds", + "e2e", + ]); + assert.deepEqual(workflow.jobs.qualified.needs, ["affected", "required"]); + const writer = workflow.jobs.qualified.steps.find((step) => step.id === "qualification_record"); + assert.match(writer.env.CI_QUALIFICATION_MODE, /needs[.]affected[.]outputs[.]qualification_mode/u); +}); + +test("ordinary non-PR planning remains full-payload", () => { + const full = renderPlanForFullRun(); + assert.equal(full.qualification_mode, "full-payload"); + assert.equal(full.qualification_base_sha, null); + assert.equal(full.qualification_head_sha, null); + assert(full.builder_jobs.length > 0); + assert(full.e2e_jobs.length > 0); + + const workflow = Bun.YAML.parse( + readFileSync(path.join(ROOT, ".github/workflows/ci.yml"), "utf8"), + ); + const plan = workflow.jobs.affected.steps.find((step) => step.id === "plan"); + assert.match(plan.env.CI_QUALIFICATION_MODE, /needs[.]release-intent[.]outputs[.]qualification_mode/u); + const script = readFileSync( + path.join(ROOT, ".github/scripts/check-release-intent.sh"), + "utf8", + ); + assert.match(script, /qualification_mode="full-payload"/u); +}); + +test("publication must explicitly request a recovery-control candidate", () => { + const root = mkdtempSync(path.join(tmpdir(), "oliphaunt-recovery-control-candidate-")); + try { + const releaseSha = "1".repeat(40); + const controllerSha = execFileSync("git", ["rev-parse", "HEAD^{commit}"], { + cwd: ROOT, + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + }).trim(); + const tree = execFileSync("git", ["rev-parse", "HEAD^{tree}"], { + cwd: ROOT, + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + }).trim(); + const planPath = path.join(root, "ci-plan.json"); + const candidatePath = path.join(root, "candidate.json"); + writeFileSync(planPath, `${JSON.stringify({ + qualification_mode: "recovery-control", + qualification_base_sha: releaseSha, + qualification_head_sha: controllerSha, + jobs: ["affected"], + projects: ["release-tools"], + extension_package_products: [], + }, null, 2)}\n`); + writeFileSync(candidatePath, `${JSON.stringify({ + schemaVersion: 2, + repository: "f0rr0/oliphaunt", + workflow: "CI", + workflowRef: "f0rr0/oliphaunt/.github/workflows/ci.yml@refs/heads/main", + runId: "123456", + runAttempt: 1, + eventName: "push", + ref: "refs/heads/main", + sha: controllerSha, + tree, + affectedPlan: affectedPlanBinding(planPath, false), + evidenceRequirements: { + wasixReleaseRegression: false, + artifacts: [], + }, + evidence: { wasixReleaseRegression: null }, + }, null, 2)}\n`); + const baseArgs = [ + ".github/scripts/verify-release-candidate.mjs", + candidatePath, + "--plan", + planPath, + "--wasix-evidence-required", + "false", + ]; + const environment = { + ...process.env, + CI_RUN_ID: "123456", + GITHUB_REPOSITORY: "f0rr0/oliphaunt", + RELEASE_HEAD_SHA: controllerSha, + }; + const accepted = spawnSync(process.execPath, [ + ...baseArgs, + "--qualification-mode", + "recovery-control", + ], { + cwd: ROOT, + encoding: "utf8", + env: environment, + stdio: ["ignore", "pipe", "pipe"], + }); + assert.equal(accepted.status, 0, accepted.stderr); + + const defaulted = spawnSync(process.execPath, baseArgs, { + cwd: ROOT, + encoding: "utf8", + env: environment, + stdio: ["ignore", "pipe", "pipe"], + }); + assert.equal(defaulted.status, 1); + assert.match(defaulted.stderr, /expected full-payload, got recovery-control/u); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); diff --git a/tools/release/recovery-promotion-attestation.mjs b/tools/release/recovery-promotion-attestation.mjs index 9018f023..cecc0e50 100644 --- a/tools/release/recovery-promotion-attestation.mjs +++ b/tools/release/recovery-promotion-attestation.mjs @@ -14,8 +14,11 @@ import path from "node:path"; import process from "node:process"; import { + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, SAME_VERSION_RECOVERY_SOURCES_SCHEMA, canonicalRecoverySourceJson, + isSameVersionRecoverySourcesDocument, + sameVersionRecoverySourceProvenanceSchema, selectSameVersionRecoverySource, validateSameVersionRecoverySource, } from "./same-version-recovery-source.mjs"; @@ -534,7 +537,7 @@ function validateSourceProvenanceRecord(record, lockBinding) { } return canonical({ recordDigest: sha256(canonicalRecoverySourceJson(record)), - schema: SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + schema: sameVersionRecoverySourceProvenanceSchema(record), }); } @@ -798,7 +801,10 @@ function validatePredicateShape(predicate) { "recovery promotion predicate.sourceProvenance", { canonicalOrder: true }, ); - if (predicate.sourceProvenance.schema !== SAME_VERSION_RECOVERY_SOURCES_SCHEMA) { + if (!new Set([ + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + ]).has(predicate.sourceProvenance.schema)) { throw error("recovery promotion predicate source provenance schema is invalid"); } requireHash( @@ -930,11 +936,7 @@ function readBoundedJson(file, context) { } function selectedProvenanceRecord(value, releaseSha) { - if ( - isPlainObject(value) - && value.schema === SAME_VERSION_RECOVERY_SOURCES_SCHEMA - && Array.isArray(value.records) - ) { + if (isSameVersionRecoverySourcesDocument(value)) { return selectSameVersionRecoverySource( value, releaseSha, diff --git a/tools/release/recovery-promotion-attestation.test.mjs b/tools/release/recovery-promotion-attestation.test.mjs index 7b73c179..20e68da3 100644 --- a/tools/release/recovery-promotion-attestation.test.mjs +++ b/tools/release/recovery-promotion-attestation.test.mjs @@ -15,6 +15,7 @@ import test from "node:test"; import { execFileSync } from "../test/fd-backed-spawn-sync.mjs"; import { DEFAULT_SAME_VERSION_RECOVERY_SOURCES, + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, loadSameVersionRecoverySources, } from "./same-version-recovery-source.mjs"; import { @@ -253,6 +254,10 @@ test("creates one deterministic canonical predicate binding both runs and every predicate.sourceProvenance.recordDigest, sha256(canonicalRecoveryPromotionJson(values.provenanceRecord)), ); + assert.equal( + predicate.sourceProvenance.schema, + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + ); const body = structuredClone(predicate); delete body.evidenceDigest; assert.equal(predicate.evidenceDigest, digestValue(body)); diff --git a/tools/release/release-candidate-lib.test.mjs b/tools/release/release-candidate-lib.test.mjs index 78a09ce0..77fe8e8e 100644 --- a/tools/release/release-candidate-lib.test.mjs +++ b/tools/release/release-candidate-lib.test.mjs @@ -13,6 +13,7 @@ import { affectedPlanBinding, assertBindingMatches, assertCandidateBindingShape, + candidateQualificationMode, wasixEvidenceBinding, } from "../../.github/scripts/release-candidate-lib.mjs"; @@ -230,3 +231,93 @@ test("rejects a changed selected-product set even when WASIX remains required", cleanup(); } }); + +test("keeps legacy F4 qualification plans backward-compatible as full-payload", () => { + const { root, cleanup } = fixture(); + try { + const planPath = path.join(root, "legacy-plan.json"); + writeFileSync(planPath, JSON.stringify({ + projects: [], + jobs: ["affected"], + extension_package_products: [], + })); + const affectedPlan = affectedPlanBinding(planPath, false); + const candidate = { + schemaVersion: 2, + affectedPlan, + evidenceRequirements: { + wasixReleaseRegression: false, + artifacts: [], + }, + evidence: { wasixReleaseRegression: null }, + }; + expect(affectedPlan.qualification).toBeUndefined(); + expect(candidateQualificationMode(candidate)).toBe("full-payload"); + expect(() => assertCandidateBindingShape(candidate)).not.toThrow(); + } finally { + cleanup(); + } +}); + +test("binds recovery-control to an exact source/controller range without payload jobs", () => { + const { root, cleanup } = fixture(); + try { + const releaseSha = "1".repeat(40); + const controllerSha = "2".repeat(40); + const planPath = path.join(root, "recovery-plan.json"); + writeFileSync(planPath, JSON.stringify({ + qualification_mode: "recovery-control", + qualification_base_sha: releaseSha, + qualification_head_sha: controllerSha, + projects: ["release-tools"], + jobs: ["affected"], + extension_package_products: [], + })); + const affectedPlan = affectedPlanBinding(planPath, false); + const candidate = { + schemaVersion: 2, + sha: controllerSha, + affectedPlan, + evidenceRequirements: { + wasixReleaseRegression: false, + artifacts: [], + }, + evidence: { wasixReleaseRegression: null }, + }; + expect(candidateQualificationMode(candidate)).toBe("recovery-control"); + expect(affectedPlan.qualification).toEqual({ + mode: "recovery-control", + baseSha: releaseSha, + headSha: controllerSha, + }); + expect(() => assertCandidateBindingShape(candidate)).not.toThrow(); + + const substituted = structuredClone(candidate); + substituted.affectedPlan.qualification.headSha = "3".repeat(40); + expect(() => assertCandidateBindingShape(substituted)).toThrow( + "controller does not match candidate SHA", + ); + } finally { + cleanup(); + } +}); + +test("rejects recovery-control plans that select any payload job", () => { + const { root, cleanup } = fixture(); + try { + const planPath = path.join(root, "invalid-recovery-plan.json"); + writeFileSync(planPath, JSON.stringify({ + qualification_mode: "recovery-control", + qualification_base_sha: "1".repeat(40), + qualification_head_sha: "2".repeat(40), + projects: [], + jobs: ["affected", "js-sdk-package"], + extension_package_products: [], + })); + expect(() => affectedPlanBinding(planPath, false)).toThrow( + "must not select builder or E2E payload jobs", + ); + } finally { + cleanup(); + } +}); diff --git a/tools/release/release-gate-topology.test.mjs b/tools/release/release-gate-topology.test.mjs index c144b8d7..746bb19c 100644 --- a/tools/release/release-gate-topology.test.mjs +++ b/tools/release/release-gate-topology.test.mjs @@ -230,8 +230,15 @@ test("qualified replay proves hosted evidence and clean source before omitting m assert.match(publisher, /assertQualifiedReplaySourceState/u); assert.match(publisher, /headRef: process[.]env[.]RELEASE_HEAD_SHA/u); assert.match(publisher, /expectedReleaseSourceSha: process[.]env[.]RELEASE_SOURCE_SHA/u); + assert.match(publisher, /qualifiedReplayCandidateBinding/u); assert.match(publisher, /verify-release-candidate[.]mjs/u); - assert.match(publisher, /target\/release-candidate\/oliphaunt-release-candidate[.]json/u); + assert.match(publisher, /replay[.]candidateRoot.*oliphaunt-release-candidate[.]json/u); + assert.match(publisher, /--qualification-mode/u); + assert.match(publisher, /RELEASE_HEAD_SHA: replay[.]candidateSha/u); + const qualifiedReplay = read("tools/release/qualified-release-replay.mjs"); + assert.match(qualifiedReplay, /target\/release-candidate/u); + assert.match(qualifiedReplay, /target\/recovery-payload-candidate/u); + assert.match(qualifiedReplay, /qualificationMode: "full-payload"/u); assert.match(publisher, /release-metadata-check[.]mjs/u); }); diff --git a/tools/release/release-publish.mjs b/tools/release/release-publish.mjs index a52b2eef..505c852b 100755 --- a/tools/release/release-publish.mjs +++ b/tools/release/release-publish.mjs @@ -85,7 +85,10 @@ import { verifyLockedRegistryIntegrity, writeRegistryReceiptEvidence, } from "./registry-integrity.mjs"; -import { assertQualifiedReplaySourceState } from "./qualified-release-replay.mjs"; +import { + assertQualifiedReplaySourceState, + qualifiedReplayCandidateBinding, +} from "./qualified-release-replay.mjs"; import { concurrentGithubReleaseAssetUploadPlan } from "./github-release-asset-upload-plan.mjs"; import { executeConcurrentGithubReleaseAssetUploadPlan, @@ -1688,6 +1691,7 @@ function verifyQualifiedCiReplay(productDryRunPlan) { fail("--qualified-ci is valid only inside the protected GitHub Actions release workflow"); } for (const name of [ + "CANDIDATE_MODE", "CI_RUN_ID", "GITHUB_REPOSITORY", "RELEASE_HEAD_SHA", @@ -1716,17 +1720,36 @@ function verifyQualifiedCiReplay(productDryRunPlan) { } catch (error) { fail(error instanceof Error ? error.message : String(error)); } + let replay; + try { + replay = qualifiedReplayCandidateBinding({ + candidateMode: process.env.CANDIDATE_MODE, + controllerSha: process.env.RELEASE_HEAD_SHA, + releaseSourceSha: process.env.RELEASE_SOURCE_SHA, + runId: process.env.CI_RUN_ID, + }); + } catch (error) { + fail(error instanceof Error ? error.message : String(error)); + } run(TOOL, [ "node", ".github/scripts/verify-release-candidate.mjs", - "target/release-candidate/oliphaunt-release-candidate.json", + `${replay.candidateRoot}/oliphaunt-release-candidate.json`, "--plan", - "target/release-candidate/affected-plan/ci-plan.json", + `${replay.candidateRoot}/affected-plan/ci-plan.json`, + "--qualification-mode", + replay.qualificationMode, "--wasix-evidence-required", process.env.WASIX_EVIDENCE_REQUIRED, "--wasix-evidence-root", - "target/release-candidate/wasix-evidence", - ]); + `${replay.candidateRoot}/wasix-evidence`, + ], { + environment: { + ...process.env, + CI_RUN_ID: replay.runId, + RELEASE_HEAD_SHA: replay.candidateSha, + }, + }); } async function runProductDryRunPlan(productDryRunPlan) { diff --git a/tools/release/release-recovery-workflow.test.mjs b/tools/release/release-recovery-workflow.test.mjs index 2a4a50c5..5f74ef6f 100644 --- a/tools/release/release-recovery-workflow.test.mjs +++ b/tools/release/release-recovery-workflow.test.mjs @@ -102,6 +102,19 @@ test("dry-run separately qualifies control HEAD and reuses the frozen release pa ); const artifactSource = namedStep(job, "Resolve exact release artifact source"); assert.match(controlCi.run, /"\$RELEASE_CONTROL_SHA"/u); + assert.match( + controlCi.env.CANDIDATE_MODE, + /verify_publication_candidate[.]outputs[.]mode/u, + ); + assert.match(controlCi.run, /CANDIDATE_MODE.*!= release-recovery/su); + assert.match( + controlCi.run, + /qualification_args\+=\(--artifact wasix-release-regression-evidence\)/u, + ); + assert.match( + controlCi.run, + /qualification_args\+=\(--artifact oliphaunt-extension-package-artifacts\)/u, + ); assert.match(payloadCi.run, /"\$RECOVERY_RELEASE_SHA"/u); assert.match( payloadCi.env.RECOVERY_RELEASE_SHA, @@ -161,6 +174,30 @@ test("dry-run separately qualifies control HEAD and reuses the frozen release pa payloadCandidate.run, /target\/recovery-payload-candidate\/oliphaunt-release-candidate[.]json/u, ); + assert.match(payloadCandidate.run, /--qualification-mode full-payload/u); + + const controlCandidate = namedStep(job, "Verify exact-SHA qualification record"); + assert.match(controlCandidate.run, /qualification_mode=recovery-control/u); + assert.match( + controlCandidate.run, + /--qualification-mode "\$qualification_mode"/u, + ); + assert.match(controlCandidate.run, /wasix_evidence_required=false/u); + assert.match( + namedStep(job, "Download required exact-SHA WASIX evidence").if, + /verify_publication_candidate[.]outputs[.]mode != 'release-recovery'/u, + ); + + const qualifiedReplay = namedStep(job, "Validate selected release product dry-runs"); + assert.match( + qualifiedReplay.env.CANDIDATE_MODE, + /verify_publication_candidate[.]outputs[.]mode/u, + ); + assert.match( + qualifiedReplay.env.CI_RUN_ID, + /release_artifact_source[.]outputs[.]ci_run_id/u, + ); + assert.match(qualifiedReplay.run, /release-publish[.]mjs publish-dry-run --qualified-ci/u); for (const name of [ "Download WASIX release assets", @@ -208,7 +245,7 @@ test("dry-run separately qualifies control HEAD and reuses the frozen release pa ); assert.match( productDryRun.env.CI_RUN_ID, - /steps[.]ci_qualification[.]outputs[.]run_id/u, + /steps[.]release_artifact_source[.]outputs[.]ci_run_id/u, ); assert.match(productDryRun.run, /--head-ref "\$RELEASE_SOURCE_SHA"/u); assert.doesNotMatch(productDryRun.run, /--head-ref "\$RELEASE_HEAD_SHA"/u); @@ -227,6 +264,30 @@ test("dry-run separately qualifies control HEAD and reuses the frozen release pa stepIndex(job, "Select original approved lock for same-version recovery") < stepIndex(job, "Freeze exhaustive publication lock"), ); + const stagedBoundary = namedStep( + job, + "Verify pinned GitHub-staged recovery boundary", + ); + assert.match( + stagedBoundary.if, + /recovery_boundary_kind == 'github-staged'/u, + ); + assert.match( + stagedBoundary.run, + /verify-github-staged-recovery-boundary[.]mjs/u, + ); + assert.match( + stagedBoundary.run, + /recovery-original-publication-lock[/]publication-lock[.]json/u, + ); + assert( + stepIndex(job, "Download original approved lock for same-version recovery") + < stepIndex(job, "Verify pinned GitHub-staged recovery boundary"), + ); + assert( + stepIndex(job, "Verify pinned GitHub-staged recovery boundary") + < stepIndex(job, "Freeze exhaustive publication lock"), + ); const prelockRegistryValidation = namedStep( job, "Validate product versions and registry state", diff --git a/tools/release/same-version-recovery-source.mjs b/tools/release/same-version-recovery-source.mjs index 622b7e81..03e97f51 100644 --- a/tools/release/same-version-recovery-source.mjs +++ b/tools/release/same-version-recovery-source.mjs @@ -13,8 +13,10 @@ import { fileURLToPath } from "node:url"; import { captureCommandOutput } from "../dev/capture-command-output.mjs"; const TOOL = "same-version-recovery-source.mjs"; -export const SAME_VERSION_RECOVERY_SOURCES_SCHEMA = +export const LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA = "oliphaunt-same-version-recovery-sources-v1"; +export const SAME_VERSION_RECOVERY_SOURCES_SCHEMA = + "oliphaunt-same-version-recovery-sources-v2"; const PUBLICATION_LOCK_SCHEMA = "oliphaunt-publication-lock-v1"; const CAPSULE_SCHEMA = "oliphaunt-bootstrap-publication-capsule-v1"; const LEDGER_SCHEMA = "oliphaunt-bootstrap-ledger-checkpoint-v1"; @@ -237,6 +239,46 @@ function successfulRun(value, context, source) { return value; } +function recoveryBoundary(value, context, source) { + if (value?.kind === "registry-partial") { + strictObject(value, ["kind"], context); + return value; + } + strictObject( + value, + ["evidenceArtifact", "job", "kind", "run"], + context, + ); + if (value.kind !== "github-staged") { + throw error(`${context}.kind must be registry-partial or github-staged`); + } + strictObject( + value.run, + ["attempt", "conclusion", "event", "headSha", "id", "status"], + `${context}.run`, + ); + positiveInteger(value.run.id, `${context}.run.id`); + positiveInteger(value.run.attempt, `${context}.run.attempt`); + if ( + value.run.event !== "workflow_dispatch" + || value.run.status !== "completed" + || value.run.conclusion !== "failure" + || value.run.headSha !== source.commit + ) { + throw error(`${context}.run must identify a failed completed source-bound workflow_dispatch`); + } + strictObject(value.job, ["conclusion", "id", "name"], `${context}.job`); + positiveInteger(value.job.id, `${context}.job.id`); + if ( + value.job.name !== "Prepare and stage release" + || value.job.conclusion !== "failure" + ) { + throw error(`${context}.job must identify the failed GitHub staging job`); + } + artifactIdentity(value.evidenceArtifact, `${context}.evidenceArtifact`); + return value; +} + function releaseEnvelope(value, context) { strictObject( value, @@ -402,12 +444,14 @@ function requiredArtifactNames(value, inventory, context) { function validateRecord(value, index) { const context = `records[${index}]`; + const hasRecoveryBoundary = Object.hasOwn(value ?? {}, "recoveryBoundary"); strictObject( value, [ "approvedDryRun", "bootstrapLedger", "payloadQualification", + ...(hasRecoveryBoundary ? ["recoveryBoundary"] : []), "releaseEnvelope", "releaseSource", ], @@ -415,6 +459,11 @@ function validateRecord(value, index) { ); const source = sourceIdentity(value.releaseSource, `${context}.releaseSource`); const envelope = releaseEnvelope(value.releaseEnvelope, `${context}.releaseEnvelope`); + if (hasRecoveryBoundary) { + recoveryBoundary(value.recoveryBoundary, `${context}.recoveryBoundary`, source); + } else if (value.bootstrapLedger === null) { + throw error(`${context} legacy recovery source requires a terminal bootstrap ledger`); + } strictObject( value.payloadQualification, @@ -468,51 +517,72 @@ function validateRecord(value, index) { envelope, ); - strictObject( - value.bootstrapLedger, - ["artifactInventory", "run", "terminalCheckpoint", "workflow"], - `${context}.bootstrapLedger`, - ); - workflow( - value.bootstrapLedger.workflow, - `${context}.bootstrapLedger.workflow`, - { name: "Release", path: ".github/workflows/release.yml" }, - ); - successfulRun(value.bootstrapLedger.run, `${context}.bootstrapLedger.run`, source); - const ledgerInventory = artifactInventory( - value.bootstrapLedger.artifactInventory, - `${context}.bootstrapLedger.artifactInventory`, - ); - exactArtifactNames( - ledgerInventory, - ["oliphaunt-bootstrap-ledger"], - `${context}.bootstrapLedger.artifactInventory`, - ); - terminalCheckpoint( - value.bootstrapLedger.terminalCheckpoint, - `${context}.bootstrapLedger.terminalCheckpoint`, - source, - envelope, - capsule, - ); - const runIds = [ value.payloadQualification.run.id, value.approvedDryRun.run.id, - value.bootstrapLedger.run.id, ]; + if (value.bootstrapLedger !== null) { + strictObject( + value.bootstrapLedger, + ["artifactInventory", "run", "terminalCheckpoint", "workflow"], + `${context}.bootstrapLedger`, + ); + workflow( + value.bootstrapLedger.workflow, + `${context}.bootstrapLedger.workflow`, + { name: "Release", path: ".github/workflows/release.yml" }, + ); + successfulRun(value.bootstrapLedger.run, `${context}.bootstrapLedger.run`, source); + const ledgerInventory = artifactInventory( + value.bootstrapLedger.artifactInventory, + `${context}.bootstrapLedger.artifactInventory`, + ); + exactArtifactNames( + ledgerInventory, + ["oliphaunt-bootstrap-ledger"], + `${context}.bootstrapLedger.artifactInventory`, + ); + terminalCheckpoint( + value.bootstrapLedger.terminalCheckpoint, + `${context}.bootstrapLedger.terminalCheckpoint`, + source, + envelope, + capsule, + ); + runIds.push(value.bootstrapLedger.run.id); + if (value.approvedDryRun.workflow.id !== value.bootstrapLedger.workflow.id) { + throw error(`${context} workflow identities are inconsistent`); + } + } if (new Set(runIds).size !== runIds.length) { - throw error(`${context} must bind three distinct workflow runs`); + throw error(`${context} must bind distinct workflow runs`); } - if ( - value.approvedDryRun.workflow.id !== value.bootstrapLedger.workflow.id - || value.payloadQualification.workflow.id === value.approvedDryRun.workflow.id - ) { + if (value.payloadQualification.workflow.id === value.approvedDryRun.workflow.id) { throw error(`${context} workflow identities are inconsistent`); } return value; } +export function sameVersionRecoverySourceProvenanceSchema(record) { + validateRecord(record, 0); + return Object.hasOwn(record, "recoveryBoundary") + ? SAME_VERSION_RECOVERY_SOURCES_SCHEMA + : LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA; +} + +export function isSameVersionRecoverySourcesDocument(value) { + return ( + value !== null + && !Array.isArray(value) + && typeof value === "object" + && new Set([ + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + ]).has(value.schema) + && Array.isArray(value.records) + ); +} + function gitCommitAndTree(repo, commit) { const result = captureCommandOutput( "git", @@ -550,9 +620,12 @@ export function validateSameVersionRecoverySources( { repo = ROOT, verifyGit = true } = {}, ) { strictObject(document, ["records", "schema"], "recovery source document"); - if (document.schema !== SAME_VERSION_RECOVERY_SOURCES_SCHEMA) { + if (!new Set([ + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + ]).has(document.schema)) { throw error( - `recovery source document schema must be ${SAME_VERSION_RECOVERY_SOURCES_SCHEMA}`, + "recovery source document schema must be a supported v1 or v2 schema", ); } if (!Array.isArray(document.records) || document.records.length === 0) { @@ -560,6 +633,12 @@ export function validateSameVersionRecoverySources( } document.records.forEach((record) => validateSameVersionRecoverySource(record, { repo, verifyGit: false })); + if ( + document.schema === LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA + && document.records.some((record) => Object.hasOwn(record, "recoveryBoundary")) + ) { + throw error("v1 recovery source documents cannot contain v2 recovery boundaries"); + } const sorted = [...document.records].sort((left, right) => compareText(left.releaseSource.commit, right.releaseSource.commit)); if (canonicalRecoverySourceJson(sorted) !== canonicalRecoverySourceJson(document.records)) { @@ -886,9 +965,17 @@ export function validateSameVersionRecoveryEvidence( { publicationLock, capsuleManifest, terminalLedger }, ) { validateRecord(record, 0); + if (record.bootstrapLedger !== null && terminalLedger === undefined) { + throw error("recovery record with bootstrap publication requires terminal ledger evidence"); + } const lock = readEvidenceFile(publicationLock, "publication lock evidence"); const capsule = readEvidenceFile(capsuleManifest, "capsule manifest evidence"); - const ledger = readEvidenceFile(terminalLedger, "terminal bootstrap ledger evidence"); + const ledger = record.bootstrapLedger === null + ? null + : readEvidenceFile(terminalLedger, "terminal bootstrap ledger evidence"); + if (record.bootstrapLedger === null && terminalLedger !== undefined) { + throw error("recovery record without bootstrap publication must not supply terminal ledger evidence"); + } verifyFileBytes( lock.bytes, record.releaseEnvelope.publicationLock, @@ -899,18 +986,20 @@ export function validateSameVersionRecoveryEvidence( record.approvedDryRun.capsuleManifest.file, "capsule manifest evidence", ); - verifyFileBytes( - ledger.bytes, - record.bootstrapLedger.terminalCheckpoint.file, - "terminal bootstrap ledger evidence", - ); validatePublicationLockEvidence(lock.value, record); validateCapsuleManifestEvidence(capsule.value, record); - validateTerminalLedgerEvidence(ledger.value, record); + if (ledger !== null) { + verifyFileBytes( + ledger.bytes, + record.bootstrapLedger.terminalCheckpoint.file, + "terminal bootstrap ledger evidence", + ); + validateTerminalLedgerEvidence(ledger.value, record); + } return { capsuleManifestSha256: sha256(capsule.bytes), publicationLockSha256: sha256(lock.bytes), - terminalLedgerSha256: sha256(ledger.bytes), + terminalLedgerSha256: ledger === null ? null : sha256(ledger.bytes), }; } @@ -920,6 +1009,8 @@ function outputLines(record) { .filter((artifact) => artifact.name === "oliphaunt-publication-lock"); const capsule = record.approvedDryRun.artifactInventory.artifacts .filter((artifact) => artifact.name === "oliphaunt-bootstrap-capsule"); + const bootstrapArtifacts = record.bootstrapLedger?.artifactInventory.artifacts ?? []; + const boundary = record.recoveryBoundary ?? { kind: "registry-partial" }; return { approved_capsule_artifact_metadata_json: artifactJson(capsule), approved_dry_run_artifact_metadata_json: artifactJson( @@ -928,9 +1019,12 @@ function outputLines(record) { approved_dry_run_id: String(record.approvedDryRun.run.id), approved_lock_artifact_metadata_json: artifactJson(lock), bootstrap_ledger_artifact_metadata_json: artifactJson( - record.bootstrapLedger.artifactInventory.artifacts, + bootstrapArtifacts, ), - bootstrap_ledger_run_id: String(record.bootstrapLedger.run.id), + bootstrap_ledger_required: String(record.bootstrapLedger !== null), + bootstrap_ledger_run_id: record.bootstrapLedger === null + ? "" + : String(record.bootstrapLedger.run.id), catalog_digest: record.releaseEnvelope.catalogDigest, lock_digest: record.releaseEnvelope.lockDigest, package_envelope_digest: record.releaseEnvelope.packageEnvelopeDigest, @@ -938,6 +1032,8 @@ function outputLines(record) { record.payloadQualification.artifactInventory.artifacts, ), payload_ci_run_id: String(record.payloadQualification.run.id), + recovery_boundary_json: canonicalRecoverySourceJson(boundary), + recovery_boundary_kind: boundary.kind, record_digest: digestValue(record), record_json: canonicalRecoverySourceJson(record), release_sha: record.releaseSource.commit, @@ -992,12 +1088,15 @@ function parseArgs(argv) { if (!SHA.test(releaseSha)) { throw error("usage: same-version-recovery-source.mjs --release-sha SHA [--record-file FILE] " + "[--github-output FILE] [--publication-lock FILE --capsule-manifest FILE " - + "--terminal-ledger FILE]"); + + "[--terminal-ledger FILE]]"); + } + const coreEvidenceNames = ["publication-lock", "capsule-manifest"]; + const coreEvidenceCount = coreEvidenceNames.filter((name) => values.has(name)).length; + if (![0, coreEvidenceNames.length].includes(coreEvidenceCount)) { + throw error("publication-lock and capsule-manifest must be supplied together"); } - const evidenceNames = ["publication-lock", "capsule-manifest", "terminal-ledger"]; - const evidenceCount = evidenceNames.filter((name) => values.has(name)).length; - if (![0, evidenceNames.length].includes(evidenceCount)) { - throw error("publication-lock, capsule-manifest, and terminal-ledger must be supplied together"); + if (values.has("terminal-ledger") && coreEvidenceCount === 0) { + throw error("terminal-ledger requires publication-lock and capsule-manifest"); } return { capsuleManifest: values.get("capsule-manifest"), diff --git a/tools/release/same-version-recovery-source.test.mjs b/tools/release/same-version-recovery-source.test.mjs index 8e116d54..d121e5a8 100644 --- a/tools/release/same-version-recovery-source.test.mjs +++ b/tools/release/same-version-recovery-source.test.mjs @@ -16,10 +16,12 @@ import test from "node:test"; import { execFileSync } from "../test/fd-backed-spawn-sync.mjs"; import { DEFAULT_SAME_VERSION_RECOVERY_SOURCES, + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, SAME_VERSION_RECOVERY_SOURCES_SCHEMA, appendSameVersionRecoverySourceGitHubOutput, canonicalRecoverySourceJson, loadSameVersionRecoverySources, + sameVersionRecoverySourceProvenanceSchema, selectSameVersionRecoverySource, validateSameVersionRecoveryEvidence, validateSameVersionRecoverySource, @@ -28,6 +30,7 @@ import { const RELEASE_SHA = "9c398f4e5c05f494f9b752a8634e74e0bc11dd19"; const RELEASE_TREE = "396cf3b10adb1a5b625e66c5ebacf8c3d364b543"; +const GITHUB_STAGED_RELEASE_SHA = "ae3d29ba16245e9345a8d337cd17c53f9bf2e853"; const TOOL = path.join(import.meta.dir, "same-version-recovery-source.mjs"); function sha256(bytes) { @@ -309,6 +312,10 @@ function evidenceFixture({ test("committed record selects the exact original release and complete frozen inventories", () => { const sources = loadSameVersionRecoverySources(); const selected = selectSameVersionRecoverySource(sources, RELEASE_SHA); + const githubStaged = selectSameVersionRecoverySource( + sources, + GITHUB_STAGED_RELEASE_SHA, + ); assert.equal(sources.schema, SAME_VERSION_RECOVERY_SOURCES_SCHEMA); assert.deepEqual(selected.releaseSource, { commit: RELEASE_SHA, @@ -331,6 +338,15 @@ test("committed record selects the exact original release and complete frozen in ], ); assert.equal(selected.bootstrapLedger.run.id, 30548314727); + assert.equal(Object.hasOwn(selected, "recoveryBoundary"), false); + assert.equal( + sameVersionRecoverySourceProvenanceSchema(selected), + LEGACY_SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + ); + assert.equal( + digestValue(selected), + "65a998a7af8be6fb043223e95fc2cf50e92ce9659fe6b1e55533e6b0525f34b9", + ); assert.equal( selected.bootstrapLedger.artifactInventory.artifacts[0].id, 8761717044, @@ -339,6 +355,12 @@ test("committed record selects the exact original release and complete frozen in selected.releaseEnvelope.lockDigest, "5ee675ab3066cca7df21dd425a5c80fd6c9b9c4b276757fc1aa84e2020761266", ); + assert.equal( + sameVersionRecoverySourceProvenanceSchema(githubStaged), + SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + ); + assert.equal(githubStaged.recoveryBoundary.kind, "github-staged"); + assert.equal(githubStaged.bootstrapLedger, null); }); test("CLI and GITHUB_OUTPUT emit the same canonical selected record and exact metadata", () => { @@ -352,6 +374,8 @@ test("CLI and GITHUB_OUTPUT emit the same canonical selected record and exact me assert.equal(lines.release_sha, RELEASE_SHA); assert.equal(lines.release_tree, RELEASE_TREE); assert.equal(lines.payload_ci_run_id, "30358387218"); + assert.equal(lines.bootstrap_ledger_required, "true"); + assert.equal(lines.recovery_boundary_kind, "registry-partial"); assert.equal( JSON.parse(lines.payload_ci_artifact_metadata_json).length, 73, @@ -377,6 +401,64 @@ test("CLI and GITHUB_OUTPUT emit the same canonical selected record and exact me } }); +test("GitHub-staged recovery boundaries bind one exact failed staging run", () => { + const selected = record(); + selected.recoveryBoundary = { + evidenceArtifact: { + digest: `sha256:${"a".repeat(64)}`, + id: 123, + name: `github-staging-recovery-${RELEASE_SHA}-456-1`, + size: 789, + }, + job: { + conclusion: "failure", + id: 321, + name: "Prepare and stage release", + }, + kind: "github-staged", + run: { + attempt: 1, + conclusion: "failure", + event: "workflow_dispatch", + headSha: RELEASE_SHA, + id: 456, + status: "completed", + }, + }; + assert.equal( + validateSameVersionRecoverySource(selected, { verifyGit: false }), + selected, + ); + selected.recoveryBoundary.run.headSha = "f".repeat(40); + assert.throws( + () => validateSameVersionRecoverySource(selected, { verifyGit: false }), + /source-bound workflow_dispatch/u, + ); +}); + +test("a post-bootstrap release may explicitly bind no bootstrap ledger", () => { + const selected = record(); + selected.recoveryBoundary = { kind: "registry-partial" }; + selected.bootstrapLedger = null; + assert.equal( + validateSameVersionRecoverySource(selected, { verifyGit: false }), + selected, + ); + + const root = mkdtempSync(path.join(tmpdir(), "oliphaunt-recovery-no-ledger-")); + try { + const lines = appendSameVersionRecoverySourceGitHubOutput( + path.join(root, "github-output"), + selected, + ); + assert.equal(lines.bootstrap_ledger_required, "false"); + assert.equal(lines.bootstrap_ledger_run_id, ""); + assert.deepEqual(JSON.parse(lines.bootstrap_ledger_artifact_metadata_json), []); + } finally { + rmSync(root, { force: true, recursive: true }); + } +}); + test("strict document and inventory mutations fail closed", () => { const cases = [ { @@ -421,7 +503,7 @@ test("strict document and inventory mutations fail closed", () => { }, { mutate: (value) => { - value.records.push(structuredClone(value.records[0])); + value.records.splice(1, 0, structuredClone(value.records[0])); }, pattern: /duplicate recovery source commit/u, }, @@ -497,6 +579,33 @@ test("downloaded lock, capsule manifest, and terminal ledger verify as one envel } }); +test("no-bootstrap recovery verifies the lock and capsule without invented ledger evidence", () => { + const fixture = evidenceFixture(); + try { + fixture.record.recoveryBoundary = { kind: "registry-partial" }; + fixture.record.bootstrapLedger = null; + assert.deepEqual( + validateSameVersionRecoveryEvidence(fixture.record, { + capsuleManifest: fixture.capsuleManifest, + publicationLock: fixture.publicationLock, + }), + { + capsuleManifestSha256: + fixture.record.approvedDryRun.capsuleManifest.file.sha256, + publicationLockSha256: + fixture.record.releaseEnvelope.publicationLock.sha256, + terminalLedgerSha256: null, + }, + ); + assert.throws( + () => validateSameVersionRecoveryEvidence(fixture.record, fixture), + /must not supply terminal ledger evidence/u, + ); + } finally { + fixture.cleanup(); + } +}); + test("raw evidence substitution and internally forged digests fail closed", () => { { const fixture = evidenceFixture(); diff --git a/tools/release/same-version-recovery-sources.json b/tools/release/same-version-recovery-sources.json index d20ecdc5..f723b80c 100644 --- a/tools/release/same-version-recovery-sources.json +++ b/tools/release/same-version-recovery-sources.json @@ -616,7 +616,597 @@ "commit": "9c398f4e5c05f494f9b752a8634e74e0bc11dd19", "tree": "396cf3b10adb1a5b625e66c5ebacf8c3d364b543" } + }, + { + "approvedDryRun": { + "artifactInventory": { + "artifacts": [ + { + "digest": "sha256:9ebe20ed1356f4fa2ee42a91da1bb196d13dd31cad0e495ac743a5c5ad6fec5d", + "id": 9028508511, + "name": "oliphaunt-bootstrap-capsule", + "size": 412670716 + }, + { + "digest": "sha256:ad3c76d46ea575b25df10b3c985d3479c414eab2c3b80caf9c79ccf2109a823e", + "id": 9028505648, + "name": "oliphaunt-publication-lock", + "size": 41296 + } + ], + "count": 2, + "inventoryDigest": "96cd7d4e6b5430b3e353797d95a7e69548f55fed5bac7157d4b33b9c82bc790d", + "totalSize": 412712012 + }, + "capsuleManifest": { + "carrierCount": 279, + "catalogDigest": "fed90590c56d9c9f0c9c2065fd4af73c178d6fb5decc58770e5eda068e4fe434", + "file": { + "path": "target/release/bootstrap-capsule-manifest.json", + "sha256": "e8835d5064c8cb451f3937d50a7e0464e2ceb833b982de2a435c497e744bfc8d", + "size": 182232 + }, + "lockDigest": "d1a9f799c1fd40582e7a824ccc6ec6650cba55b8a95592d3d2f626ba33cd6188", + "packageEnvelopeDigest": "557d1fd630de735458d748cad5085dc4f56868c307378f84f442e8497b2210e2", + "productCount": 18, + "publicationLock": { + "path": "target/release/publication-lock.json", + "sha256": "4eabf3e328cde05a05a1da0165f187137df2f6bf8fe38355eca385a3e18358c0", + "size": 438770 + }, + "schema": "oliphaunt-bootstrap-publication-capsule-v1", + "source": { + "commit": "ae3d29ba16245e9345a8d337cd17c53f9bf2e853", + "tree": "673e8f249d2f51d10997f0036a7e471bf35a388e" + } + }, + "run": { + "attempt": 1, + "conclusion": "success", + "event": "workflow_dispatch", + "headSha": "ae3d29ba16245e9345a8d337cd17c53f9bf2e853", + "id": 31280255709, + "status": "completed" + }, + "workflow": { + "id": 265799825, + "name": "Release", + "path": ".github/workflows/release.yml" + } + }, + "bootstrapLedger": null, + "payloadQualification": { + "artifactInventory": { + "artifacts": [ + { + "digest": "sha256:cfa533f26b26572a611be2b2af5f733675725e4d9fccf9d8627528b8bb2e53aa", + "id": 9027073781, + "name": "artifact-build-plan", + "size": 2798 + }, + { + "digest": "sha256:57acd4ae884552a89795e442cdb977f9c07db692178d78b9896fa7c01b92a801", + "id": 9027495034, + "name": "f0rr0~oliphaunt~O4355S.dockerbuild", + "size": 33844 + }, + { + "digest": "sha256:858344114ad913b7dd834f189b123bbefe1de095f0053bf05ff8af57a6bc72f7", + "id": 9027607557, + "name": "liboliphaunt-native-extension-artifacts-android-arm64-v8a", + "size": 68651629 + }, + { + "digest": "sha256:c7ccac7aed3d1621d326d7a8b4457dfbe7a5ad0df0c8690cc6acf2ddd8c6d753", + "id": 9027607218, + "name": "liboliphaunt-native-extension-artifacts-android-x86_64", + "size": 26072795 + }, + { + "digest": "sha256:4d039e0e1d727666696dd6ac2162e158816b8a626e9af9645fb47deba9ad5f9c", + "id": 9027866887, + "name": "liboliphaunt-native-extension-artifacts-ios-xcframework", + "size": 74766558 + }, + { + "digest": "sha256:a3cfc7b5d7ad39180e006d9dc5bc185de09edccbae131069dfa29da6d1e825cb", + "id": 9027316701, + "name": "liboliphaunt-native-extension-artifacts-linux-arm64-gnu", + "size": 18126523 + }, + { + "digest": "sha256:a10d7bde22845955a5a4440fc2269082a6fdc7654db98540d4448a0758850903", + "id": 9027396428, + "name": "liboliphaunt-native-extension-artifacts-linux-x64-gnu", + "size": 18701273 + }, + { + "digest": "sha256:b6fa45d7436a30ff0298d576e08ac8383925760c2649eae2dbeea5de0b6fc491", + "id": 9027339964, + "name": "liboliphaunt-native-extension-artifacts-macos-arm64", + "size": 21970749 + }, + { + "digest": "sha256:088aac05f529e0137b0a7035751ba884325de7a5d49a539d0827913b3665d5f1", + "id": 9027435877, + "name": "liboliphaunt-native-extension-artifacts-windows-x64-msvc", + "size": 18755302 + }, + { + "digest": "sha256:9c1ed69dcc5d9666cd16fda76b82206c0eb9f6aa83a1a16328e71e08ec27c09b", + "id": 9027279468, + "name": "liboliphaunt-native-icu-data", + "size": 13193248 + }, + { + "digest": "sha256:a524f51304ca8cb487dc97c4c4a16ad6a19c46bbb188a0df0cf1e823a93f5bcd", + "id": 9027383831, + "name": "liboliphaunt-native-release-assets", + "size": 119131018 + }, + { + "digest": "sha256:79378328f4b149104490642603a78f452f800318dccf74059fc08f6e0940930c", + "id": 9027232198, + "name": "liboliphaunt-native-release-assets-android-arm64-v8a", + "size": 5394166 + }, + { + "digest": "sha256:cb0c7d96b41d6e7895c90a6b7f431a20f0ea1f7d38b0599500bbb8026b4c97eb", + "id": 9027259074, + "name": "liboliphaunt-native-release-assets-android-x86_64", + "size": 5606334 + }, + { + "digest": "sha256:0e3861f6c86e82547a6f35a1ef4851730b49fa48586a794d515382bad85e395e", + "id": 9027348143, + "name": "liboliphaunt-native-release-assets-ios-xcframework", + "size": 54961005 + }, + { + "digest": "sha256:fc3c415d79436af69df69003c1a4493459f0ef9ab7da3de35d1c7136e000a8fe", + "id": 9027199592, + "name": "liboliphaunt-native-release-assets-linux-arm64-gnu", + "size": 14031656 + }, + { + "digest": "sha256:0fc38ec2ffe2883404254ce9f02879f5559da8f89e2f0eb8aa9b0151ffdea8f8", + "id": 9027284300, + "name": "liboliphaunt-native-release-assets-linux-x64-gnu", + "size": 14423452 + }, + { + "digest": "sha256:837230154224d9c6d64d168bf70832f4f8bd9f1529ac9a1b1c12277b1bc2722e", + "id": 9027279046, + "name": "liboliphaunt-native-release-assets-macos-arm64", + "size": 12713086 + }, + { + "digest": "sha256:5b60258fa6dc2143dcf0f3c0be3011043382be360fe3f8c5dcdef2a6f23d348b", + "id": 9027248087, + "name": "liboliphaunt-native-release-assets-windows-x64-msvc", + "size": 12000532 + }, + { + "digest": "sha256:28450da3edb99609522cadbc80f41933f1c8cebf1e0394559346b7ea3624ac57", + "id": 9027233263, + "name": "liboliphaunt-native-target-android-arm64-v8a", + "size": 35728903 + }, + { + "digest": "sha256:65fc842c5aa02e9cc02e9d8ebe78ba6227f60dc16212c6d1de5458e6aba1a8dc", + "id": 9027260333, + "name": "liboliphaunt-native-target-android-x86_64", + "size": 35830531 + }, + { + "digest": "sha256:61e1d6375e3f72fb188049918ff792bd6ccbeb411e426b48d7909f80c5652c21", + "id": 9027349013, + "name": "liboliphaunt-native-target-ios-xcframework", + "size": 38077905 + }, + { + "digest": "sha256:6a9fc68c6d607d35d6fb4166d4a284ecdb60ff89dcb62b848d06e26b4a757048", + "id": 9027626040, + "name": "liboliphaunt-wasix-extension-aot-linux-arm64-gnu", + "size": 13440244 + }, + { + "digest": "sha256:ce0d4c635e4797f79d1bb1102546cabb88194d18ab5bc4af2ea19b17605f4150", + "id": 9027644298, + "name": "liboliphaunt-wasix-extension-aot-linux-x64-gnu", + "size": 13833674 + }, + { + "digest": "sha256:d20de3d9f3210208c2b2b7c8b83f5651ea54cfc78904fdaa3654781ec30ef878", + "id": 9027640763, + "name": "liboliphaunt-wasix-extension-aot-macos-arm64", + "size": 12844191 + }, + { + "digest": "sha256:16407fc539993c8a5802251fefdaef6b3b492d78ff3b3af9eea88641196beccc", + "id": 9027750023, + "name": "liboliphaunt-wasix-extension-aot-windows-x64-msvc", + "size": 13519489 + }, + { + "digest": "sha256:7a9f5ed3b05c7149786e775acefaa5fe44a8d2d74bb1a83906e5e7825ddaaf91", + "id": 9027509012, + "name": "liboliphaunt-wasix-extension-artifacts-wasix-portable", + "size": 9198736 + }, + { + "digest": "sha256:a38b1c4350ba347720d6e31eeba45e21d8e62358d5f39fa03823ee9a8d6507a0", + "id": 9027784450, + "name": "liboliphaunt-wasix-release-assets", + "size": 54106879 + }, + { + "digest": "sha256:1da6369a676694568b8e4b7ac78e5199c1462b94a5c72cfd242f7f8d6c1cc31e", + "id": 9027625842, + "name": "liboliphaunt-wasix-runtime-aot-linux-arm64-gnu", + "size": 9072980 + }, + { + "digest": "sha256:074defa17a52387d7d7514119a21d4d13c40bf106b1296c93f16c4ffc5bf43a0", + "id": 9027643784, + "name": "liboliphaunt-wasix-runtime-aot-linux-x64-gnu", + "size": 9400879 + }, + { + "digest": "sha256:13dcf3c864ff537b2141ee10012098c422081ddeb692b6a8dc68e93aafbfed96", + "id": 9027640285, + "name": "liboliphaunt-wasix-runtime-aot-macos-arm64", + "size": 8613639 + }, + { + "digest": "sha256:0ccc0d3ece0506e9a33ed0a63603e94f6aa2ffbc7d284b5ab62fe5d499978330", + "id": 9027749486, + "name": "liboliphaunt-wasix-runtime-aot-windows-x64-msvc", + "size": 9118767 + }, + { + "digest": "sha256:99b2c428135bfc605e5140a1c837a41ef9fa8720c2548810374d8d1a2bb38e40", + "id": 9027494726, + "name": "liboliphaunt-wasix-runtime-portable", + "size": 58976692 + }, + { + "digest": "sha256:d4eb5161e0dd5e03a3a22f4e1a7357d6cc8b0a36d130fc99b2f4cd75b1570d41", + "id": 9027878101, + "name": "native-extension-lifecycle-evidence", + "size": 36604 + }, + { + "digest": "sha256:e7d3169c8cdb5f04c0952a42d4768c48f933a4199240028372587a17962c41a7", + "id": 9027876106, + "name": "native-extension-lifecycle-evidence-0", + "size": 10607 + }, + { + "digest": "sha256:1cbcad39d23dda1c4c38cec6421d868f29402023da9c483d256424056984ed0d", + "id": 9027875617, + "name": "native-extension-lifecycle-evidence-1", + "size": 10564 + }, + { + "digest": "sha256:52536aa86b9b414235958f3fb7f205ae3a744a8539c0fdf64ebb71d0d3cf98a5", + "id": 9027876028, + "name": "native-extension-lifecycle-evidence-2", + "size": 10939 + }, + { + "digest": "sha256:d8d36cfd36fdc67d3f7b408a45aed3992dbcb4bec9bee864f776fe2a9bcbd380", + "id": 9027216373, + "name": "oliphaunt-broker-release-assets-linux-arm64-gnu", + "size": 484692 + }, + { + "digest": "sha256:91a990567e980b179fe5d044e2f9b9ac92804d329a1620fafa1c0bf6f4bd62c4", + "id": 9027112105, + "name": "oliphaunt-broker-release-assets-linux-x64-gnu", + "size": 538537 + }, + { + "digest": "sha256:4fbab450327c41d0d52702ae16a7d765011cbb6890b6fe3a1712d84e3dbcd8f0", + "id": 9027138657, + "name": "oliphaunt-broker-release-assets-macos-arm64", + "size": 468721 + }, + { + "digest": "sha256:a611d2f8f20763e43de841246447b4b44e84f993dfc1c97db4bef0c90dd081f2", + "id": 9027139737, + "name": "oliphaunt-broker-release-assets-windows-x64-msvc", + "size": 619720 + }, + { + "digest": "sha256:12c472627deb02bc1a27ca0048ef953d5d9270e4929877526e54eba4ad481e26", + "id": 9027990179, + "name": "oliphaunt-extension-package-artifacts", + "size": 351204298 + }, + { + "digest": "sha256:db0561a439c320c1209c90ac0241e7ac2ef3cc8cfa7a58c280f19cf16878e8b8", + "id": 9027907066, + "name": "oliphaunt-js-exact-candidate-consumer-linux-arm64-gnu", + "size": 76886 + }, + { + "digest": "sha256:b13284b98b623623c0a08bdeb137995740400e1b548206129c9ddb62b590523e", + "id": 9027914915, + "name": "oliphaunt-js-exact-candidate-consumer-linux-x64-gnu", + "size": 77354 + }, + { + "digest": "sha256:475b2b12632f849a871f20813ca1c074ece4e53086c3a2516521dd2ce77432f0", + "id": 9027916083, + "name": "oliphaunt-js-exact-candidate-consumer-macos-arm64", + "size": 76894 + }, + { + "digest": "sha256:4d65dfc969940a68366465f0c3a8a39d57d8b773a70d3b5f34bf89b352215478", + "id": 9027932263, + "name": "oliphaunt-js-exact-candidate-consumer-windows-x64-msvc", + "size": 84318 + }, + { + "digest": "sha256:c9a4bf737647afbc302a1e904588c702077f0de8def8aebc600f117d2f8335e1", + "id": 9027097661, + "name": "oliphaunt-js-sdk-package-artifacts", + "size": 304934 + }, + { + "digest": "sha256:50a0501172b7af2523253b7e6e80a8fa963e1ad5abe9df81d56779d4c17ab9eb", + "id": 9027189161, + "name": "oliphaunt-kotlin-sdk-package-artifacts", + "size": 1887677 + }, + { + "digest": "sha256:92805fb2addcbecc6deb18811b63e6be6f1169a14fbc1e3f0aa33ee87892a27e", + "id": 9027889712, + "name": "oliphaunt-mobile-extension-package-artifacts", + "size": 198843492 + }, + { + "digest": "sha256:d0a40a4f82abfee3b3f5b5b33f871ee45eae44557c074e79f1569d0011041155", + "id": 9027133775, + "name": "oliphaunt-native-extension-proof-linux-x64-gnu", + "size": 637912 + }, + { + "digest": "sha256:4a38c7c13ece645e9a584ca2fccba58e4e83db580823f71d38ee51157a52396a", + "id": 9027107465, + "name": "oliphaunt-node-direct-npm-package-linux-arm64-gnu", + "size": 24906 + }, + { + "digest": "sha256:b728531906d672e54cb0aa7c231ccdd636a280241bbd2a2d73411684c65ce2c6", + "id": 9027092435, + "name": "oliphaunt-node-direct-npm-package-linux-x64-gnu", + "size": 26171 + }, + { + "digest": "sha256:ba7571102c09f16bb449e17f53c794259390b4193149951a59a4317c0ef47aa6", + "id": 9027157694, + "name": "oliphaunt-node-direct-npm-package-macos-arm64", + "size": 19570 + }, + { + "digest": "sha256:93753f4302f97fd73a2d6610410dea5176bc63c4ce4abb581c9a5cf71ce2408a", + "id": 9027127071, + "name": "oliphaunt-node-direct-npm-package-windows-x64-msvc", + "size": 84949 + }, + { + "digest": "sha256:cf8dcdf0dd48185a618e3e123edd1f03a736736ed969bf0f005ccb0b34865d5c", + "id": 9027107329, + "name": "oliphaunt-node-direct-release-assets-linux-arm64-gnu", + "size": 24720 + }, + { + "digest": "sha256:98e3e1cee2c22cc89ffa0c42189b31a2654f64c587ce4ff1ed51d0924a3d494b", + "id": 9027092219, + "name": "oliphaunt-node-direct-release-assets-linux-x64-gnu", + "size": 26155 + }, + { + "digest": "sha256:33f21691b1cc672b6fa78334c9980de2545b1a1f7602a88a1437484f3c4b48ea", + "id": 9027157520, + "name": "oliphaunt-node-direct-release-assets-macos-arm64", + "size": 19495 + }, + { + "digest": "sha256:c9eca37334c5cbcf5d35de8b48f831e7d07c92dbafc7fa47e75fc8e8e7be242f", + "id": 9027126971, + "name": "oliphaunt-node-direct-release-assets-windows-x64-msvc", + "size": 82158 + }, + { + "digest": "sha256:1134350898f36d23b4a8d82ef5ae00b6276fce71a690d3bbad95bc2267fd2d05", + "id": 9028078550, + "name": "oliphaunt-react-native-ios-carriers", + "size": 123760 + }, + { + "digest": "sha256:340233b3392bf01077090b5a6de3b3baa11aa39735060967ca2a91c23fc8f3aa", + "id": 9027379111, + "name": "oliphaunt-react-native-sdk-package-artifacts", + "size": 147092 + }, + { + "digest": "sha256:883f3bff49d134c73fed287725eebd2bd3c11b589a82c268c83c4efbb44fc8ee", + "id": 9028188528, + "name": "oliphaunt-release-candidate", + "size": 1162 + }, + { + "digest": "sha256:00b675489b5a9a85eefb3d4cb87c2bae160f7f910cb7e5bdd37135781fee9e10", + "id": 9027881425, + "name": "oliphaunt-rust-exact-candidate-consumer-evidence", + "size": 5027 + }, + { + "digest": "sha256:4dd870669ce8738109183269c98c99379b825fac804776d272e7ab8baf35ee5d", + "id": 9027126015, + "name": "oliphaunt-rust-sdk-package-artifacts", + "size": 258796 + }, + { + "digest": "sha256:dd310e89bc87d661dd3387ae23fc29e90cf5ee32c8a6933a003fc8f87fc769dd", + "id": 9027425547, + "name": "oliphaunt-swift-exact-candidate-consumer-evidence", + "size": 136531 + }, + { + "digest": "sha256:5138ff0f726c5a780cc706544c83916d014e3f44e6bb3840b8d71744e41d5bb7", + "id": 9027426456, + "name": "oliphaunt-swift-sdk-package-artifacts", + "size": 15169863 + }, + { + "digest": "sha256:cc3a660cf309f0b83ad8d02b2d47b0cbc37eaffc34c5caa21be361884f3cf25d", + "id": 9027803798, + "name": "oliphaunt-wasix-rust-exact-candidate-consumer-evidence", + "size": 34353 + }, + { + "digest": "sha256:66090fc1911885e93509fe665091c60f70d866688e9828bc2f3f5c2c08c8469e", + "id": 9027226605, + "name": "oliphaunt-wasix-rust-package-artifacts", + "size": 369388 + }, + { + "digest": "sha256:28d2e1bdd15bcad204167029978ac8f639fedc122e6ef1bf20824c7ea1d598ee", + "id": 9028076300, + "name": "react-native-mobile-android-app-android-arm64-v8a", + "size": 72065170 + }, + { + "digest": "sha256:057e12cd775a5e49521761163317db79cb17eb348d9d891babffe6d0be5f998c", + "id": 9028060818, + "name": "react-native-mobile-android-app-android-x86_64", + "size": 72631649 + }, + { + "digest": "sha256:5d65ab02bfb4cc4016db9607fb5041415c896f0693fbf51c8d3fcb55852d0ef8", + "id": 9028105861, + "name": "react-native-mobile-android-e2e-reports", + "size": 7301 + }, + { + "digest": "sha256:803c8bfd75d8ff7f083de742619a547a16232650562e4c1a099ce5c58189affd", + "id": 9028078449, + "name": "react-native-mobile-ios-app", + "size": 55176454 + }, + { + "digest": "sha256:347c3417ff6fdae10c54bc2f990c79e881f8ecbb10368cc63997340ec89bf0f7", + "id": 9028078064, + "name": "react-native-mobile-ios-build-logs", + "size": 835225 + }, + { + "digest": "sha256:9b2de11a4ccd08c2bf8fb7246760ae3a60c64ee3324d82fdae984c2b39e049ae", + "id": 9028179460, + "name": "react-native-mobile-ios-e2e-reports", + "size": 40820 + }, + { + "digest": "sha256:f6adbac6fd94607c4b086d336e90799e343a9ef4c92cd5f96ab6fa4f46c3829e", + "id": 9027847714, + "name": "wasix-release-regression-evidence", + "size": 13686 + } + ], + "count": 73, + "inventoryDigest": "694a130fc995edc5f4d5c785d19a23d564690f1a34954b1d31348cce4478e9df", + "totalSize": 1592998027 + }, + "requiredArtifactNames": [ + "artifact-build-plan", + "liboliphaunt-native-release-assets", + "liboliphaunt-wasix-release-assets", + "liboliphaunt-wasix-runtime-aot-linux-arm64-gnu", + "liboliphaunt-wasix-runtime-aot-linux-x64-gnu", + "liboliphaunt-wasix-runtime-aot-macos-arm64", + "liboliphaunt-wasix-runtime-aot-windows-x64-msvc", + "liboliphaunt-wasix-runtime-portable", + "oliphaunt-broker-release-assets-linux-arm64-gnu", + "oliphaunt-broker-release-assets-linux-x64-gnu", + "oliphaunt-broker-release-assets-macos-arm64", + "oliphaunt-broker-release-assets-windows-x64-msvc", + "oliphaunt-extension-package-artifacts", + "oliphaunt-js-sdk-package-artifacts", + "oliphaunt-kotlin-sdk-package-artifacts", + "oliphaunt-node-direct-npm-package-linux-arm64-gnu", + "oliphaunt-node-direct-npm-package-linux-x64-gnu", + "oliphaunt-node-direct-npm-package-macos-arm64", + "oliphaunt-node-direct-npm-package-windows-x64-msvc", + "oliphaunt-node-direct-release-assets-linux-arm64-gnu", + "oliphaunt-node-direct-release-assets-linux-x64-gnu", + "oliphaunt-node-direct-release-assets-macos-arm64", + "oliphaunt-node-direct-release-assets-windows-x64-msvc", + "oliphaunt-react-native-sdk-package-artifacts", + "oliphaunt-release-candidate", + "oliphaunt-rust-sdk-package-artifacts", + "oliphaunt-swift-sdk-package-artifacts", + "oliphaunt-wasix-rust-package-artifacts", + "wasix-release-regression-evidence" + ], + "run": { + "attempt": 1, + "conclusion": "success", + "event": "workflow_dispatch", + "headSha": "ae3d29ba16245e9345a8d337cd17c53f9bf2e853", + "id": 31276212829, + "status": "completed" + }, + "workflow": { + "id": 265767525, + "name": "CI", + "path": ".github/workflows/ci.yml" + } + }, + "recoveryBoundary": { + "evidenceArtifact": { + "digest": "sha256:0f024d7d41ef105ef91a24dccc9048e2b6834aacf005f652e6c53bbb54fc7eef", + "id": 9029551798, + "name": "github-staging-recovery-ae3d29ba16245e9345a8d337cd17c53f9bf2e853-31281649203-1", + "size": 56669 + }, + "job": { + "conclusion": "failure", + "id": 93163883359, + "name": "Prepare and stage release" + }, + "kind": "github-staged", + "run": { + "attempt": 1, + "conclusion": "failure", + "event": "workflow_dispatch", + "headSha": "ae3d29ba16245e9345a8d337cd17c53f9bf2e853", + "id": 31281649203, + "status": "completed" + } + }, + "releaseEnvelope": { + "carrierCount": 303, + "catalogDigest": "fed90590c56d9c9f0c9c2065fd4af73c178d6fb5decc58770e5eda068e4fe434", + "lockDigest": "d1a9f799c1fd40582e7a824ccc6ec6650cba55b8a95592d3d2f626ba33cd6188", + "packageEnvelopeDigest": "557d1fd630de735458d748cad5085dc4f56868c307378f84f442e8497b2210e2", + "productArtifactCount": 152, + "productCount": 18, + "publicationLock": { + "path": "target/release/publication-lock.json", + "sha256": "4eabf3e328cde05a05a1da0165f187137df2f6bf8fe38355eca385a3e18358c0", + "size": 438770 + }, + "schema": "oliphaunt-publication-lock-v1" + }, + "releaseSource": { + "commit": "ae3d29ba16245e9345a8d337cd17c53f9bf2e853", + "tree": "673e8f249d2f51d10997f0036a7e471bf35a388e" + } } ], - "schema": "oliphaunt-same-version-recovery-sources-v1" + "schema": "oliphaunt-same-version-recovery-sources-v2" } diff --git a/tools/release/verify-github-release-attestation-receipt.test.mjs b/tools/release/verify-github-release-attestation-receipt.test.mjs index 941b1239..c32ac24e 100644 --- a/tools/release/verify-github-release-attestation-receipt.test.mjs +++ b/tools/release/verify-github-release-attestation-receipt.test.mjs @@ -7,6 +7,7 @@ import { afterAll, describe, expect, test } from "bun:test"; import { assertAttestationSubjectCoverage, + assertGhVerifiedBundleMatchesSupplied, assertGithubReleaseSnapshotMatchesReceipt, buildGithubAttestationReceipt, frozenGithubReleaseAssets, @@ -914,6 +915,37 @@ describe("GitHub release attestation receipt", () => { ).toEqual(["--source-digest", controller]); }); + test("accepts only gh's known empty RFC3161 protobuf canonicalization", () => { + const supplied = bundleFor(receiptSubjects()[0].subjects); + supplied.verificationMaterial = { + certificate: { rawBytes: "certificate" }, + timestampVerificationData: { rfc3161Timestamps: [] }, + tlogEntries: [{ logIndex: "1" }], + }; + const verified = structuredClone(supplied); + verified.verificationMaterial.timestampVerificationData = {}; + + expect(() => assertGhVerifiedBundleMatchesSupplied(verified, supplied)).not.toThrow(); + + const changedEnvelope = structuredClone(verified); + changedEnvelope.dsseEnvelope.signatures[0].sig = "different"; + expect(() => assertGhVerifiedBundleMatchesSupplied(changedEnvelope, supplied)).toThrow( + "does not contain the supplied bundle", + ); + + const changedCertificate = structuredClone(verified); + changedCertificate.verificationMaterial.certificate.rawBytes = "different"; + expect(() => assertGhVerifiedBundleMatchesSupplied(changedCertificate, supplied)).toThrow( + "does not contain the supplied bundle", + ); + + const unexpectedCanonicalization = structuredClone(verified); + unexpectedCanonicalization.verificationMaterial.tlogEntries = []; + expect(() => assertGhVerifiedBundleMatchesSupplied(unexpectedCanonicalization, supplied)).toThrow( + "does not contain the supplied bundle", + ); + }); + test("publishes receipt files atomically, cleans interrupted temps, and permits only identical reruns", async () => { const root = await fs.mkdtemp(path.join(process.cwd(), "target/github-receipt-write-test.")); fixtureRoots.push(root); diff --git a/tools/release/verify-github-staged-recovery-boundary.mjs b/tools/release/verify-github-staged-recovery-boundary.mjs new file mode 100644 index 00000000..42917ad8 --- /dev/null +++ b/tools/release/verify-github-staged-recovery-boundary.mjs @@ -0,0 +1,630 @@ +#!/usr/bin/env bun + +import { createHash } from "node:crypto"; +import { + lstatSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import process from "node:process"; +import { fileURLToPath } from "node:url"; + +import { captureCommandBytes, captureCommandOutput } from "../dev/capture-command-output.mjs"; +import { + runGitHubPaginatedJsonSync, + runGitHubReadSync, +} from "./github-read.mjs"; +import { validatePublicationLock } from "./publication-lock.mjs"; + +const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../.."); +const PREFIX = "verify-github-staged-recovery-boundary"; +const MAX_ARTIFACT_BYTES = 2 * 1024 * 1024; +const MAX_JSON_BYTES = 16 * 1024 * 1024; +const RELEASE_WORKFLOW_PATH = ".github/workflows/release.yml"; +const RELEASE_WORKFLOW_NAME = "Release / publish / main"; +const STAGING_JOB_NAME = "Prepare and stage release"; +const FAILING_STEP_NAME = "Freeze exact GitHub release asset and attestation evidence"; +const LOCK_MEMBER = "oliphaunt/oliphaunt/target/release/publication-lock.json"; +const UTF8 = new TextDecoder("utf-8", { fatal: true }); + +export const EXPECTED_RECOVERY_ARCHIVE_MEMBERS = Object.freeze([ + "_temp/oliphaunt-github-content-write-pacer.json", + "_temp/oliphaunt-github-core-request-journal.json", + "_temp/oliphaunt-github-release-asset-upload-report.json", + "oliphaunt/oliphaunt/target/release/normal-publication-plan.json", + LOCK_MEMBER, +].sort(compareText)); + +export const EXPECTED_RELEASE_JOB_OUTCOMES = Object.freeze(new Map([ + ["Bootstrap registry identities", "skipped"], + ["Dispatch verified bootstrap continuation", "skipped"], + ["Dispatch verified registry continuation", "skipped"], + [STAGING_JOB_NAME, "failure"], + ["Prepare release dry run", "skipped"], + ["Prepare release PR", "skipped"], + ["Publish exact registry topology", "skipped"], + ["Validate release inputs", "success"], + ["Verify consumers and publish GitHub releases", "skipped"], +])); + +export const EXPECTED_STAGING_STEP_OUTCOMES = Object.freeze(new Map([ + ["Freeze exhaustive publication lock", { conclusion: "success", number: 56 }], + ["Match prior approved publication lock", { conclusion: "success", number: 60 }], + ["Classify pre-tag registry publication state", { conclusion: "success", number: 68 }], + ["Upload publication lock audit evidence", { conclusion: "success", number: 75 }], + ["Re-admit the live GitHub request envelope immediately before mutation", { conclusion: "success", number: 76 }], + ["Cool down and open the shared GitHub content-write journal", { conclusion: "success", number: 77 }], + ["Admit or recover exact immutable release transport ref", { conclusion: "success", number: 78 }], + ["Stage exact-SHA product tags and draft releases", { conclusion: "success", number: 79 }], + ["Verify exact product tags", { conclusion: "success", number: 80 }], + ["Verify exact-SHA GitHub release staging", { conclusion: "success", number: 81 }], + ["Publish all selected GitHub release asset sets concurrently", { conclusion: "success", number: 82 }], + ["Resolve exact selected extension attestation subjects", { conclusion: "success", number: 86 }], + ["Reserve extension attestation content write (shard 1)", { conclusion: "success", number: 87 }], + ["Attest selected extension release assets (shard 1)", { conclusion: "success", number: 88 }], + ["Reserve extension attestation content write (shard 2)", { conclusion: "success", number: 89 }], + ["Attest selected extension release assets (shard 2)", { conclusion: "success", number: 90 }], + ["Reserve liboliphaunt attestation content write", { conclusion: "success", number: 91 }], + ["Attest liboliphaunt release assets", { conclusion: "success", number: 92 }], + ["Publish Swift SDK GitHub release and SwiftPM tags", { conclusion: "success", number: 93 }], + ["Reserve broker attestation content write", { conclusion: "success", number: 94 }], + ["Attest broker release assets", { conclusion: "success", number: 95 }], + ["Reserve Node direct attestation content write", { conclusion: "success", number: 96 }], + ["Attest Node direct release assets", { conclusion: "success", number: 97 }], + ["Reserve WASIX attestation content write", { conclusion: "success", number: 98 }], + ["Attest WASIX release assets", { conclusion: "success", number: 99 }], + [FAILING_STEP_NAME, { conclusion: "failure", number: 100 }], + ["Seal immutable GitHub-stage handoff", { conclusion: "skipped", number: 101 }], + ["Preserve immutable GitHub-stage handoff", { conclusion: "skipped", number: 102 }], + ["Preserve failed GitHub staging evidence", { conclusion: "success", number: 103 }], +])); + +function fail(message) { + throw new Error(`${PREFIX}: ${message}`); +} + +function compareText(left, right) { + return left < right ? -1 : left > right ? 1 : 0; +} + +function sha256(bytes) { + return createHash("sha256").update(bytes).digest("hex"); +} + +function plainObject(value) { + return value !== null + && typeof value === "object" + && !Array.isArray(value) + && [Object.prototype, null].includes(Object.getPrototypeOf(value)); +} + +function assertObject(value, label) { + if (!plainObject(value)) fail(`${label} must be an object`); + return value; +} + +function assertKeySet(value, expected, label) { + assertObject(value, label); + const actual = Object.keys(value).sort(compareText); + const wanted = [...expected].sort(compareText); + if (JSON.stringify(actual) !== JSON.stringify(wanted)) { + fail(`${label} keys must be exactly ${wanted.join(", ")}`); + } +} + +function positiveInteger(value, label) { + if (!Number.isSafeInteger(value) || value < 1) fail(`${label} must be a positive safe integer`); + return value; +} + +function sha(value, label) { + if (typeof value !== "string" || !/^[0-9a-f]{40}$/u.test(value)) { + fail(`${label} must be a lowercase full commit SHA`); + } + return value; +} + +function digest(value, label) { + if (typeof value !== "string" || !/^sha256:[0-9a-f]{64}$/u.test(value)) { + fail(`${label} must be a lowercase sha256 digest`); + } + return value; +} + +function hash(value, label) { + if (typeof value !== "string" || !/^[0-9a-f]{64}$/u.test(value)) { + fail(`${label} must be a lowercase SHA-256 hash`); + } + return value; +} + +function exact(value, expected, label) { + if (value !== expected) { + fail(`${label} must be ${JSON.stringify(expected)}, got ${JSON.stringify(value)}`); + } +} + +function boundedBuffer(value, maximum, label) { + if (!Buffer.isBuffer(value)) fail(`${label} must be bytes`); + if (value.length === 0 || value.length > maximum) { + fail(`${label} must contain between 1 and ${maximum} bytes`); + } + return value; +} + +function parseJsonBytes(bytes, label) { + boundedBuffer(bytes, MAX_JSON_BYTES, label); + let text; + try { + text = UTF8.decode(bytes); + } catch { + fail(`${label} is not valid UTF-8`); + } + try { + return JSON.parse(text); + } catch (cause) { + fail(`${label} is not strict JSON: ${cause.message}`); + } +} + +function unwrapBoundary(value) { + const document = assertObject(value, "boundary document"); + if (Object.hasOwn(document, "recoveryBoundary")) { + if (!Object.hasOwn(document, "releaseSource")) { + fail("selected recovery record must contain releaseSource"); + } + assertKeySet(document.releaseSource, ["commit", "tree"], "releaseSource"); + return { + boundary: document.recoveryBoundary, + releaseSource: document.releaseSource, + }; + } + return { boundary: document, releaseSource: null }; +} + +export function validateGithubStagedRecoveryBoundary(value) { + const { boundary, releaseSource } = unwrapBoundary(value); + assertKeySet(boundary, ["evidenceArtifact", "job", "kind", "run"], "recoveryBoundary"); + exact(boundary.kind, "github-staged", "recoveryBoundary.kind"); + + assertKeySet( + boundary.run, + ["attempt", "conclusion", "event", "headSha", "id", "status"], + "recoveryBoundary.run", + ); + const run = { + attempt: positiveInteger(boundary.run.attempt, "recoveryBoundary.run.attempt"), + conclusion: boundary.run.conclusion, + event: boundary.run.event, + headSha: sha(boundary.run.headSha, "recoveryBoundary.run.headSha"), + id: positiveInteger(boundary.run.id, "recoveryBoundary.run.id"), + status: boundary.run.status, + }; + exact(run.status, "completed", "recoveryBoundary.run.status"); + exact(run.conclusion, "failure", "recoveryBoundary.run.conclusion"); + exact(run.event, "workflow_dispatch", "recoveryBoundary.run.event"); + + assertKeySet(boundary.job, ["conclusion", "id", "name"], "recoveryBoundary.job"); + const job = { + conclusion: boundary.job.conclusion, + id: positiveInteger(boundary.job.id, "recoveryBoundary.job.id"), + name: boundary.job.name, + }; + exact(job.conclusion, "failure", "recoveryBoundary.job.conclusion"); + exact(job.name, STAGING_JOB_NAME, "recoveryBoundary.job.name"); + + assertKeySet( + boundary.evidenceArtifact, + ["digest", "id", "name", "size"], + "recoveryBoundary.evidenceArtifact", + ); + const evidenceArtifact = { + digest: digest(boundary.evidenceArtifact.digest, "recoveryBoundary.evidenceArtifact.digest"), + id: positiveInteger(boundary.evidenceArtifact.id, "recoveryBoundary.evidenceArtifact.id"), + name: boundary.evidenceArtifact.name, + size: positiveInteger(boundary.evidenceArtifact.size, "recoveryBoundary.evidenceArtifact.size"), + }; + if (evidenceArtifact.size > MAX_ARTIFACT_BYTES) { + fail(`recoveryBoundary.evidenceArtifact.size exceeds ${MAX_ARTIFACT_BYTES} bytes`); + } + exact( + evidenceArtifact.name, + `github-staging-recovery-${run.headSha}-${run.id}-${run.attempt}`, + "recoveryBoundary.evidenceArtifact.name", + ); + + if (releaseSource !== null) { + sha(releaseSource.commit, "releaseSource.commit"); + sha(releaseSource.tree, "releaseSource.tree"); + exact(releaseSource.commit, run.headSha, "releaseSource.commit"); + } + return { evidenceArtifact, job, releaseSource, run }; +} + +function jsonRead(repo, endpoint, label) { + let value; + try { + value = JSON.parse(runGitHubReadSync( + ["api", "-H", "X-GitHub-Api-Version: 2022-11-28", `repos/${repo}/${endpoint}`], + { + cwd: ROOT, + label, + maxBuffer: 8 * 1024 * 1024, + onRetry: ({ attempt, delayMs }) => { + console.error(`${PREFIX}: ${label} transiently failed after attempt ${attempt}; retrying in ${delayMs}ms`); + }, + }, + )); + } catch (cause) { + fail(`${label} failed: ${cause.message}`); + } + return value; +} + +function defaultGithub() { + return { + downloadArtifact({ artifactId, repo }) { + try { + return runGitHubReadSync( + [ + "api", + "-H", + "X-GitHub-Api-Version: 2022-11-28", + `repos/${repo}/actions/artifacts/${artifactId}/zip`, + ], + { + binary: true, + cwd: ROOT, + label: `download pinned recovery artifact ${artifactId}`, + maxBuffer: MAX_ARTIFACT_BYTES, + onRetry: ({ attempt, delayMs }) => { + console.error( + `${PREFIX}: recovery artifact download transiently failed after attempt ${attempt}; ` + + `retrying in ${delayMs}ms`, + ); + }, + }, + ); + } catch (cause) { + fail(`cannot download pinned recovery artifact ${artifactId}: ${cause.message}`); + } + }, + getArtifact({ artifactId, repo }) { + return jsonRead(repo, `actions/artifacts/${artifactId}`, `read pinned recovery artifact ${artifactId}`); + }, + getJobs({ attempt, repo, runId }) { + try { + return runGitHubPaginatedJsonSync( + `repos/${repo}/actions/runs/${runId}/attempts/${attempt}/jobs`, + { + cwd: ROOT, + itemsField: "jobs", + label: `read pinned Release run ${runId} attempt ${attempt} jobs`, + maxPages: 2, + onRetry: ({ attempt: readAttempt, delayMs }) => { + console.error( + `${PREFIX}: job inventory read transiently failed after attempt ${readAttempt}; ` + + `retrying in ${delayMs}ms`, + ); + }, + }, + ); + } catch (cause) { + fail(`cannot read pinned Release run job inventory: ${cause.message}`); + } + }, + getRun({ repo, runId }) { + return jsonRead(repo, `actions/runs/${runId}`, `read pinned Release run ${runId}`); + }, + }; +} + +function validateLiveRun(live, boundary, repo) { + assertObject(live, "live Release run"); + exact(live.id, boundary.id, "live Release run id"); + exact(live.run_attempt, boundary.attempt, "live Release run attempt"); + exact(live.status, boundary.status, "live Release run status"); + exact(live.conclusion, boundary.conclusion, "live Release run conclusion"); + exact(live.event, boundary.event, "live Release run event"); + exact(live.head_sha, boundary.headSha, "live Release run head SHA"); + exact(live.head_branch, "main", "live Release run head branch"); + exact(live.path, RELEASE_WORKFLOW_PATH, "live Release workflow path"); + exact(live.name, RELEASE_WORKFLOW_NAME, "live Release workflow name"); + exact(live.display_title, RELEASE_WORKFLOW_NAME, "live Release display title"); + exact(live.repository?.full_name, repo, "live Release repository"); + exact(live.head_repository?.full_name, repo, "live Release head repository"); + exact(live.head_commit?.id, boundary.headSha, "live Release head commit"); +} + +function validateCriticalStagingSteps(steps) { + if (!Array.isArray(steps) || steps.length === 0) fail("live staging job has no steps"); + const byName = new Map(); + const seenNumbers = new Set(); + let previousNumber = 0; + for (const step of steps) { + assertObject(step, "live staging step"); + if (typeof step.name !== "string" || step.name.length === 0 || byName.has(step.name)) { + fail("live staging steps must have unique nonempty names"); + } + positiveInteger(step.number, `live staging step ${step.name} number`); + if (seenNumbers.has(step.number) || step.number <= previousNumber) { + fail("live staging steps must have unique strictly increasing numbers"); + } + previousNumber = step.number; + seenNumbers.add(step.number); + exact(step.status, "completed", `live staging step ${step.name} status`); + byName.set(step.name, step); + } + for (const [name, expected] of EXPECTED_STAGING_STEP_OUTCOMES) { + const step = byName.get(name); + if (step === undefined) fail(`live staging job is missing critical step ${JSON.stringify(name)}`); + exact(step.number, expected.number, `live staging step ${name} number`); + exact(step.conclusion, expected.conclusion, `live staging step ${name} conclusion`); + } + const failed = steps.filter(({ conclusion }) => conclusion === "failure"); + if (failed.length !== 1 || failed[0].name !== FAILING_STEP_NAME) { + fail(`live staging job must have exactly one failed step: ${FAILING_STEP_NAME}`); + } + const disallowed = steps.filter(({ conclusion }) => + !["success", "skipped", "failure"].includes(conclusion)); + if (disallowed.length > 0) { + fail(`live staging job contains unsupported step outcome ${JSON.stringify(disallowed[0].conclusion)}`); + } +} + +function validateLiveJobs(jobs, boundary, run) { + if (!Array.isArray(jobs)) fail("live Release job inventory must be an array"); + if (jobs.length !== EXPECTED_RELEASE_JOB_OUTCOMES.size) { + fail( + `live Release job inventory must contain exactly ${EXPECTED_RELEASE_JOB_OUTCOMES.size} jobs, ` + + `got ${jobs.length}`, + ); + } + const byName = new Map(); + const ids = new Set(); + for (const job of jobs) { + assertObject(job, "live Release job"); + positiveInteger(job.id, "live Release job id"); + if (ids.has(job.id)) fail(`live Release job inventory repeats job id ${job.id}`); + ids.add(job.id); + if (typeof job.name !== "string" || job.name.length === 0 || byName.has(job.name)) { + fail("live Release jobs must have unique nonempty names"); + } + exact(job.run_id, run.id, `live Release job ${job.name} run id`); + exact(job.run_attempt, run.attempt, `live Release job ${job.name} run attempt`); + exact(job.head_sha, run.headSha, `live Release job ${job.name} head SHA`); + exact(job.status, "completed", `live Release job ${job.name} status`); + exact(job.workflow_name, RELEASE_WORKFLOW_NAME, `live Release job ${job.name} workflow name`); + byName.set(job.name, job); + } + for (const [name, expectedConclusion] of EXPECTED_RELEASE_JOB_OUTCOMES) { + const job = byName.get(name); + if (job === undefined) fail(`live Release job inventory is missing ${JSON.stringify(name)}`); + exact(job.conclusion, expectedConclusion, `live Release job ${name} conclusion`); + if (expectedConclusion === "skipped" && (!Array.isArray(job.steps) || job.steps.length !== 0)) { + fail(`skipped Release job ${name} must expose no executed steps`); + } + } + const staging = byName.get(STAGING_JOB_NAME); + exact(staging.id, boundary.id, "live staging job id"); + exact(staging.conclusion, boundary.conclusion, "live staging job conclusion"); + if (JSON.stringify(staging.labels) !== JSON.stringify(["macos-26"])) { + fail("live staging job must have the exact macos-26 runner label"); + } + validateCriticalStagingSteps(staging.steps); +} + +function validateLiveArtifact(live, boundary, run, repo) { + assertObject(live, "live recovery artifact"); + exact(live.id, boundary.id, "live recovery artifact id"); + exact(live.name, boundary.name, "live recovery artifact name"); + exact(live.size_in_bytes, boundary.size, "live recovery artifact size"); + exact(live.digest, boundary.digest, "live recovery artifact digest"); + exact(live.expired, false, "live recovery artifact expired state"); + exact(live.workflow_run?.id, run.id, "live recovery artifact workflow run id"); + exact(live.workflow_run?.head_sha, run.headSha, "live recovery artifact workflow run head SHA"); + exact(live.workflow_run?.head_branch, "main", "live recovery artifact workflow run head branch"); + exact( + live.url, + `https://api.github.com/repos/${repo}/actions/artifacts/${boundary.id}`, + "live recovery artifact API URL", + ); + exact( + live.archive_download_url, + `https://api.github.com/repos/${repo}/actions/artifacts/${boundary.id}/zip`, + "live recovery artifact download URL", + ); +} + +function checkedCommand(command, args, { binary = false, maxOutputBytes, stdoutTerminator } = {}) { + const capture = binary ? captureCommandBytes : captureCommandOutput; + const result = capture(command, args, { + cwd: ROOT, + label: `${command} ${args.join(" ")}`, + maxOutputBytes, + stdoutTerminator, + }); + if (result.error !== undefined || result.status !== 0) { + const stderr = Buffer.isBuffer(result.stderr) ? result.stderr.toString("utf8") : result.stderr; + fail(`${command} ${args.join(" ")} failed: ${(stderr || result.error?.message || "unknown error").trim()}`); + } + return result.stdout; +} + +export function extractBoundaryPublicationLock(archiveBytes) { + boundedBuffer(archiveBytes, MAX_ARTIFACT_BYTES, "recovery artifact ZIP"); + if (archiveBytes.length < 4 || archiveBytes[0] !== 0x50 || archiveBytes[1] !== 0x4b) { + fail("recovery artifact is not a ZIP archive"); + } + const directory = mkdtempSync(path.join(os.tmpdir(), "oliphaunt-github-stage-boundary-")); + try { + const archive = path.join(directory, "artifact.zip"); + writeFileSync(archive, archiveBytes, { flag: "wx", mode: 0o600 }); + checkedCommand("unzip", ["-tqq", archive], { maxOutputBytes: 64 * 1024 }); + const memberOutput = checkedCommand("unzip", ["-Z1", archive], { + maxOutputBytes: 64 * 1024, + stdoutTerminator: "\n", + }); + const members = memberOutput.split(/\r?\n/u).filter(Boolean); + if (new Set(members).size !== members.length) { + fail("recovery artifact ZIP repeats a member"); + } + const canonicalMembers = [...members].sort(compareText); + if (JSON.stringify(canonicalMembers) !== JSON.stringify(EXPECTED_RECOVERY_ARCHIVE_MEMBERS)) { + fail("recovery artifact ZIP member inventory is not the exact pinned GitHub-staging evidence set"); + } + return checkedCommand("unzip", ["-p", archive, LOCK_MEMBER], { + binary: true, + maxOutputBytes: MAX_JSON_BYTES, + }); + } finally { + rmSync(directory, { force: true, recursive: true }); + } +} + +function defaultValidateApprovedLock(bytes) { + const parsed = parseJsonBytes(bytes, "approved publication lock"); + try { + return validatePublicationLock(parsed); + } catch (cause) { + fail(`approved publication lock is invalid: ${cause.message}`); + } +} + +export function verifyGithubStagedRecoveryBoundary({ + approvedLockBytes, + boundaryDocument, + repo, +}, dependencies = {}) { + if (typeof repo !== "string" || !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repo)) { + fail("repo must be owner/repository"); + } + boundedBuffer(approvedLockBytes, MAX_JSON_BYTES, "approved publication lock"); + const boundary = validateGithubStagedRecoveryBoundary(boundaryDocument); + const github = dependencies.github ?? defaultGithub(); + const validateApprovedLock = dependencies.validateApprovedLock ?? defaultValidateApprovedLock; + const extractPublicationLock = dependencies.extractPublicationLock ?? extractBoundaryPublicationLock; + const approvedLock = validateApprovedLock(approvedLockBytes); + assertObject(approvedLock, "validated approved publication lock"); + exact(approvedLock.schema, "oliphaunt-publication-lock-v1", "approved publication lock schema"); + exact(approvedLock.source?.commit, boundary.run.headSha, "approved publication lock source commit"); + if (boundary.releaseSource !== null) { + exact( + approvedLock.source?.tree, + boundary.releaseSource.tree, + "approved publication lock source tree", + ); + } + hash(approvedLock.lockDigest, "approved publication lock lockDigest"); + + const liveRun = github.getRun({ repo, runId: boundary.run.id }); + validateLiveRun(liveRun, boundary.run, repo); + const liveJobs = github.getJobs({ attempt: boundary.run.attempt, repo, runId: boundary.run.id }); + validateLiveJobs(liveJobs, boundary.job, boundary.run); + const liveArtifact = github.getArtifact({ artifactId: boundary.evidenceArtifact.id, repo }); + validateLiveArtifact(liveArtifact, boundary.evidenceArtifact, boundary.run, repo); + + const archiveBytes = boundedBuffer( + github.downloadArtifact({ artifactId: boundary.evidenceArtifact.id, repo }), + MAX_ARTIFACT_BYTES, + "downloaded recovery artifact", + ); + exact(archiveBytes.length, boundary.evidenceArtifact.size, "downloaded recovery artifact size"); + exact( + `sha256:${sha256(archiveBytes)}`, + boundary.evidenceArtifact.digest, + "downloaded recovery artifact digest", + ); + const recoveredLockBytes = boundedBuffer( + extractPublicationLock(archiveBytes), + MAX_JSON_BYTES, + "recovered embedded publication lock", + ); + if (!recoveredLockBytes.equals(approvedLockBytes)) { + fail("recovered embedded publication-lock bytes do not equal the approved lock"); + } + + return Object.freeze({ + artifactDigest: boundary.evidenceArtifact.digest, + artifactId: boundary.evidenceArtifact.id, + failedStep: FAILING_STEP_NAME, + jobId: boundary.job.id, + lockDigest: approvedLock.lockDigest, + releaseSource: boundary.run.headSha, + runAttempt: boundary.run.attempt, + runId: boundary.run.id, + }); +} + +function readBoundedRegularFile(file, maximum, label) { + const absolute = path.resolve(file); + let metadata; + try { + metadata = lstatSync(absolute); + } catch (cause) { + fail(`cannot inspect ${label} ${file}: ${cause.message}`); + } + if (!metadata.isFile() || metadata.isSymbolicLink() || metadata.size === 0 || metadata.size > maximum) { + fail(`${label} ${file} must be a nonempty regular non-symlink file of at most ${maximum} bytes`); + } + return readFileSync(absolute); +} + +function parseArgs(argv) { + const values = new Map(); + for (let index = 0; index < argv.length; index += 2) { + const flag = argv[index]; + const value = argv[index + 1]; + if (!["--approved-lock", "--boundary", "--repo"].includes(flag)) { + fail(`unknown argument ${JSON.stringify(flag)}`); + } + if (value === undefined || value.length === 0 || value.startsWith("--")) { + fail(`${flag} requires a value`); + } + if (values.has(flag)) fail(`${flag} may be supplied only once`); + values.set(flag, value); + } + const approvedLock = values.get("--approved-lock"); + const boundary = values.get("--boundary"); + const repo = values.get("--repo") ?? process.env.GH_REPO; + if (!approvedLock || !boundary) { + fail("usage: verify-github-staged-recovery-boundary.mjs --boundary FILE --approved-lock FILE [--repo OWNER/REPO]"); + } + if (!repo) fail("--repo or GH_REPO is required"); + return { approvedLock, boundary, repo }; +} + +export function main(argv = process.argv.slice(2)) { + const args = parseArgs(argv); + const boundaryDocument = parseJsonBytes( + readBoundedRegularFile(args.boundary, MAX_JSON_BYTES, "boundary document"), + "boundary document", + ); + const approvedLockBytes = readBoundedRegularFile( + args.approvedLock, + MAX_JSON_BYTES, + "approved publication lock", + ); + const result = verifyGithubStagedRecoveryBoundary({ + approvedLockBytes, + boundaryDocument, + repo: args.repo, + }); + console.log( + `GitHub-staged recovery boundary verified: run ${result.runId} attempt ${result.runAttempt}, ` + + `job ${result.jobId}, artifact ${result.artifactId}, lock ${result.lockDigest}`, + ); +} + +if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { + main(); + } catch (cause) { + console.error(cause instanceof Error ? cause.message : String(cause)); + process.exit(1); + } +} diff --git a/tools/release/verify-github-staged-recovery-boundary.test.mjs b/tools/release/verify-github-staged-recovery-boundary.test.mjs new file mode 100644 index 00000000..c8845719 --- /dev/null +++ b/tools/release/verify-github-staged-recovery-boundary.test.mjs @@ -0,0 +1,321 @@ +#!/usr/bin/env bun + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import test from "node:test"; + +import { execFileSync } from "../test/fd-backed-spawn-sync.mjs"; +import { + EXPECTED_RECOVERY_ARCHIVE_MEMBERS, + EXPECTED_RELEASE_JOB_OUTCOMES, + EXPECTED_STAGING_STEP_OUTCOMES, + extractBoundaryPublicationLock, + validateGithubStagedRecoveryBoundary, + verifyGithubStagedRecoveryBoundary, +} from "./verify-github-staged-recovery-boundary.mjs"; + +const REPO = "f0rr0/oliphaunt"; +const RELEASE_SHA = "ae3d29ba16245e9345a8d337cd17c53f9bf2e853"; +const RELEASE_TREE = "673e8f249d2f51d10997f0036a7e471bf35a388e"; +const RUN_ID = 31281649203; +const JOB_ID = 93163883359; +const ARTIFACT_ID = 9029551798; +const ARCHIVE_BYTES = Buffer.from("exact pinned recovery artifact fixture"); +const APPROVED_LOCK_BYTES = Buffer.from("exact approved publication lock fixture\n"); + +function sha256(bytes) { + return createHash("sha256").update(bytes).digest("hex"); +} + +function boundaryDocument() { + return { + evidenceArtifact: { + digest: `sha256:${sha256(ARCHIVE_BYTES)}`, + id: ARTIFACT_ID, + name: `github-staging-recovery-${RELEASE_SHA}-${RUN_ID}-1`, + size: ARCHIVE_BYTES.length, + }, + job: { + conclusion: "failure", + id: JOB_ID, + name: "Prepare and stage release", + }, + kind: "github-staged", + run: { + attempt: 1, + conclusion: "failure", + event: "workflow_dispatch", + headSha: RELEASE_SHA, + id: RUN_ID, + status: "completed", + }, + }; +} + +function liveRun() { + return { + conclusion: "failure", + display_title: "Release / publish / main", + event: "workflow_dispatch", + head_branch: "main", + head_commit: { id: RELEASE_SHA }, + head_repository: { full_name: REPO }, + head_sha: RELEASE_SHA, + id: RUN_ID, + name: "Release / publish / main", + path: ".github/workflows/release.yml", + repository: { full_name: REPO }, + run_attempt: 1, + status: "completed", + }; +} + +function stagingSteps() { + const rows = [ + { conclusion: "success", name: "Set up job", number: 1 }, + ...[...EXPECTED_STAGING_STEP_OUTCOMES].map(([name, expected]) => ({ + conclusion: expected.conclusion, + name, + number: expected.number, + })), + { conclusion: "success", name: "Complete job", number: 207 }, + ].sort((left, right) => left.number - right.number); + return rows.map((row) => ({ ...row, status: "completed" })); +} + +function liveJobs() { + let nextId = JOB_ID + 1; + return [...EXPECTED_RELEASE_JOB_OUTCOMES].map(([name, conclusion]) => { + const staging = name === "Prepare and stage release"; + return { + conclusion, + head_sha: RELEASE_SHA, + id: staging ? JOB_ID : nextId++, + labels: staging ? ["macos-26"] : [], + name, + run_attempt: 1, + run_id: RUN_ID, + status: "completed", + steps: staging ? stagingSteps() : [], + workflow_name: "Release / publish / main", + }; + }); +} + +function liveArtifact() { + const boundary = boundaryDocument(); + return { + archive_download_url: `https://api.github.com/repos/${REPO}/actions/artifacts/${ARTIFACT_ID}/zip`, + digest: boundary.evidenceArtifact.digest, + expired: false, + id: ARTIFACT_ID, + name: boundary.evidenceArtifact.name, + size_in_bytes: ARCHIVE_BYTES.length, + url: `https://api.github.com/repos/${REPO}/actions/artifacts/${ARTIFACT_ID}`, + workflow_run: { + head_branch: "main", + head_sha: RELEASE_SHA, + id: RUN_ID, + }, + }; +} + +function fixture() { + const state = { + artifact: liveArtifact(), + archive: Buffer.from(ARCHIVE_BYTES), + jobs: liveJobs(), + recoveredLock: Buffer.from(APPROVED_LOCK_BYTES), + run: liveRun(), + }; + const dependencies = { + extractPublicationLock: () => state.recoveredLock, + github: { + downloadArtifact: () => state.archive, + getArtifact: () => state.artifact, + getJobs: () => state.jobs, + getRun: () => state.run, + }, + validateApprovedLock: () => ({ + lockDigest: "a".repeat(64), + schema: "oliphaunt-publication-lock-v1", + source: { commit: RELEASE_SHA, tree: RELEASE_TREE }, + }), + }; + return { dependencies, state }; +} + +function verify(document = boundaryDocument(), mutate = () => {}) { + const { dependencies, state } = fixture(); + mutate(state, dependencies); + return verifyGithubStagedRecoveryBoundary({ + approvedLockBytes: APPROVED_LOCK_BYTES, + boundaryDocument: document, + repo: REPO, + }, dependencies); +} + +test("accepts only the exact pinned failed GitHub-stage boundary", () => { + const result = verify(); + assert.deepEqual(result, { + artifactDigest: `sha256:${sha256(ARCHIVE_BYTES)}`, + artifactId: ARTIFACT_ID, + failedStep: "Freeze exact GitHub release asset and attestation evidence", + jobId: JOB_ID, + lockDigest: "a".repeat(64), + releaseSource: RELEASE_SHA, + runAttempt: 1, + runId: RUN_ID, + }); +}); + +test("binds a full selected recovery record to its release source", () => { + const document = { + recoveryBoundary: boundaryDocument(), + releaseSource: { commit: RELEASE_SHA, tree: RELEASE_TREE }, + }; + assert.equal(validateGithubStagedRecoveryBoundary(document).releaseSource.tree, RELEASE_TREE); + verify(document); + + document.releaseSource.commit = "b".repeat(40); + assert.throws( + () => validateGithubStagedRecoveryBoundary(document), + /releaseSource[.]commit/u, + ); +}); + +test("rejects unpinned boundary shapes and non-GitHub-stage kinds", () => { + const extra = boundaryDocument(); + extra.unchecked = true; + assert.throws( + () => validateGithubStagedRecoveryBoundary(extra), + /keys must be exactly/u, + ); + + const wrongKind = boundaryDocument(); + wrongKind.kind = "registry-partial"; + assert.throws( + () => validateGithubStagedRecoveryBoundary(wrongKind), + /recoveryBoundary[.]kind/u, + ); +}); + +test("rejects live run identity, attempt, event, status, and failure drift", () => { + for (const [field, value, pattern] of [ + ["head_sha", "b".repeat(40), /head SHA/u], + ["run_attempt", 2, /run attempt/u], + ["event", "push", /run event/u], + ["status", "in_progress", /run status/u], + ["conclusion", "success", /run conclusion/u], + ]) { + assert.throws( + () => verify(boundaryDocument(), (state) => { state.run[field] = value; }), + pattern, + ); + } +}); + +test("rejects any failed-step or critical staging-outcome drift", () => { + assert.throws( + () => verify(boundaryDocument(), (state) => { + const step = state.jobs + .find(({ name }) => name === "Prepare and stage release") + .steps.find(({ name }) => name === "Attest WASIX release assets"); + step.conclusion = "skipped"; + }), + /Attest WASIX release assets conclusion/u, + ); + assert.throws( + () => verify(boundaryDocument(), (state) => { + const step = state.jobs + .find(({ name }) => name === "Prepare and stage release") + .steps.find(({ name }) => name === "Freeze exact GitHub release asset and attestation evidence"); + step.conclusion = "success"; + }), + /Freeze exact GitHub release asset and attestation evidence conclusion/u, + ); +}); + +test("requires registry, continuation, and finalization jobs to remain skipped", () => { + for (const jobName of [ + "Dispatch verified registry continuation", + "Publish exact registry topology", + "Verify consumers and publish GitHub releases", + ]) { + assert.throws( + () => verify(boundaryDocument(), (state) => { + state.jobs.find(({ name }) => name === jobName).conclusion = "success"; + }), + new RegExp(`${jobName} conclusion`, "u"), + ); + } +}); + +test("rejects artifact metadata, workflow binding, expiry, and downloaded-byte drift", () => { + assert.throws( + () => verify(boundaryDocument(), (state) => { state.artifact.expired = true; }), + /expired state/u, + ); + assert.throws( + () => verify(boundaryDocument(), (state) => { state.artifact.workflow_run.id += 1; }), + /workflow run id/u, + ); + assert.throws( + () => verify(boundaryDocument(), (state) => { state.artifact.digest = `sha256:${"b".repeat(64)}`; }), + /artifact digest/u, + ); + assert.throws( + () => verify(boundaryDocument(), (state) => { state.archive = Buffer.from("different archive bytes"); }), + /downloaded recovery artifact size/u, + ); + assert.throws( + () => verify(boundaryDocument(), (state) => { + state.archive = Buffer.from(ARCHIVE_BYTES); + state.archive[0] ^= 0xff; + }), + /downloaded recovery artifact digest/u, + ); +}); + +test("requires the recovery artifact's embedded lock bytes to equal the approved lock", () => { + assert.throws( + () => verify(boundaryDocument(), (state) => { + state.recoveredLock = Buffer.from("different publication lock\n"); + }), + /do not equal the approved lock/u, + ); +}); + +test("extracts only the exact pinned artifact member inventory", () => { + const root = mkdtempSync(path.join(tmpdir(), "oliphaunt-github-stage-boundary-test-")); + try { + const archiveRoot = path.join(root, "archive-root"); + mkdirSync(archiveRoot); + for (const member of EXPECTED_RECOVERY_ARCHIVE_MEMBERS) { + const file = path.join(archiveRoot, member); + mkdirSync(path.dirname(file), { recursive: true }); + writeFileSync(file, member.endsWith("publication-lock.json") ? APPROVED_LOCK_BYTES : `${member}\n`); + } + const archive = path.join(root, "artifact.zip"); + execFileSync("zip", ["-q", archive, ...EXPECTED_RECOVERY_ARCHIVE_MEMBERS], { cwd: archiveRoot }); + assert.deepEqual(extractBoundaryPublicationLock(readFileSync(archive)), APPROVED_LOCK_BYTES); + + writeFileSync(path.join(archiveRoot, "unexpected.json"), "{}\n"); + execFileSync("zip", ["-q", archive, "unexpected.json"], { cwd: archiveRoot }); + assert.throws( + () => extractBoundaryPublicationLock(readFileSync(archive)), + /member inventory is not the exact pinned/u, + ); + } finally { + rmSync(root, { force: true, recursive: true }); + } +}); diff --git a/tools/release/verify-release-recovery-publication.mjs b/tools/release/verify-release-recovery-publication.mjs index ca71d76b..847e778d 100644 --- a/tools/release/verify-release-recovery-publication.mjs +++ b/tools/release/verify-release-recovery-publication.mjs @@ -20,7 +20,7 @@ import { const TOOL = "verify-release-recovery-publication.mjs"; const INVENTORY_SCHEMA = "oliphaunt-release-registry-inventory-v1"; export const RECOVERY_PUBLICATION_STATE_SCHEMA = - "oliphaunt-release-recovery-publication-state-v1"; + "oliphaunt-release-recovery-publication-state-v2"; const SHA = /^[0-9a-f]{40}$/u; const KIND_ECOSYSTEM = new Map([ ["crates", "cargo"], @@ -101,11 +101,23 @@ function inventoryPackageMap(packages, context) { } export function classifyReleaseRecoveryPublication({ + immutableGithubTagCount = 0, lock, inventory, products, + recoveryBoundaryKind = "registry-partial", } = {}) { const selectedProducts = uniqueStrings(products, "products"); + if (!new Set(["github-staged", "registry-partial"]).has(recoveryBoundaryKind)) { + throw error("recovery boundary kind must be github-staged or registry-partial"); + } + if ( + !Number.isSafeInteger(immutableGithubTagCount) + || immutableGithubTagCount < 0 + || immutableGithubTagCount > selectedProducts.length + ) { + throw error("immutable GitHub tag count must cover zero through all selected products"); + } exactKeys(inventory, ["products", "results", "schema", "source"], "registry inventory"); if (inventory.schema !== INVENTORY_SCHEMA) { throw error(`registry inventory schema must be ${INVENTORY_SCHEMA}`); @@ -188,12 +200,19 @@ export function classifyReleaseRecoveryPublication({ } publicCarrierIds.sort(compareText); missingCarrierIds.sort(compareText); - if (publicCarrierIds.length === 0) { + if (recoveryBoundaryKind === "registry-partial" && publicCarrierIds.length === 0) { throw error( - "same-version recovery requires at least one already-public immutable registry carrier", + "registry-partial recovery requires at least one already-public immutable registry carrier", ); } + if ( + recoveryBoundaryKind === "github-staged" + && immutableGithubTagCount !== selectedProducts.length + ) { + throw error("github-staged recovery requires every selected product tag to be exact and staged"); + } return { + immutableGithubTagCount, source: lock.source, lockDigest: lock.lockDigest, products: [...selectedProducts].sort(compareText), @@ -202,6 +221,7 @@ export function classifyReleaseRecoveryPublication({ missingCarrierIds, needsCargoToken: missingCarrierIds.some((id) => id.startsWith("cargo:")), needsNpmToken: missingCarrierIds.some((id) => id.startsWith("npm:")), + recoveryBoundaryKind, }; } @@ -219,6 +239,8 @@ export function releaseRecoveryPublicationReceipt(classification, receipts) { source: classification.source, lockDigest: classification.lockDigest, products: classification.products, + immutableGithubTagCount: classification.immutableGithubTagCount, + recoveryBoundaryKind: classification.recoveryBoundaryKind, selectedCarrierCount: classification.selectedCarrierCount, publicCarrierCount: classification.publicCarrierIds.length, missingCarrierCount: classification.missingCarrierIds.length, @@ -242,6 +264,7 @@ export function validateReleaseRecoveryPublicationReceipt({ } = {}) { const fields = [ "evidenceDigest", + "immutableGithubTagCount", "lockDigest", "missingCarrierCount", "missingCarrierIds", @@ -250,6 +273,7 @@ export function validateReleaseRecoveryPublicationReceipt({ "products", "publicCarrierCount", "publicCarrierIds", + "recoveryBoundaryKind", "receipts", "schema", "selectedCarrierCount", @@ -263,17 +287,25 @@ export function validateReleaseRecoveryPublicationReceipt({ || stableJson(receipt.source) !== stableJson(lock.source) || receipt.lockDigest !== lock.lockDigest || !sameStrings(receipt.products, selectedProducts) + || !new Set(["github-staged", "registry-partial"]).has(receipt.recoveryBoundaryKind) ) { throw error("recovery publication receipt is not bound to the selected publication lock"); } - uniqueStrings(receipt.publicCarrierIds, "receipt publicCarrierIds"); + uniqueStrings(receipt.publicCarrierIds, "receipt publicCarrierIds", { nonempty: false }); uniqueStrings(receipt.missingCarrierIds, "receipt missingCarrierIds", { nonempty: false }); const selectedCarriers = lock.carriers.filter((carrier) => selectedProducts.includes(carrier.product) && REGISTRY_ECOSYSTEMS.has(carrier.ecosystem)); const selectedIds = selectedCarriers.map(({ id }) => id); if ( receipt.publicCarrierCount !== receipt.publicCarrierIds.length - || receipt.publicCarrierCount < 1 + || !Number.isSafeInteger(receipt.immutableGithubTagCount) + || receipt.immutableGithubTagCount < 0 + || receipt.immutableGithubTagCount > selectedProducts.length + || (receipt.recoveryBoundaryKind === "registry-partial" && receipt.publicCarrierCount === 0) + || ( + receipt.recoveryBoundaryKind === "github-staged" + && receipt.immutableGithubTagCount !== selectedProducts.length + ) || receipt.missingCarrierCount !== receipt.missingCarrierIds.length || receipt.selectedCarrierCount !== selectedCarriers.length || !sameStrings( @@ -307,8 +339,10 @@ function appendGitHubOutputs(file, receipt) { [ `needs_cargo_token=${receipt.needsCargoToken}`, `needs_npm_token=${receipt.needsNpmToken}`, + `immutable_github_tag_count=${receipt.immutableGithubTagCount}`, `public_carrier_count=${receipt.publicCarrierCount}`, `missing_carrier_count=${receipt.missingCarrierCount}`, + `recovery_boundary_kind=${receipt.recoveryBoundaryKind}`, "", ].join("\n"), ); @@ -317,18 +351,22 @@ function appendGitHubOutputs(file, receipt) { function parseArgs(argv) { const options = { githubOutput: "", + immutableGithubTagCount: "0", inventory: "", lock: "", output: "", productsJson: "", + recoveryBoundaryKind: "registry-partial", verifyReceipt: "", }; const flags = new Map([ ["--github-output", "githubOutput"], + ["--immutable-github-tag-count", "immutableGithubTagCount"], ["--inventory", "inventory"], ["--lock", "lock"], ["--output", "output"], ["--products-json", "productsJson"], + ["--recovery-boundary-kind", "recoveryBoundaryKind"], ["--verify-receipt", "verifyReceipt"], ]); for (let index = 0; index < argv.length; index += 1) { @@ -355,6 +393,10 @@ function parseArgs(argv) { } catch (cause) { throw error(`--products-json is invalid JSON: ${cause.message}`); } + options.immutableGithubTagCount = Number(options.immutableGithubTagCount); + if (!Number.isSafeInteger(options.immutableGithubTagCount)) { + throw error("--immutable-github-tag-count must be a non-negative integer"); + } return options; } @@ -376,9 +418,11 @@ if (import.meta.main) { } const inventory = JSON.parse(readFileSync(path.resolve(options.inventory), "utf8")); const classification = classifyReleaseRecoveryPublication({ + immutableGithubTagCount: options.immutableGithubTagCount, lock, inventory, products: options.products, + recoveryBoundaryKind: options.recoveryBoundaryKind, }); const receipts = await verifyLockedRegistryIntegrity(lock, { carrierIds: classification.publicCarrierIds, diff --git a/tools/release/verify-release-recovery-publication.test.mjs b/tools/release/verify-release-recovery-publication.test.mjs index 8c8cd489..e5bb6cd1 100644 --- a/tools/release/verify-release-recovery-publication.test.mjs +++ b/tools/release/verify-release-recovery-publication.test.mjs @@ -86,6 +86,8 @@ test("classifies an exact partial publication and derives only actually needed b [{ id: "cargo:alpha", proof: "exact" }], ); assert.equal(receipt.schema, RECOVERY_PUBLICATION_STATE_SCHEMA); + assert.equal(receipt.immutableGithubTagCount, 0); + assert.equal(receipt.recoveryBoundaryKind, "registry-partial"); assert.equal(receipt.publicCarrierCount, 1); assert.equal(receipt.missingCarrierCount, 3); assert.match(receipt.evidenceDigest, /^[0-9a-f]{64}$/u); @@ -115,7 +117,36 @@ test("classifies an exact partial publication and derives only actually needed b ); }); -test("rejects recovery before any immutable carrier is public", () => { +test("accepts exact immutable GitHub staging before any registry carrier is public", () => { + const value = inventory(); + for (const result of value.results) { + result.missing = result.packages; + result.published = []; + } + const classification = classifyReleaseRecoveryPublication({ + immutableGithubTagCount: 2, + lock: lock(), + inventory: value, + products: ["alpha", "beta"], + recoveryBoundaryKind: "github-staged", + }); + assert.equal(classification.immutableGithubTagCount, 2); + assert.deepEqual(classification.publicCarrierIds, []); + assert.equal(classification.missingCarrierIds.length, 4); + + const receipt = releaseRecoveryPublicationReceipt(classification, []); + assert.equal(validateReleaseRecoveryPublicationReceipt({ + lock: lock(), + products: ["alpha", "beta"], + receipt, + validateReceipts: (_lock, { carrierIds, receipts }) => { + assert.deepEqual(carrierIds, []); + assert.deepEqual(receipts, []); + }, + }), receipt); +}); + +test("rejects recovery before any immutable GitHub or registry state exists", () => { const value = inventory(); for (const result of value.results) { result.missing = result.packages; @@ -127,7 +158,7 @@ test("rejects recovery before any immutable carrier is public", () => { inventory: value, products: ["alpha", "beta"], }), - /at least one already-public immutable registry carrier/u, + /registry-partial recovery requires at least one already-public immutable registry carrier/u, ); }); diff --git a/tools/release/verify_github_release_attestations.mjs b/tools/release/verify_github_release_attestations.mjs index 99313af4..71fdd54a 100755 --- a/tools/release/verify_github_release_attestations.mjs +++ b/tools/release/verify_github_release_attestations.mjs @@ -32,7 +32,7 @@ import { validateRecoveryPromotionStatement, } from "./recovery-promotion-attestation.mjs"; import { - SAME_VERSION_RECOVERY_SOURCES_SCHEMA, + isSameVersionRecoverySourcesDocument, selectSameVersionRecoverySource, validateSameVersionRecoverySource, } from "./same-version-recovery-source.mjs"; @@ -1177,6 +1177,40 @@ function stableStringify(value) { return JSON.stringify(value); } +function canonicalSigstoreBundleForGhComparison(bundle) { + const canonical = structuredClone(bundle); + const timestampVerificationData = + canonical?.verificationMaterial?.timestampVerificationData; + if ( + timestampVerificationData !== null + && !Array.isArray(timestampVerificationData) + && typeof timestampVerificationData === "object" + && Array.isArray(timestampVerificationData.rfc3161Timestamps) + && timestampVerificationData.rfc3161Timestamps.length === 0 + ) { + // actions/attest v3 serializes this protobuf default as an empty repeated + // field, while gh's Go protobuf serializer omits it after successfully + // verifying the exact supplied bundle. Canonicalize only that known + // representation difference; every signed and verification-material byte + // represented by the bundle must still compare exactly. + delete timestampVerificationData.rfc3161Timestamps; + } + return canonical; +} + +export function assertGhVerifiedBundleMatchesSupplied( + verifiedBundle, + suppliedBundle, + context = "gh verification output", +) { + if ( + stableStringify(canonicalSigstoreBundleForGhComparison(verifiedBundle)) + !== stableStringify(canonicalSigstoreBundleForGhComparison(suppliedBundle)) + ) { + throw new Error(`${context} does not contain the supplied bundle`); + } +} + export function assertExactReleaseAssetNames({ product, tag, expectedNames, actualNames }) { const expected = new Set(expectedNames); const actual = new Set(actualNames); @@ -2271,9 +2305,11 @@ function runGhBundleVerification({ } const verified = requireObject(output[0], `gh verification output for ${bundlePath}`); const verifiedBundle = verified.attestation?.bundle; - if (stableStringify(verifiedBundle) !== stableStringify(bundle)) { - throw new Error(`gh verification output for ${bundlePath} does not contain the supplied bundle`); - } + assertGhVerifiedBundleMatchesSupplied( + verifiedBundle, + bundle, + `gh verification output for ${bundlePath}`, + ); const statement = verified.verificationResult?.statement; requireObject(statement, `gh verification statement for ${bundlePath}`); return statementSubjects( @@ -2546,10 +2582,7 @@ async function loadRecoveryAttestationExpectations(args, lock) { "same-version recovery provenance", ); let provenanceRecord; - if ( - provenance?.schema === SAME_VERSION_RECOVERY_SOURCES_SCHEMA - && Array.isArray(provenance.records) - ) { + if (isSameVersionRecoverySourcesDocument(provenance)) { provenanceRecord = selectSameVersionRecoverySource( provenance, lock.source.commit,