Build Week: generate an auditable judge evidence report

.gitignore

@@ -34,3 +34,7 @@ dist/
 # IDE
 .vscode/
 .idea/
+
+# Generated judge-demo evidence (keep only the directory marker)
+artifacts/demo-platform/*
+!artifacts/demo-platform/.gitkeep

BUILD_WEEK.md

@@ -44,6 +44,60 @@ Work started during the submission period includes:
 | Phase 5 | Real-cluster agent and console lifecycle integration | Issue #101 / PR #102 |
 | Phase 6 | OpenAI-native reasoning, configuration, deployment, UI status, and documentation | Issue #103 / PR #104 |
 | Phase 7 | One-command deterministic Argus, Phoenix, Sentinel, and SOG judge demo | Issue #110 |
+| Phase 8 | Auditable judge evidence scorecard and JSON/Markdown run artifacts | Issue #112 |
+| Phase 9 | Cluster-free full-platform judge demo and explicit live-k3s command | Issue #114 |
+| Phase 10 | Guarded live k3s detection, chaos, recovery, correlation, and measured evidence | Issue #116 |
+| Phase 11 | Judge onboarding doctor, single-path setup, and rehearsal diagnostics | Issue #118 |
+| Phase 12 | Canonical resilience lifecycle and unambiguous evidence provenance | Argus #120 / Sentinel #63 |
+| Final video | Two-minute judge story with frame-by-frame screen, action, narration, timing, and fallback plan | Pending |
+
+### Phase 12 — truthful resilience proof timeline
+
+Every correlated judge incident can now carry the same seven-stage lifecycle: Healthy,
+Fault injected, Detection, Decision, Human approval, Recovery, and Verification. The
+live producer supplies actual timestamps, measured detection/recovery durations, HTTP
+availability, experiment ID, and evidence source. Human approval occurs only after
+observed Argus evidence exists. The portable producer supplies a visually equivalent
+timeline, but every stage is explicitly simulator/replay provenance and availability is
+reported as not measured. Sentinel renders four mutually exclusive labels—Live
+Observed, Synthetic Simulator, Live Chaos Mesh, and Replayed Evidence—and never infers
+missing proof stages.
+
+### Phase 11 — judge onboarding and rehearsal doctor
+
+`make doctor` and `make doctor-live` are read-only entry points for the portable and live
+proofs. They validate the sibling repository layout, required tools, secret-safe OpenAI
+configuration status, runtime availability, local console dependencies, and—only in
+live mode—the complete k3s stack. Each run ends with an unambiguous verdict and exact
+next command. `setup.md`, the root README, and Make help now present the same choose-one
+path and distinguish replay/simulator claims from observed/live-chaos measurements.
+
+### Phase 10 — observed live-cluster proof
+
+`make demo-platform-live-dry-run` validates the actual k3s security, chaos, agent, and
+SOG stack without changing it. `make demo-platform-live` then requires two explicit
+human confirmations, creates an isolated two-replica service, waits for observed Argus
+evidence, asks Phoenix to create one real Chaos Mesh PodChaos, continuously measures
+HTTP availability, verifies Kubernetes replacement and full readiness, and publishes
+only the verified cross-agent lifecycle to Sentinel. The resulting JSON/Markdown
+scorecard distinguishes observed/live-chaos provenance from the portable simulator.
+
+### Phase 9 — portable judge demo
+
+`make demo-platform` now defaults to the cluster-free path: a disposable local SOG,
+real local product services, an explicitly synthetic three-node topology, deterministic
+Argus and Phoenix lifecycles, and a bounded dynamic feed. `make demo-platform-live`
+preserves the k3s-backed integration proof. The split prevents “safe” from being confused
+with “cluster-free” and keeps simulated evidence visually useful without mislabeling it
+as live production telemetry.
+
+### Phase 8 — judge evidence report
+
+Every successful deterministic platform run now emits a concise PASS scorecard and
+stores the verified lifecycle as JSON and Markdown. The report preserves correlation,
+sources, provenance, seed, governance, OpenAI configuration, recovery state, and measured
+orchestration timings. It explicitly avoids claiming live detection latency or 100%
+availability from replayed and simulated evidence.
 
 ### Phase 7 — deterministic platform demo
 

Makefile

@@ -1,6 +1,6 @@
 .PHONY: help cluster-up cluster-down cluster-status deploy-cilium deploy-falco \
         deploy-kyverno deploy-observability deploy-agent deploy-ui \
-        setup-local dev-agent dev-ui demo-local demo-platform demo-platform-dry-run demo-cluster demo-cluster-dry-run hubble-ui grafana-ui k9s \
+        setup-local doctor doctor-live dev-agent dev-ui demo-local demo-platform demo-platform-local demo-platform-live demo-platform-dry-run demo-platform-live-dry-run demo-cluster demo-cluster-dry-run hubble-ui grafana-ui k9s \
         test test-agent test-ui test-cluster-demo test-platform-demo simulate-threats clean
 
 THREAT_COUNT ?= 10
@@ -27,10 +27,15 @@ help:
 	@echo "    make deploy-observability  Install Prometheus + Grafana + Loki"
 	@echo ""
 	@echo "  Application"
+	@echo "    make doctor                Diagnose the portable judge path without changes"
+	@echo "    make doctor-live           Diagnose the live k3s proof without changes"
 	@echo "    make setup-local           Install local backend and UI dependencies"
 	@echo "    make demo-local            Start a populated cluster-free demo"
-	@echo "    make demo-platform         Start the full Argus + Phoenix + Sentinel judge demo"
-	@echo "    make demo-platform-dry-run Validate the full platform demo without starting anything"
+	@echo "    make demo-platform         Start the cluster-free full-platform judge demo"
+	@echo "    make demo-platform-local   Same cluster-free full-platform judge demo"
+	@echo "    make demo-platform-live    Start the real k3s-backed platform proof"
+	@echo "    make demo-platform-dry-run Validate local demo prerequisites without changes"
+	@echo "    make demo-platform-live-dry-run Validate live k3s prerequisites without changes"
 	@echo "    make dev-agent             Start only the backend on localhost:8000"
 	@echo "    make dev-ui                Start only the console on localhost:5173"
 	@echo "    make deploy-agent          Build and deploy AI agent"
@@ -104,6 +109,12 @@ setup-local:
 	.venv/bin/pip install -r agent/requirements.txt
 	npm --prefix ui ci
 
+doctor:
+	@bash scripts/demo-doctor.sh local
+
+doctor-live:
+	@bash scripts/demo-doctor.sh live
+
 dev-agent:
 	@test -x .venv/bin/python || (echo "Missing local environment. Run: make setup-local" && exit 1)
 	@cd agent/src && ../../.venv/bin/python -m uvicorn main:app --reload --host 127.0.0.1 --port 8000
@@ -118,12 +129,26 @@ demo-local:
 demo-platform:
 	@PHOENIX_ROOT="$${PHOENIX_ROOT:-$(abspath ../sentinel-stack/phoenix)}" \
 	 SENTINEL_ROOT="$${SENTINEL_ROOT:-$(abspath ../sentinel-stack/sentinel)}" \
-	 bash scripts/demo-platform.sh
+	 SENTINEL_PLATFORM_ROOT="$${SENTINEL_PLATFORM_ROOT:-$(abspath ../sentinel-stack/sentinel-platform)}" \
+	 bash scripts/demo-platform-local.sh
+
+demo-platform-local: demo-platform
+
+demo-platform-live:
+	@PHOENIX_ROOT="$${PHOENIX_ROOT:-$(abspath ../sentinel-stack/phoenix)}" \
+	 SENTINEL_ROOT="$${SENTINEL_ROOT:-$(abspath ../sentinel-stack/sentinel)}" \
+	 bash scripts/demo-platform-live-proof.sh
 
 demo-platform-dry-run:
 	@PHOENIX_ROOT="$${PHOENIX_ROOT:-$(abspath ../sentinel-stack/phoenix)}" \
 	 SENTINEL_ROOT="$${SENTINEL_ROOT:-$(abspath ../sentinel-stack/sentinel)}" \
-	 DEMO_PLATFORM_DRY_RUN=true bash scripts/demo-platform.sh
+	 SENTINEL_PLATFORM_ROOT="$${SENTINEL_PLATFORM_ROOT:-$(abspath ../sentinel-stack/sentinel-platform)}" \
+	 DEMO_PLATFORM_DRY_RUN=true bash scripts/demo-platform-local.sh
+
+demo-platform-live-dry-run:
+	@PHOENIX_ROOT="$${PHOENIX_ROOT:-$(abspath ../sentinel-stack/phoenix)}" \
+	 SENTINEL_ROOT="$${SENTINEL_ROOT:-$(abspath ../sentinel-stack/sentinel)}" \
+	 LIVE_DEMO_DRY_RUN=true bash scripts/demo-platform-live-proof.sh
 
 demo-cluster:
 	@DEMO_CLUSTER_CONTEXT="$(DEMO_CLUSTER_CONTEXT)" \
@@ -147,6 +172,8 @@ test-ui:
 
 test-platform-demo:
 	@bash scripts/tests/test-demo-platform.sh
+	@bash scripts/tests/test-demo-platform-live.sh
+	@bash scripts/tests/test-demo-doctor.sh
 
 test-cluster-demo:
 	@bash scripts/tests/test-demo-cluster.sh

README.md

@@ -54,7 +54,8 @@ cross-agent judge experience.
 |---|---|---|
 | **Local synthetic** | You want the fastest judge/reviewer experience and do not have Kubernetes | `make demo-local` |
 | **Live k3s** | You have the three-node Argus cluster and want real Falco, Cilium, and Kyverno evidence | `make demo-cluster-dry-run`, then `make demo-cluster` |
-| **Full platform** | You want Argus, Phoenix, Sentinel, and the SOG operating as one deterministic demo | `make demo-platform-dry-run`, then `make demo-platform` |
+| **Full platform, cluster-free** | Recommended judge path: all three products, realistic synthetic topology, no Kubernetes | `make demo-platform-dry-run`, then `make demo-platform` |
+| **Full platform, live k3s** | Maintainer/video path: observed evidence, approved live chaos, measured recovery | `make doctor-live`, then `make demo-platform-live` |
 
 ### Path A — local synthetic, no cluster required
 
@@ -132,19 +133,19 @@ injects the real workloads, prints terminal evidence, and keeps the console avai
 at **http://127.0.0.1:5173** until `Ctrl-C`. Cleanup stops both local processes and
 deletes only the namespace created by that run.
 
-### Path C — full deterministic platform demo
+### Path C — full cluster-free platform demo (recommended)
 
-Use this path for the complete judge story. It requires sibling Phoenix and Sentinel
-checkouts plus the deployed `sentinel-platform` and `phoenix-system` services in the
-selected Kubernetes context. Validate everything without starting processes, opening
-port-forwards, or publishing evidence:
+Use this path for the complete judge story. It requires sibling Phoenix, Sentinel, and
+Sentinel Platform checkouts plus Docker or OrbStack for one disposable Redis container.
+It does **not** require Kubernetes, kubectl, k3s, Cilium, Falco, or Chaos Mesh.
 
 ```text
 Projects/
 ├── argus-k8s/                 # run the command here
 └── sentinel-stack/
     ├── phoenix/
-    └── sentinel/
+    ├── sentinel/
+    └── sentinel-platform/
 ```
 
 Install each repository's local dependencies once:
@@ -155,11 +156,10 @@ make -C ../sentinel-stack/sentinel setup-local
 npm --prefix ../sentinel-stack/phoenix/dashboard install
 ```
 
-Select the real three-node k3s context, then run the non-mutating preflight:
+Run the non-mutating preflight:
 
 ```bash
-kubectl config use-context argus
-make demo-platform-dry-run
+make doctor
 ```
 
 Then launch the complete experience:
@@ -168,22 +168,55 @@ Then launch the complete experience:
 make demo-platform
 ```
 
-The command starts or reuses all required APIs, port-forwards, and dashboards; publishes
-one deterministic Argus security finding and one Phoenix recovery outcome against the
-same resource; and refuses to report success until Sentinel exposes their correlated
-incident with explicit `replayed` and `simulator` provenance.
+The command installs missing local dependencies, starts a disposable Redis-backed real
+SOG and the real local Argus, Phoenix, and Sentinel services, then seeds a three-node,
+multi-namespace service graph. It populates Argus with twelve threats, Phoenix with a
+synthetic dependency graph, and Sentinel with multiple correlated lifecycles. A bounded
+feed adds a new replay/simulator lifecycle every 20 seconds so refresh timestamps,
+counters, timelines, and risk views visibly move during the demo.
+
+Every successful run prints a judge-readable PASS scorecard and writes the exact proof
+to `artifacts/demo-platform/latest-demo.json` and
+`artifacts/demo-platform/latest-demo.md`. The report includes evidence-publication and
+correlation timings, recovery status, approval policy, OpenAI availability, sources,
+seed, and provenance. It deliberately reports availability as **not measured** for this
+deterministic path instead of turning a simulator recovery into a false production-SLA
+claim.
 
 | Console | URL | What to show |
 |---|---|---|
 | Argus | **http://127.0.0.1:5173** | Security evidence and response |
 | Phoenix | **http://127.0.0.1:5174** | Resilience outcome and recovery |
 | Sentinel | **http://127.0.0.1:5175** | Unified correlated incident and fleet decision |
 
-Existing healthy services are reused. On `Ctrl-C`, the command stops only processes it
-started. The default sibling paths are `../sentinel-stack/phoenix` and
-`../sentinel-stack/sentinel`; override them with `PHOENIX_ROOT` and `SENTINEL_ROOT` when
-your checkouts live elsewhere. This deterministic path does not inject live Chaos Mesh
-faults; the existing live k3s path remains available for real runtime security evidence.
+Every synthetic entity has `demo-data=synthetic`; evidence is explicitly labeled
+`replayed` or `simulator`. The local demo never claims live Falco detection, live chaos,
+or measured production availability. On `Ctrl-C`, it stops its local processes and
+removes its disposable Redis container. It replaces only known project listeners and
+kubectl port-forwards on reserved demo ports; unrelated listeners cause a safe failure.
+The Sentinel incident drawer uses the same seven-stage resilience timeline as the live
+proof, while labeling every portable stage as replayed or simulated and leaving
+availability explicitly unmeasured.
+
+### Path D — live k3s-backed platform proof
+
+Run the guarded real-cluster proof separately:
+
+```bash
+kubectl config use-context argus
+make doctor-live
+make demo-platform-live
+```
+
+The dry-run is read-only. The real command verifies Cilium, Falco, Kyverno, Argus,
+Phoenix, Chaos Mesh, and the SOG; asks for the exact Kubernetes context and the phrase
+`INJECT LIVE FAULT`; creates only `sentinel-live-demo`; and launches a two-replica HTTP
+service. Argus must observe a bounded Falco-triggering workload before Phoenix creates a
+real Chaos Mesh `PodChaos` against one disposable replica. The proof passes only after a
+new replacement pod is Ready, both replicas are Ready, continuous HTTP availability is
+measured, and Sentinel exposes the correlated Argus + Phoenix incident. `Ctrl-C` stops
+the consoles and deletes only the isolated demo namespace. Evidence is written to
+`artifacts/demo-platform/latest-live-demo.{json,md}`.
 
 ## Part of the Sentinel multi-agent platform
 

artifacts/demo-platform/.gitkeep

@@ -0,0 +1 @@
+

scripts/demo-doctor.sh

@@ -0,0 +1,115 @@
+#!/usr/bin/env bash
+set -Eeuo pipefail
+
+repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+mode="${1:-local}"
+phoenix_root="${PHOENIX_ROOT:-${repo_root}/../sentinel-stack/phoenix}"
+sentinel_root="${SENTINEL_ROOT:-${repo_root}/../sentinel-stack/sentinel}"
+platform_root="${SENTINEL_PLATFORM_ROOT:-${repo_root}/../sentinel-stack/sentinel-platform}"
+failures=0
+warnings=0
+
+usage() {
+  cat <<'EOF'
+Usage: scripts/demo-doctor.sh [local|live]
+
+  local  Check the recommended cluster-free judge path (default).
+  live   Check the observed k3s proof, including deployed stack readiness.
+
+This command is read-only. It never starts services, installs dependencies, publishes
+evidence, creates Kubernetes resources, or injects a fault.
+EOF
+}
+
+pass() { printf '  [PASS] %s\n' "$1"; }
+warn() { printf '  [WARN] %s\n' "$1"; warnings=$((warnings + 1)); }
+fail_check() { printf '  [FAIL] %s\n' "$1"; failures=$((failures + 1)); }
+
+check_command() {
+  local command_name="$1" remedy="$2"
+  if command -v "${command_name}" >/dev/null 2>&1; then
+    pass "${command_name} is installed"
+  else
+    fail_check "${command_name} is missing — ${remedy}"
+  fi
+}
+
+check_directory() {
+  local path="$1" label="$2" remedy="$3"
+  if [[ -d "${path}" ]]; then pass "${label}: ${path}"; else fail_check "${label} is missing — ${remedy}"; fi
+}
+
+case "${mode}" in
+  local|live) ;;
+  --help|-h) usage; exit 0 ;;
+  *) usage >&2; exit 2 ;;
+esac
+
+echo "Sentinel judge demo doctor"
+echo "  Mode:      ${mode}"
+echo "  Read-only: yes"
+echo "  Argus:     ${repo_root}"
+echo
+echo "==> Repository layout"
+check_directory "${repo_root}/agent" "Argus" "run this command from the argus-k8s checkout"
+check_directory "${phoenix_root}/dashboard" "Phoenix" "clone Phoenix at ../sentinel-stack/phoenix or set PHOENIX_ROOT"
+check_directory "${sentinel_root}/dashboard" "Sentinel" "clone Sentinel at ../sentinel-stack/sentinel or set SENTINEL_ROOT"
+check_directory "${platform_root}/world_model" "Sentinel Platform / SOG" "clone Sentinel Platform at ../sentinel-stack/sentinel-platform or set SENTINEL_PLATFORM_ROOT"
+
+echo
+echo "==> Required tools"
+for tool in bash curl jq npm python3 lsof; do
+  check_command "${tool}" "install ${tool} and retry"
+done
+
+if [[ -n "${OPENAI_API_KEY:-}" ]] || { [[ -f "${repo_root}/.env" ]] && grep -Eq '^[[:space:]]*OPENAI_API_KEY=.+' "${repo_root}/.env"; }; then
+  pass "OPENAI_API_KEY is configured (value not displayed)"
+else
+  warn "OPENAI_API_KEY is not configured — deterministic proof works, but the OpenAI evidence briefing will be unavailable"
+fi
+
+if [[ "${mode}" == "local" ]]; then
+  echo
+  echo "==> Portable demo runtime"
+  check_command docker "install Docker or start OrbStack"
+  if command -v docker >/dev/null 2>&1 && docker info >/dev/null 2>&1; then
+    pass "Docker-compatible runtime is reachable"
+  else
+    fail_check "Docker-compatible runtime is not reachable — start Docker or OrbStack"
+  fi
+else
+  echo
+  echo "==> Local console dependencies"
+  [[ -x "${repo_root}/.venv/bin/python" ]] && pass "Argus Python environment is installed" || fail_check "Argus Python environment is missing — run: make setup-local"
+  [[ -d "${repo_root}/ui/node_modules" ]] && pass "Argus UI dependencies are installed" || fail_check "Argus UI dependencies are missing — run: make setup-local"
+  [[ -x "${sentinel_root}/.venv/bin/python" ]] && pass "Sentinel Python environment is installed" || fail_check "Sentinel Python environment is missing — run: make -C ${sentinel_root} setup-local"
+  [[ -d "${sentinel_root}/dashboard/node_modules" ]] && pass "Sentinel UI dependencies are installed" || fail_check "Sentinel UI dependencies are missing — run Sentinel setup-local"
+  [[ -d "${phoenix_root}/dashboard/node_modules" ]] && pass "Phoenix UI dependencies are installed" || fail_check "Phoenix UI dependencies are missing — run: npm --prefix ${phoenix_root}/dashboard install"
+
+  echo
+  echo "==> Live k3s stack"
+  check_command kubectl "install kubectl and configure the argus context"
+  if command -v kubectl >/dev/null 2>&1; then
+    if LIVE_DEMO_DRY_RUN=true PHOENIX_ROOT="${phoenix_root}" SENTINEL_ROOT="${sentinel_root}" \
+      bash "${repo_root}/scripts/demo-platform-live-proof.sh"; then
+      pass "live security, chaos, agent, and SOG preflight passed"
+    else
+      fail_check "live stack preflight failed — follow the component named above, then retry"
+    fi
+  fi
+fi
+
+echo
+echo "==> Verdict"
+if ((failures > 0)); then
+  echo "NOT READY — ${failures} blocking check(s), ${warnings} warning(s)."
+  exit 1
+fi
+
+echo "READY — 0 blocking checks, ${warnings} warning(s)."
+if [[ "${mode}" == "local" ]]; then
+  echo "Next: make demo-platform"
+else
+  echo "Next: make demo-platform-live"
+  echo "The live command will still require the exact context and INJECT LIVE FAULT."
+fi