Developer portal + governed compute-mesh plane (closes #17)

.gitignore

@@ -15,6 +15,9 @@ node_modules/
 # Promotion-gate runtime evidence (emitted at rollout, not committed; dir kept via .gitkeep)
 artifacts/gate-decisions/*.json
 
+# Live mesh telemetry heartbeats (emitted by nodes at runtime, not committed; dir kept via .gitkeep)
+artifacts/mesh-heartbeats/*.json
+
 # Porter control-plane build outputs (compiled locally, not committed)
 artifacts/porter-shim/shim
 artifacts/cloudshell-hardened-pack/culler/culler

Makefile

@@ -1,10 +1,32 @@
 # SourceOS Continuum — lifecycle entry points.
 # Control-plane targets delegate to Makefile.porter (the rehomed Porter control plane).
-.PHONY: validate onboard dev-up dev-down shim-test test rollout promotion-gate
+.PHONY: validate onboard dev-up dev-down shim-test test tools-test rollout promotion-gate portal compute
 
 validate: ## repo hygiene + CapD validity
 	python3 tools/validate.py
 
+tools-test: ## unit-test the governed tools (portal router, compute plane, promotion gate, MCP surface)
+	cd tools && python3 -m pytest -q
+
+portal: ## developer portal: read-only web console over the governed surface (scale-to-zero, stdlib only)
+	python3 tools/portal_server.py $(PORT)
+
+compute: ## compute plane: route one workload across the mesh under per-project policy + availability
+	python3 tools/compute_plane.py
+
+mesh-demo: ## seed a live demo mesh (heartbeats) so the portal shows live telemetry
+	@python3 tools/mesh_telemetry.py heartbeat artifacts/mesh-heartbeats k8s-a k8s 8 >/dev/null
+	@python3 tools/mesh_telemetry.py heartbeat artifacts/mesh-heartbeats slurm-login hpc-slurm 120 >/dev/null
+	@python3 tools/mesh_telemetry.py heartbeat artifacts/mesh-heartbeats edge-1 wasm-edge 20 >/dev/null
+	@python3 tools/mesh_telemetry.py heartbeat artifacts/mesh-heartbeats boinc-grid volunteer-boinc 400 >/dev/null
+	@python3 tools/mesh_telemetry.py view artifacts/mesh-heartbeats
+
+grant: ## demo the zero-trust attach flow: Attest -> Decide -> Grant -> verify-at-node
+	python3 tools/mcp_a2a_grant.py
+
+commons: ## reproducible knowledge commons: ingest the estate's CapDs + workloads as citable records
+	python3 tools/commons.py
+
 onboard: ## bring up a workstation: local sovereign forge + local cluster + sourceosctl
 	@echo "[continuum] onboard — scaffold: wires Gitea bring-up + kind/k3s + sourceos-devtools/sourceosctl"
 

capd/cloudshell-fog.capd.json

@@ -0,0 +1,49 @@
+{
+  "capability_id": "caps.compute.cloudshell-fog@0.1.0",
+  "kind": "compute.zero-trust-fog",
+  "status": "experimental",
+  "name": "Sovereign Agentic Cloud-Shell — zero-trust MCP-A2A fog deployment",
+  "description": "The full Attest -> Decide -> Grant -> verify-at-node -> enforce-constraints flow from the cloud-shell fog spec, made first-class over the continuum. A low-mem box starts a session; the Control-Plane Agent (compute_plane.place) decides a fog node from live HyperSwarm discovery (mesh_telemetry); the MCP-A2A Policy Authority attests (TPM/TEE + cosign) then mints a signed, session-bound Grant carrying its constraints (mcp_a2a_grant.issue_grant); the fog node's Policy Gate re-verifies that Grant on attach and on every PTY/FS op (mcp_a2a_grant.verify_grant). Fail-closed at every step; every decision sealed to the append-only ledger.",
+  "links": {
+    "control_plane_agent": "tools/compute_plane.py",
+    "hyperswarm_discovery": "tools/mesh_telemetry.py",
+    "policy_authority": "tools/mcp_a2a_grant.py",
+    "policy_gate": "tools/mcp_a2a_grant.py",
+    "capability_registry": "capd/",
+    "agent_surface": "tools/mcp_ops_server.py",
+    "portal": "tools/portal_server.py",
+    "supply_chain_gate": "prophet-platform/tools/advisory_check.py + attest.py (estate)",
+    "ledger": "artifacts/gate-decisions + mcp-receipts (sealed)",
+    "spec_witness": "docs/CLOUDSHELL_FOG.md",
+    "shape_authority": "SourceOS-Linux/mcp-a2a-zero-trust (canonical Grant/AttestationBundle/QuorumProof; vendored hash-pinned at schemas/a2a/)"
+  },
+  "composes_with": {
+    "compute_plane": "caps.compute.mesh-plane@0.1.0",
+    "inner_loop": "caps.dev.devspace-inner-loop@0.1.0",
+    "control_plane": "caps.infra.paas.continuum-local@0.1.0",
+    "scales_up_to": "caps.infra.cluster-scaleup.hyperswarm@0.1.0"
+  },
+  "policy": {
+    "zero_trust": true,
+    "attest_decide_grant": true,
+    "fail_closed": true,
+    "evidence_emitting": true,
+    "session_bound_grants": true,
+    "quorum_when_required": true,
+    "read_only_portal": true
+  },
+  "flow": [
+    "0  login / refresh (IdP OIDC/OAuth2 + optional FIDO2)",
+    "1  start shell / agent session (Browser Terminal UI over WSS/WebTransport)",
+    "2  SessionRequest: capability + agent (Console API -> Control-Plane Agent)",
+    "3  resolve CapD + MCP tool schemas (Capability Registry)",
+    "4  verify artifact: OCI digest + SBOM + signature (Supply-chain Gate)",
+    "5  find candidate nodes (HyperSwarm Mesh discovery / mesh_telemetry)",
+    "6  route to selected node",
+    "7  policy eval: user + node + capability (MCP-A2A Policy Authority)",
+    "8  issue Grant, session-bound; +QuorumProof when required (Key Authority HSM/KMS)",
+    "9  TriRPC attach over a Grant-bound channel (Edge Gateway <-> Fog Node)",
+    "10 authorize attach (fog-node MCP-A2A Policy Gate: verify Grant)",
+    "11 allow PTY/FS ops, enforce constraints (Shell Runtime Container/Pod sandbox)"
+  ]
+}

capd/compute-plane.mesh.capd.json

@@ -0,0 +1,27 @@
+{
+  "capability_id": "caps.compute.mesh-plane@0.1.0",
+  "kind": "compute.placement-plane",
+  "status": "experimental",
+  "name": "Compute plane — one governed door to any substrate",
+  "description": "A user develops on a low-mem box and the same workload scales out, seamlessly, over whatever the mesh offers: a k8s service, an HPC/SLURM supercomputer, WASM at the edge, a p2p/hyperswarm mesh, volunteer compute (BOINC / Folding@home / open-HEP-style), or an RLC-style blockchain compute market. The substrate is chosen by the placement broker from per-project/per-account policy and live mesh availability, scaling out where it can and where volunteer compute is offered. Governed and fail-closed: sensitive workloads never land on an untrusted (volunteer/p2p/blockchain) backend, every placement is sealed, and with no allowed+available backend it falls back to local or blocks rather than shipping work somewhere the policy forbids. Configured in the portal dashboard.",
+  "links": {
+    "broker": "tools/compute_plane.py",
+    "portal": "tools/portal_server.py",
+    "agent_surface": "tools/mcp_ops_server.py",
+    "reference_pattern": "BOINC computing-preferences + IBM Parallel Environment (HPC toolkit) + Docker volume drivers — met and bettered: one governed plane across all substrates, not per-substrate config silos"
+  },
+  "composes_with": {
+    "inner_loop": "caps.dev.devspace-inner-loop@0.1.0",
+    "control_plane": "caps.infra.paas.continuum-local@0.1.0",
+    "scales_up_to": "caps.infra.cluster-scaleup.hyperswarm@0.1.0"
+  },
+  "policy": {
+    "fail_closed": true,
+    "evidence_emitting": true,
+    "sensitive_never_untrusted": true,
+    "per_project_config": true,
+    "per_account_config": true,
+    "read_only_portal": true
+  },
+  "backends": ["local", "k8s", "hpc-slurm", "wasm-edge", "p2p-mesh", "volunteer-boinc", "blockchain-rlc"]
+}

capd/devspace.local-dev.capd.json

@@ -0,0 +1,24 @@
+{
+  "capability_id": "caps.dev.devspace-inner-loop@0.1.0",
+  "kind": "dev.inner-loop",
+  "status": "experimental",
+  "name": "SourceOS DevSpace — inner-loop dev-environment",
+  "description": "Nocalhost-style inner-loop over the continuum: an isolated per-user namespace, hot code sync into a running workload, a sidecar dev container, and port-forward + local debug — shortening the modify->result feedback loop. Consumes the control plane; surfaced to humans via the portal and to agents via the MCP ops surface (one governed source, two views).",
+  "links": {
+    "portal": "tools/portal_server.py",
+    "agent_surface": "tools/mcp_ops_server.py",
+    "reference_pattern": "Nocalhost DevSpace (inner-loop) — met and bettered: sovereign + evidence-emitting"
+  },
+  "composes_with": {
+    "control_plane": "caps.infra.paas.continuum-local@0.1.0",
+    "operator_surface": "SourceOS-Linux/sourceos-devtools",
+    "compute_plane": "caps.compute.mesh-plane@0.1.0",
+    "scales_up_to": "caps.infra.cluster-scaleup.hyperswarm@0.1.0"
+  },
+  "policy": {
+    "local_first": true,
+    "evidence_emitting": true,
+    "isolation": "per-user-namespace",
+    "read_only_portal": true
+  }
+}

capd/knowledge-commons.mesh.capd.json

@@ -0,0 +1,30 @@
+{
+  "capability_id": "caps.knowledge.commons@0.1.0",
+  "kind": "knowledge.reproducible-commons",
+  "status": "experimental",
+  "name": "Reproducible Knowledge Commons",
+  "description": "A Zenodo-style citable, content-addressed, reproducible, reusable layer over the whole estate. Every capability, workload, model, dataset, or semantic action becomes a commons record with a DOI-like id (commons:<domain>/<name>@<version>+<digest>), its reproducibility provenance, and its reuse history. Folds three demonstrated systems into one governed plane: Zenodo/reproducible-fusion (content-addressed + citable + reproducibility-graded, fail-closed on the claim), ARM / Asset Reuse Manager (Domain->Category->Asset + recommendation + use/evaluate feedback), and the Semantic API/ontology (records carry a declarative semantic_action). Ingests the estate's own CapDs + suite workloads, so the compute-mesh / cloud-shell fog work is itself first-class citable here.",
+  "links": {
+    "engine": "tools/commons.py",
+    "portal": "tools/portal_server.py",
+    "agent_surface": "tools/mcp_ops_server.py",
+    "ledger": "artifacts/gate-decisions + mcp-receipts (sealed)",
+    "spec_witness": "docs/KNOWLEDGE_COMMONS.md",
+    "reference_pattern": "IBM Asset Reuse Manager (ARM) + MLOps 3-phase reproducible pipeline + Semantic API/ontology — met and bettered: one governed, content-addressed, reproducibility-graded commons instead of three separate systems"
+  },
+  "composes_with": {
+    "compute_plane": "caps.compute.mesh-plane@0.1.0",
+    "fog": "caps.compute.cloudshell-fog@0.1.0",
+    "control_plane": "caps.infra.paas.continuum-local@0.1.0",
+    "scales_up_to": "caps.infra.cluster-scaleup.hyperswarm@0.1.0"
+  },
+  "policy": {
+    "content_addressed": true,
+    "citable": true,
+    "reproducibility_gate": true,
+    "arm_reuse_feedback": true,
+    "semantic_action_catalog": true,
+    "evidence_emitting": true,
+    "read_only_portal": true
+  }
+}

docs/CLOUDSHELL_FOG.md

@@ -0,0 +1,58 @@
+# Sovereign Agentic Cloud-Shell — fog deployment, made first-class
+
+This witnesses the cloud-shell fog spec (*Zero-Trust MCP-A2A + TriRPC transport + HyperSwarm
+placement*) against its implementation in this repo. Every box in the diagram points at real code —
+no box is decoration.
+
+## Planes → implementation
+
+| Spec plane / box | Implementation | Status |
+|---|---|---|
+| **Capability Build / Packaging** — OCI image (digest+SBOM+sig), publish CapD + tool schema | estate build + `attest.py` (SLSA/in-toto/DSSE), `capd/*.capd.json` | estate |
+| **Capability Registry** (CapDs + MCP tool catalog) | `capd/` + `tools/mcp_ops_server.py` | ✅ |
+| **Control Plane Agent** (placement + quotas + admission; session lifecycle) | `tools/compute_plane.py` — `place()` is the *Decide* | ✅ |
+| **HyperSwarm Mesh** (discovery + gossip + rendezvous) / *find candidate nodes* | `tools/mesh_telemetry.py` — live liveness registry | ✅ |
+| **Supply-chain Gate** (OCI digest + SBOM + signatures) | `prophet-platform/tools/advisory_check.py` + `attest.py` | estate |
+| **MCP-A2A Policy Authority** (Attest → Decide → Grant, +QuorumProof) | `tools/mcp_a2a_grant.py` — `issue_grant()` | ✅ |
+| **Key Authority** (HSM/KMS — sign grants / derive session keys) | `hmac_signer` interface in `mcp_a2a_grant.py` (swap in HSM/ed25519) | ✅ iface |
+| **fog-node MCP-A2A Policy Gate** (verify Grant + bind to session + streams) | `tools/mcp_a2a_grant.py` — `verify_grant()` | ✅ |
+| **Shell Runtime Container/Pod** (PTY + FS sandbox) — enforce constraints | gate returns per-op authorization + redactions; runtime sandbox = estate | ✅ policy |
+| **Attestation Verifier / Node Attestor** (TPM/TEE + cosign) | attestation inputs consumed fail-closed by `issue_grant` | ✅ iface |
+| **Observability Sink** (OTEL) | estate OTel collector (`prophet-platform/deploy/superiority-march/observability`) | estate |
+| **Ledger / Audit** (append-only evidence) | sealed receipts → `artifacts/{gate-decisions,mcp-receipts}` | ✅ |
+| **Browser Terminal UI / Console API** | portal (read-only view) `tools/portal_server.py`; mutations via MCP surface | ✅ view |
+
+## The numbered flow (0–11)
+
+`0` login/refresh (IdP) → `1` start session (UI) → `2` SessionRequest {capability, agent} →
+`3` resolve CapD + tool schemas (registry) → `4` verify artifact digest/SBOM/sig (supply-chain gate) →
+`5` **find candidate nodes** (HyperSwarm / `mesh_telemetry`) → `6` route to selected node →
+`7` **policy eval** user+node+capability (`place()` Decide) → `8` **issue Grant** session-bound,
++QuorumProof when required (`issue_grant`) → `9` TriRPC attach over the Grant-bound channel →
+`10` **authorize attach** (`verify_grant`) → `11` **allow PTY/FS ops**, enforce constraints
+(`verify_grant(requested_op=...)`).
+
+## Why this is zero-trust, not just RBAC
+
+A placement decision is **not** permission to run. The node re-verifies the Grant itself — signature,
+session binding, expiry, attestation, and that the *specific* op is inside the granted constraints —
+on attach **and on every PTY/FS op**. A missing, stale, tampered, or over-reaching Grant is denied,
+fail-closed. Sensitive workloads never even get a Grant for an untrusted (volunteer/p2p/blockchain)
+node — the Decide stage refuses them first (`compute_plane`), and the Authority refuses to mint a
+Grant against a blocked or unattested decision.
+
+See `tools/test_mcp_a2a_grant.py` (Authority + Gate) and `tools/test_compute_plane.py` (Decide).
+
+## Shape conformance — we consume the canonical spec, we do not fork it
+
+The Grant, AttestationBundle, and QuorumProof that `tools/mcp_a2a_grant.py` emits/consumes are the
+**canonical** shapes owned by `SourceOS-Linux/mcp-a2a-zero-trust` (the estate's zero-trust authority
+repo — "owns the zero-trust authority model … grant request, grant decision, and grant ledger
+contracts"). Those schemas are vendored here under `schemas/a2a/`, **hash-pinned** to the authority's
+`schemas/index.json` (see `schemas/a2a/PROVENANCE.md`), and `test_mcp_a2a_grant.py` validates every
+emitted Grant / QuorumProof / AttestationBundle against them. So `issue_grant()` produces a `Grant`
+with the canonical `binding` / `capability` / `constraints` / `policy_hash` / `quorum_proof` /
+`evidence_refs` / `sig{issuer,sig}`, and `verify_grant()` returns a canonical `tool_grant.validate`
+result `{valid, expired, revoked, reason}` — identical to `examples/grant.example.json` and
+`examples/tool_grant_check.example.json` in the authority repo. If the authority updates a schema,
+re-vendor and re-pin; the authority stays the source of truth for the shape.

docs/KNOWLEDGE_COMMONS.md

@@ -0,0 +1,35 @@
+# Reproducible Knowledge Commons — three demonstrated systems, one governed plane
+
+The commons folds the three systems those diagrams show — IBM's **Asset Reuse Manager**, the 3-phase
+**MLOps** reproducible pipeline, and the **Semantic API / ontology** — into one content-addressed,
+citable, reproducibility-graded plane (`tools/commons.py`). It ingests the estate's own CapDs and
+suite workloads, so the compute-mesh / cloud-shell fog work is itself first-class and citable here.
+
+## System → implementation
+
+| Demonstrated system (diagram) | In the commons |
+|---|---|
+| **Zenodo / reproducible-fusion commons** — versioned, citable, DOI-like, reproducible | `mint_id()` → content-addressed `commons:<domain>/<name>@<version>+<digest>`; every record carries a `cite` string |
+| **ARM: Domain → Category → Asset** | every record has `domain` / `category` / `asset_type`; `search()` navigates them |
+| **ARM: Recommendation** | `recommend()` ranks by reuse score, within a domain |
+| **ARM: Use / Evaluate → Feedback** | `record_use(id, outcome)` raises `reuse.score = evaluations / uses` |
+| **MLOps: reproducible pipeline + model registry + monitoring** | the **reproducibility gate**: a record is `reproducible` only if `provenance` carries a `source_digest` **and** an `attestation_ref` or `sbom_digest` — else honestly `declared` |
+| **Semantic API: declarative action + ontology constraints + executor** | a record may carry a `semantic_action` (signature + policy/ontology constraints); the suite workloads ingest with their governing policy as the semantic action |
+
+## The reproducibility gate is fail-closed on the *claim*
+
+This is the point that makes it a *reproducible* commons and not just a catalog: you cannot mint a
+record that claims `reproducible` unless the provenance actually carries what you would need to
+reproduce it. A model deposited with only weights is `declared`; a model deposited with its
+`source_digest` + a cosign/SLSA `attestation_ref` is `reproducible`. The portal shows the split
+honestly (`N reproducible` of the total), so the dashboard never overstates.
+
+## The estate ingests itself
+
+`estate_commons(root)` deposits every `capd/*.capd.json` and every `mesh/suite-workloads.json` entry.
+So `caps.compute.mesh-plane`, `caps.compute.cloudshell-fog`, `caps.dev.devspace-inner-loop`, and the
+five app-suite workloads (Noetica, memory-mesh, TurtleTern, Goose Notes, BearBrowser) are all citable
+commons records. The cloud-shell fog capability ingests as `reproducible` because its CapD references
+the sealed ledger + the mcp-a2a-zero-trust shape authority — provenance enough to reproduce the claim.
+
+See `tools/test_commons.py`. Surfaced read-only at the portal's `/api/commons`.