From 81b8662f7534d241b525ca1718495173d25afb9c Mon Sep 17 00:00:00 2001 From: Giovanni Giovanni Date: Mon, 24 Aug 2026 21:57:55 -0700 Subject: [PATCH] CAL-BENCHKIT-RETIRE: afi-docs: remove afi-benchkit from the architecture record; point analyst calibration at afi.analyst-calibration.v1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CAL-GOV D-CAL-6(3), slot CAL-BENCHKIT-RETIRE (afi-governance PR #53 merged 2026-08-25; authorization recorded in PR #54): AFI_Full_Architecture.md (repository table, role grouping 18→17, support list, researcher audience, the benchkit paragraph → the governed analyst calibration record; structure count 13→12), ARCHITECTURE_STATUS.md, the three specs, the Atlas mirror re-vendored byte-identical to afi-config PR #89 (PROVENANCE.txt sha updated), and scripts/check_architecture_doc.py (afi-benchkit → REMOVED_REPOS; count literal 17; STALE_COUNTS gains 18). Both checkers PASS. PoI / PoInsight are untouched (D-CAL-5(4)): they remain the reserved on-chain per-analyst reputation primitives; the calibration record is their off-chain input. Co-Authored-By: Claude Fable 5 --- AFI_Full_Architecture.md | 18 ++++++------------ ARCHITECTURE_STATUS.md | 2 +- atlas/PROVENANCE.txt | 2 +- atlas/afi-protocol-atlas.v1.json | 19 +------------------ scripts/check_architecture_doc.py | 10 +++++----- specs/AFI_ANALYST_SHOP_MVP.md | 2 +- specs/AFI_PORTABLE_PROTOCOL_SURFACE.v0.1.md | 4 ++-- ...EARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md | 3 +-- 8 files changed, 18 insertions(+), 42 deletions(-) diff --git a/AFI_Full_Architecture.md b/AFI_Full_Architecture.md index 2a55f45..c1876f5 100644 --- a/AFI_Full_Architecture.md +++ b/AFI_Full_Architecture.md @@ -26,7 +26,7 @@ A governed requirement is not automatically implemented; implemented code is not ## The organization -The AFI organization is exactly **18 repositories** — 17 public and 1 private. They fall into five roles. +The AFI organization is exactly **17 repositories** — 16 public and 1 private. They fall into five roles. | Repository | Current role | Authority status | Runtime status | |---|---|---|---| @@ -44,12 +44,11 @@ The AFI organization is exactly **18 repositories** — 17 public and 1 private. | **afi-docs** | Documentation hub | Reference | Not runtime | | **afi-tiny-brains** *(private)* | Bounded first-party aiMl orchestration service: one internal orchestrator runs the ProviderInstance-selected profile of approved experts through a deterministic resolver, fail-closed (does not affect UWR scoring); also hosts the deterministic pattern kernel | Support | Bounded, fenced enrichment | | **afi-econ** | Economic research kit (non-canonical) | Research | Not runtime | -| **afi-benchkit** | PoI / PoInsight benchmark + reproducibility harness | Research | Standalone benchmark tool | | **afi-artifacts** | Frozen, DOI-minted paper reproducibility bundle | Reference | Frozen snapshot | | **afi-protocol** | Public flagship repository map ("start here") | Reference | Not runtime | | **.github** | Organization profile and defaults | Reference | Not runtime | -Grouped by role: **3** normative + **6** governed implementation + **4** support/reference (`afi-factory`, `afi-xerc20`, `afi-docs`, `afi-tiny-brains`) + **3** research/records (`afi-econ`, `afi-benchkit`, `afi-artifacts`) + **2** organization surfaces (`afi-protocol`, `.github`) = **18**. +Grouped by role: **3** normative + **6** governed implementation + **4** support/reference (`afi-factory`, `afi-xerc20`, `afi-docs`, `afi-tiny-brains`) + **2** research/records (`afi-econ`, `afi-artifacts`) + **2** organization surfaces (`afi-protocol`, `.github`) = **17**. --- @@ -85,7 +84,7 @@ The precedence order is governance → config → math → implementation → te **Governed implementation** is carried by six repositories — `afi-core`, `afi-reactor`, `afi-infra`, `afi-gateway`, `afi-mint`, `afi-token` — each replaceable by any implementation honoring the same contracts and handoffs. -**Support, research, and reference** repositories (`afi-factory`, `afi-xerc20`, `afi-docs`, `afi-tiny-brains`, `afi-econ`, `afi-benchkit`, `afi-artifacts`, `afi-protocol`, `.github`) never hold normative authority. +**Support, research, and reference** repositories (`afi-factory`, `afi-xerc20`, `afi-docs`, `afi-tiny-brains`, `afi-econ`, `afi-artifacts`, `afi-protocol`, `.github`) never hold normative authority. ### Canonical objects and identity **(Governed)** @@ -200,12 +199,7 @@ Every record persisted today carries `lifecycleState = SCORED` and `finalized = ### 8. Benchmarking and research plane **(Research)** -`afi-benchkit` is a standalone public benchmark and reproducibility harness with no private-repository dependency (a guardrail test enforces this). - -- It provides two seeded suites: **PoI** (capability — coverage, vitality, determinism, latency) and **PoInsight** (usefulness — information coefficient, hit rate, Sharpe over 1h/4h horizons). -- It is deterministic and reproducible: fixed seeds, stamped artifacts (data hash + git SHA + timestamp), golden-hash-locked plots, an `afi-bench` CLI, and a hash-locked container **published by CI to `ghcr.io/afi-protocol/afi-benchkit`**. -- Its input is **static CSV fixtures**; it has **no live finalized-receipt feed and no production reputation or incentive integration**. Its composite reputation number (a weighted combination of PoI and PoInsight) is a local research computation over benchmark outputs, **not** the protocol's reputation or incentive law. Fixture-backed PoInsight is not PoInsight computed from finalized live receipts. -- Current test state: outside the pinned container, exactly **two** golden-image plot tests fail (`test_calibration_plot_golden`, `test_poi_capability_golden`) as SHA-256 mismatches from a matplotlib version difference; the remaining benchmark tests pass. +Analyst calibration is recorded by the governed **analyst calibration record** (`afi.analyst-calibration.v1`, afi-config `schemas/analyst-calibration/v1`; builder in afi-reactor; CAL-GOV) — a sealed, per-analyst, append-only record of counts, rates with cluster-robust intervals, and a reliability table, never a scalar. PoI and PoInsight remain the reserved on-chain per-analyst reputation primitives (CONST-GOV D-CONST-5); the calibration record is their off-chain evidentiary input and names no member for either. `afi-econ` is a non-canonical economic research kit (its models are self-declared placeholder/toy and are not protocol law unless promoted by governance). `afi-artifacts` is a frozen, DOI-minted paper reproducibility bundle (Zenodo). **AFI Research Institute** is AFI's sole research and legal identity — the copyright holder across the organization's licenses — reflecting an open-protocol, auditable, replayable research posture. Committed to open research, it is **designated to operate AFI's official open reference services (Reference)**: a hosted **Gateway reference service** (§4) and a designated **oracle-ingress / CPJ-normalization reference service** (§5), governed by `research-institute-reference-services-v0.1` (INST-GOV) and specified in [`specs/AFI_RESEARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md`](specs/AFI_RESEARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md). These are **non-exclusive** reference instances — operating them confers no protocol authority, no economic or validator privilege, and no exclusive network role; independent conforming operators remain free to run their own ingress and collectors. No Institute-hosted deployment is claimed (none exists). @@ -326,7 +320,7 @@ The **District / API Atlas** is AFI's canonical **discoverability and relationsh **The Atlas describes; it does not decide or execute.** It replaces no runtime, schema, contract, or governance authority (ATLAS-GOV D-ATLAS-1). Where the Atlas and a higher authority disagree, the higher authority wins and the Atlas is the surface to correct. It runs no graph, calls no provider, scores nothing, builds no evidence, routes no live traffic, resolves no secret, activates no participant, and transfers no value. -The registry records **2 active Districts** (plus **1 reserved** capability-domain), **6 capabilities**, **13 structures**, **16 interfaces**, **9 typed routes**, **21 referenced contracts**, **18 repositories**, and **7 participant roles** with onboarding descriptors. +The registry records **2 active Districts** (plus **1 reserved** capability-domain), **6 capabilities**, **12 structures**, **16 interfaces**, **9 typed routes**, **21 referenced contracts**, **17 repositories**, and **7 participant roles** with onboarding descriptors. ### Districts (from the registry) @@ -371,6 +365,6 @@ The Atlas is not the runtime (it describes the runtime, and the runtime authorit | **Analyst / strategy author** | the UWR profile registry and KATs in `afi-config`; author pipelines and configs with `afi-factory`. | | **Validator** | the `scored-signal-evidence` v3 schema and vectors in `afi-config`; store semantics in `afi-infra`. | | **Operator** | `afi-gateway` (submission boundary) and `afi-infra` (canonical store). | -| **Researcher** | `afi-docs`, `afi-econ`, `afi-benchkit`, and the frozen record in `afi-artifacts`. | +| **Researcher** | `afi-docs`, `afi-econ`, and the frozen record in `afi-artifacts`. | Conformance is defined by the contracts, registries, and KATs — a conforming analyst pipeline needs no permission. Authority lives in `afi-governance`, `afi-config`, and `afi-math`; everything else is a replaceable implementation of those contracts. diff --git a/ARCHITECTURE_STATUS.md b/ARCHITECTURE_STATUS.md index 975e93a..37806ea 100644 --- a/ARCHITECTURE_STATUS.md +++ b/ARCHITECTURE_STATUS.md @@ -66,7 +66,7 @@ Implementation lives under: | Canonical evidence store | `afi-infra` | | On-chain mint | `afi-token` | | Off-chain mint coordination | `afi-mint` | -| Validator benchmarks | `afi-benchkit` | +| Analyst calibration record | `afi-config` (`afi.analyst-calibration.v1`) + `afi-reactor` (builder) | --- diff --git a/atlas/PROVENANCE.txt b/atlas/PROVENANCE.txt index da6b0a2..99730c3 100644 --- a/atlas/PROVENANCE.txt +++ b/atlas/PROVENANCE.txt @@ -1 +1 @@ -c754841f6df1fac9d61e9bca6c79a8d06fba97786d55f20ca943e5b16eb45ff7 666b2475a72153416ef2262b84089d11f395a1de +e6f633d8297c44a774587fbd4449a2e4246ff5112b37a0827ec75c0e4b6e22f6 666b2475a72153416ef2262b84089d11f395a1de diff --git a/atlas/afi-protocol-atlas.v1.json b/atlas/afi-protocol-atlas.v1.json index 476edb4..267a0bd 100644 --- a/atlas/afi-protocol-atlas.v1.json +++ b/atlas/afi-protocol-atlas.v1.json @@ -406,22 +406,6 @@ { "kind": "repoPath", "ref": "afi-mint:src/orchestrator/MintExecutor.ts:32", "note": "abstract IMintCoordinatorContract; not wired on-chain" }, { "kind": "test", "ref": "afi-mint:test/emissions-schedule-parity.test.ts", "note": "86B parity vs afi-math" } ] - }, - { - "structureId": "afi-benchkit", - "name": "afi-benchkit benchmark harness", - "kind": "benchmark", - "purpose": "A standalone deterministic PoI/PoInsight benchmark and reproducibility harness over static CSV fixtures. Research surface: it has no live finalized-receipt feed and no protocol reputation or incentive integration; its composite reputation number is a local research computation, not protocol law.", - "maturity": "partial", - "visibility": "public", - "capabilityRefs": [], - "servedDistrictRefs": [], - "repositoryRefs": ["repo-afi-benchkit"], - "exposedInterfaceRefs": [], - "routingInterfaceRefs": [], - "evidence": [ - { "kind": "repoPath", "ref": "afi-benchkit:src/afi_benchkit/cli.py", "note": "afi-bench CLI; two golden-plot tests fail pre-existing" } - ] } ], @@ -864,7 +848,6 @@ { "repositoryId": "repo-afi-token", "identity": "AFI-Protocol/afi-token", "role": "On-chain token (86B hard cap xERC20); Base Sepolia testnet; reserved settlement domain", "hostedStructureRefs": ["afi-token"], "implementedCapabilityRefs": [], "visibility": "public" }, { "repositoryId": "repo-afi-xerc20", "identity": "AFI-Protocol/afi-xerc20", "role": "Vendored xERC20 standard (dependency of afi-token)", "hostedStructureRefs": [], "implementedCapabilityRefs": [], "visibility": "public" }, { "repositoryId": "repo-afi-econ", "identity": "AFI-Protocol/afi-econ", "role": "Economic research kit (non-canonical, placeholder)", "hostedStructureRefs": [], "implementedCapabilityRefs": [], "visibility": "public" }, - { "repositoryId": "repo-afi-benchkit", "identity": "AFI-Protocol/afi-benchkit", "role": "PoI / PoInsight benchmark + reproducibility harness (research)", "hostedStructureRefs": ["afi-benchkit"], "implementedCapabilityRefs": [], "visibility": "public" }, { "repositoryId": "repo-afi-artifacts", "identity": "AFI-Protocol/afi-artifacts", "role": "Frozen, DOI-minted paper reproducibility bundle", "hostedStructureRefs": [], "implementedCapabilityRefs": [], "visibility": "public" }, { "repositoryId": "repo-afi-docs", "identity": "AFI-Protocol/afi-docs", "role": "Documentation hub; single authoritative current architecture map", "hostedStructureRefs": ["docs-hub"], "implementedCapabilityRefs": [], "visibility": "public" }, { "repositoryId": "repo-afi-protocol", "identity": "AFI-Protocol/afi-protocol", "role": "Public flagship repository/navigation map (start here)", "hostedStructureRefs": [], "implementedCapabilityRefs": [], "visibility": "public" }, @@ -1057,7 +1040,7 @@ "audienceDescriptors": [ { "audienceId": "audience-investor", "name": "Investor", "note": "A non-conformance audience descriptor only. 'Investor' is NOT an AFI protocol-conformance role and implies no protocol activation, validator/provider privilege, or economic authority (ATLAS-GOV D-ATLAS-10)." }, - { "audienceId": "audience-researcher", "name": "Researcher", "note": "A non-conformance audience descriptor: consumes afi-benchkit, afi-artifacts, and afi-docs. Implies no protocol activation or conformance role." }, + { "audienceId": "audience-researcher", "name": "Researcher", "note": "A non-conformance audience descriptor: consumes afi-artifacts and afi-docs. Implies no protocol activation or conformance role." }, { "audienceId": "audience-observer", "name": "Ecosystem observer", "note": "A non-conformance audience descriptor for ecosystem observers; implies no protocol activation or conformance role." } ] } diff --git a/scripts/check_architecture_doc.py b/scripts/check_architecture_doc.py index 076e127..dc1a246 100644 --- a/scripts/check_architecture_doc.py +++ b/scripts/check_architecture_doc.py @@ -26,7 +26,7 @@ "afi-governance", "afi-config", "afi-math", "afi-core", "afi-reactor", "afi-infra", "afi-gateway", "afi-mint", "afi-token", "afi-factory", "afi-xerc20", "afi-docs", "afi-tiny-brains", - "afi-econ", "afi-benchkit", "afi-artifacts", + "afi-econ", "afi-artifacts", "afi-protocol", ".github", ] @@ -36,7 +36,7 @@ # repo) does not trip the "afi-ops" pattern. REMOVED_REPOS = [ "afi-assets", "afi-plugins", "afi-cli-framework", - "afi-skills", "afi-labs", "afi-ops", + "afi-skills", "afi-labs", "afi-ops", "afi-benchkit", ] # Historical-transition vocabulary. The document is present-tense only. @@ -46,7 +46,7 @@ ] # Stale organization counts that must not appear. -STALE_COUNTS = [r"\b19 repositor", r"\b20 repositor", r"\b21 repositor", r"\b22 repositor"] +STALE_COUNTS = [r"\b18 repositor", r"\b19 repositor", r"\b20 repositor", r"\b21 repositor", r"\b22 repositor"] # District/Atlas anchors. Originally mandated by D1CAP-GOV D-D1CAP-8 item 3; # AMENDED by ATLAS-GOV D-ATLAS-9(3) (afi-governance @@ -106,8 +106,8 @@ def main() -> int: line = low.count("\n", 0, m.start()) + 1 failures.append(f"stale org count: {text[m.start():m.start()+16]!r} (line {line})") - if not re.search(r"\b18 repositor", low): - failures.append("missing the current organization count ('18 repositories')") + if not re.search(r"\b17 repositor", low): + failures.append("missing the current organization count ('17 repositories')") for repo in CURRENT_REPOS: if repo not in text: diff --git a/specs/AFI_ANALYST_SHOP_MVP.md b/specs/AFI_ANALYST_SHOP_MVP.md index 4eff132..395a798 100644 --- a/specs/AFI_ANALYST_SHOP_MVP.md +++ b/specs/AFI_ANALYST_SHOP_MVP.md @@ -161,7 +161,7 @@ These reduce time-to-shop more than Ably: | TradingView | Webhook URL + secret + field mapping doc | USS v1.1 (see `afi-reactor` TV path) | | Telegram | Bot + CPJ parser preset | CPJ v0.1 → USS | | REST / SDK | `POST /api/v1/signals` + OpenAPI snippet | Gateway ingest | -| Manual | CLI `afi submit` for testnet | Demo / benchkit | +| Manual | CLI `afi submit` for testnet | Demo | **Priority for T1:** TradingView + Telegram templates (highest analyst channel overlap). diff --git a/specs/AFI_PORTABLE_PROTOCOL_SURFACE.v0.1.md b/specs/AFI_PORTABLE_PROTOCOL_SURFACE.v0.1.md index 1d9537a..8b547c2 100644 --- a/specs/AFI_PORTABLE_PROTOCOL_SURFACE.v0.1.md +++ b/specs/AFI_PORTABLE_PROTOCOL_SURFACE.v0.1.md @@ -68,7 +68,7 @@ AFI is best understood as **separated planes**, not one monolithic stack: | Immutable audit (server logs, CDN edge) | BASE commitments (events, receipts) | | Message body | Off-chain evidence vault (operator’s choice) | | Client (curl, browser, fetch) | Reactor, custom Python, any conforming orchestrator | -| CDN / analytics / log pipeline | warehouses, streams, benchkit, research tooling | +| CDN / analytics / log pipeline | warehouses, streams, research tooling | HTTP did not mandate Apache. AFI must not mandate Mongo or `afi-reactor`—only **conforming artifacts and behavior**. @@ -175,7 +175,7 @@ Audit **all repositories** in the AFI-Protocol GitHub organization—public **an **Known local workspace repos (non-exhaustive):** -`afi-artifacts`, `afi-benchkit`, `afi-config`, `afi-core`, `afi-docs`, `afi-econ`, `afi-factory`, `afi-gateway`, `afi-governance`, `afi-infra`, `afi-math`, `afi-mint`, `afi-protocol`, `afi-reactor`, `afi-research-site`, `afi-tiny-brains`, `afi-token`, `afi-xerc20` +`afi-artifacts`, `afi-config`, `afi-core`, `afi-docs`, `afi-econ`, `afi-factory`, `afi-gateway`, `afi-governance`, `afi-infra`, `afi-math`, `afi-mint`, `afi-protocol`, `afi-reactor`, `afi-research-site`, `afi-tiny-brains`, `afi-token`, `afi-xerc20` Also check for renamed, archived, or external-reference repos (e.g. historical `afi-pipeline` references in older docs). diff --git a/specs/AFI_RESEARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md b/specs/AFI_RESEARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md index b108c28..b0b01df 100644 --- a/specs/AFI_RESEARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md +++ b/specs/AFI_RESEARCH_INSTITUTE_REFERENCE_SERVICES.v0.1.md @@ -215,8 +215,7 @@ instructions are within the designated services' scope. This role does **not** imply that every submitted signal, raw message, or proprietary alpha artifact is public, and it does **not** claim any production PoInsight or reputation integration exists — those remain research surfaces -([`afi-benchkit`](https://github.com/AFI-Protocol/afi-benchkit) computes PoInsight from -fixtures only). Institute-operated services may set their own transparent research and +(the per-analyst off-chain input is the governed analyst calibration record, `afi.analyst-calibration.v1` — CAL-GOV; PoI and PoInsight themselves remain reserved primitives, CONST-GOV D-CONST-5). Institute-operated services may set their own transparent research and privacy policies (retention, PII, consent, disclosure, access control); those policies apply to Institute-operated instances and do not become protocol law. Raw source content is not automatically written on-chain and is not automatically published.