feat(storage-bench): storage/DB backend perf + integrity benchmark, aligned to the storage standard - #51
Merged
Merged
Conversation
…ligned to the storage standard A benchmark harness for storage backends where INTEGRITY is a first-class result, not a footnote — aligned to SocioProphet/socioprophet-standards-storage (evidence-first: a number is a claim only if backed by a reproducible record). - ts/src/storage-bench.ts: benchObjectBackend measures any ObjectBackend (InMemory/S3-BYOS/RocksDB/…) on put/get/verify — latency p50/p95/p99 + throughput (ops/s, MB/s) — over deterministic seeded content-addressed blobs. FAIL-CLOSED: a round-tripped blob that doesn't byte-match, or whose content hash doesn't re-derive, makes the whole run a failure. toEvaluationRecord emits a record conforming to evaluation-record-standard.v1 (subject_type=platform_capability, attempt_mode=benchmark_run), carrying the raw measurements as evidence. - ts/src/storage-bench.test.ts: 5 tests, teeth both ways — a correct backend passes with full integrity + ordered percentiles; the run is deterministic for a fixed seed; a corrupting backend (wrong bytes) FAILS; a losing backend (missing blob) FAILS; the EvaluationRecord conforms. - docs/STORAGE_BACKEND_MATRIX.md: the Table-2.1 analog — estate backends (Canonical object store, S3-BYOS, RocksDB, Hypercore, AtomSpace, FederatedAtomSpace) + external field, scored on the STANDARD's dimensions (content-addressed / provenance / integrity-at-rest / fault tolerance / sovereign-BYOS / distributed / fail-closed verify) — perf numbers live in EvaluationRecords, not the table. Typecheck clean; storage-bench suite 5/5; dist rebuilt.
Member
Author
|
Merge rationale (self-hosted CI queued, not failing): ran the full CI-equivalent locally — Adversarial self-review: integrity is fail-closed and proven both ways — a corrupting backend (wrong bytes) and a losing backend (missing blob) both FAIL the run, a correct backend passes with full integrity; runs are deterministic for a fixed seed; the EvaluationRecord conforms to evaluation-record-standard.v1 (evidence carried, no bare numbers). Merging on the local proof. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A benchmark suite for storage / DB backends where integrity is a first-class result, aligned to SocioProphet/socioprophet-standards-storage (evidence-first: a number is a claim only if backed by a reproducible record).
ts/src/storage-bench.ts—benchObjectBackendmeasures anyObjectBackend(InMemory / S3-BYOS / RocksDB / …) on put / get / verify: latency p50/p95/p99 + throughput (ops/s, MB/s), over deterministic seeded content-addressed blobs. Fail-closed: a blob that doesn't byte-round-trip, or whose content hash doesn't re-derive, fails the run — a fast backend that returns the wrong bytes fails here, by design.toEvaluationRecordemits a record conforming toevaluation-record-standard.v1(subject_type=platform_capability,attempt_mode=benchmark_run), carrying the raw measurements as evidence.storage-bench.test.ts— 5 tests, teeth both ways: correct backend passes with full integrity + ordered percentiles; deterministic for a fixed seed; a corrupting backend fails; a losing backend fails; the EvaluationRecord conforms.docs/STORAGE_BACKEND_MATRIX.md— the Table-2.1 analog: estate backends (Canonical object store, S3-BYOS, RocksDB, Hypercore, AtomSpace, FederatedAtomSpace) + the external field, scored on the standard's dimensions (content-addressed / provenance / integrity-at-rest·codex / fault tolerance / sovereign-BYOS / distributed / fail-closed verify). Perf numbers live in EvaluationRecords, never inline.Typecheck clean; suite 5/5; dist rebuilt. Runs under the existing
agent-machine/ts-ci test step.🤖 Generated with Claude Code