feat(memory): measure continual learning against stateless runs

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## Unreleased
+
+### Added
+
+- Added `runAgentMemoryLearningExperiment` for matched stateful-versus-stateless memory measurement under one shared cost limit.
+- Added exact paired gain, explicitly labeled transfer probes, and repeated-probe forgetting reports.
+- Limited learning gain to post-first-step probes, averaged repetitions within independent sequences, and added per-candidate intervals.
+- Added recorded arm order for counterbalanced runs and exact sequence references for safe crash recovery.
+- Added abort and resume support, content-addressed comparison and probe evidence, and exact cell artifact hashes.
+
+### Changed
+
+- Memory experiment artifacts and cache identities now record `memoryMode` and a full `comparisonRef`; non-equivalent arms fail comparison.
+
 ## 7.0.1
 
 ### Changed
@@ -55,7 +69,7 @@
 
 - Load `proper-lockfile` with a dynamic import inside the functions that take a lock, instead of at module scope.
   It pulls in `graceful-fs`, which patches Node's `fs` at import time (`fs.close = ...`).
-  workerd exposes those as getter-only accessors, so the assignment threw while Cloudflare validated an uploaded Worker (`Cannot set property close of #<Object> which has only a getter [code: 10021]`), rejecting the whole Worker — including consumers that never take a lock.
+  workerd exposes those as getter-only accessors, so the assignment threw while Cloudflare validated an uploaded Worker (`Cannot set property close of #<Object> which has only a getter [code: 10021]`), rejecting the whole Worker, including consumers that never take a lock.
   `verify:package` now fails on any static import of a module that patches a Node builtin, because `wrangler deploy --dry-run` bundles without executing and cannot see this class of failure.
 
 ## 6.1.8

README.md

@@ -239,6 +239,36 @@ Positive external work without a receipt is recorded as incomplete cost accounti
 Use `0` only for a free local path.
 Paid memory improvement defaults to a zero-dollar total limit; set `maxTotalCostUsd` and `maximumEvaluationCostUsd` before enabling paid work.
 
+Use `runAgentMemoryLearningExperiment` to measure whether retained memory helps across ordered steps:
+
+```ts
+import { runAgentMemoryLearningExperiment } from '@tangle-network/agent-knowledge/memory'
+
+const result = await runAgentMemoryLearningExperiment({
+  experimentId: 'support-memory',
+  runDir: 'support-memory',
+  candidates: [memoryCandidate],
+  sequences,
+  seed: 42,
+  reps: 5,
+  armOrder: 'stateful-first',
+  costCeiling: 10,
+})
+
+console.log(result.comparison.gain)
+```
+
+The function runs matched stateful and stateless arms with the same immutable candidate, tasks, executor, policy, seed, and repetitions.
+The stateless arm clears declared scopes between steps; adapters must support scoped `clear`.
+Gain excludes first-step probes and averages candidates and repetitions within each independent sequence.
+Use `transferKey` on later probes for transfer and repeat one `retentionKey` across steps for forgetting.
+Unmarked probes are not assigned those meanings.
+
+Both arms share one cost limit and must have identical comparison references.
+Run independent experiments with opposite `armOrder` values when provider behavior may drift.
+Each saved probe includes the exact scoring input and content hash, so protect the run directory like the memory data itself.
+Pass `signal` to cancel; rerun the same options and directory to resume completed work and cost records.
+
 ## Run benchmarks
 
 `@tangle-network/agent-knowledge/benchmarks` provides common case and report types for:

docs/architecture.md

@@ -38,17 +38,18 @@ When that string is the canonical `<root>/.agent-knowledge` directory, both form
 | --- | --- |
 | `.agent-knowledge/index.json` | the built knowledge index (`writeKnowledgeIndex` writes it through this store) |
 | `.agent-knowledge/events.json` | the knowledge event log, including one `research.iteration` per research-loop round |
-| `.agent-knowledge/claim-ledgers/<id>.json` | one research run's claim ledger — corroboration counts, contradiction edges, open deep questions |
+| `.agent-knowledge/claim-ledgers/<id>.json` | one research run's claim ledger: corroboration counts, contradiction edges, open deep questions |
 | `.agent-knowledge/sources.json` | the immutable source registry |
 | `.agent-knowledge/mutation.lock.durable`, `mutation-epoch.json`, `file-transactions/` | the cross-process mutation lock and its crash-recovery state |
 
 The root is also the directory `withKnowledgeMutation` locks, so every record above is written under one lock and one epoch.
 There is exactly one writer per file: a second index writer alongside this one is a defect, not a variation.
 
-A claim ledger is the one record several writers legitimately share — a resumed run beside a live one, or several workers researching one goal in parallel.
+A claim ledger is the one record several writers legitimately share, such as a resumed run beside a live one or several workers researching one goal in parallel.
 They reach it through `mergeClaimLedger(id, merge)`, which holds the mutation lock across the read, the merge, and the write, so no writer can build its record from a value another writer has already replaced.
 `putClaimLedger` writes the whole record and is correct only for a single writer.
-The combining rule is `mergeClaimLedgers`: support and contradiction edges union, `contested` and `addressed` latch on, `firstSeenRound` moves earlier, and every collection is sorted — so the merge is commutative, associative, and idempotent, and the bytes on disk depend on the evidence rather than on scheduling.
+The combining rule is `mergeClaimLedgers`: support and contradiction edges union, `contested` and `addressed` latch on, `firstSeenRound` moves earlier, and every collection is sorted.
+The merge is commutative, associative, and idempotent, so the bytes on disk depend on the evidence rather than on scheduling.
 Ledgers for two different goals refuse to merge (`ClaimLedgerGoalConflictError`) rather than pooling unrelated evidence into one corroboration count.
 The live driver exposes the published Set-based `TrackedClaim`; the ledger stores a separate `ResearchClaimRecord` with sorted arrays so JSON serialization cannot erase those sets.
 Source verification snapshots the proposal and persists a `ResearchClaimEvidence` observation containing the expected registry id, original URI, and full content hash; that observation cannot affect claim support or completion by itself.
@@ -57,7 +58,8 @@ The ledger materializes only observations whose complete source identity matches
 Unversioned URI-only ledgers cannot prove which bytes produced their observations; reads and writes fail with `ClaimLedgerMigrationRequiredError` and preserve the original file for an explicit archive-and-reverify migration.
 Before synchronous question generation, the persistent driver records `preparedRounds`; a resume reconstructs and checkpoints any prepared round whose questions were interrupted, and the loop publishes its `research.iteration` event only after that checkpoint succeeds.
 
-Every write in this layer goes through `durable-fs` (`writeFileDurable`, `writeJsonDurableWithinRoot`) — temp file, fsync, atomic rename, fsync parent, through `O_NOFOLLOW` descriptors anchored via `/proc/self/fd` so a directory swapped for a symlink mid-write cannot redirect it outside the root.
+Every write in this layer goes through `durable-fs` (`writeFileDurable`, `writeJsonDurableWithinRoot`): temp file, fsync, atomic rename, and parent fsync.
+`O_NOFOLLOW` descriptors anchored through `/proc/self/fd` prevent a directory swapped for a symlink during a write from redirecting it outside the root.
 These are exported from the package entrypoint; consumers that keep their own journals should use them rather than reimplement them.
 
 ## Runtime Loop

src/memory/experiment.ts

@@ -3,25 +3,43 @@ export {
   buildAgentMemorySequenceScenarios,
   buildAgentMemorySequencesFromBenchmarkCases,
 } from './experiment/cases'
+export {
+  compareAgentMemoryLearning,
+  runAgentMemoryLearningExperiment,
+} from './experiment/learning'
 export { runAgentMemoryExperiment } from './experiment/run'
 export type {
   AgentMemoryAttemptEvent,
+  AgentMemoryEvidenceRef,
   AgentMemoryExecutionContext,
   AgentMemoryExecutionCostMeter,
   AgentMemoryExecutionCostReceipt,
   AgentMemoryExecutionPaidCallInput,
   AgentMemoryExecutionPaidCallResult,
   AgentMemoryExecutionStep,
   AgentMemoryExperimentCandidate,
+  AgentMemoryExperimentCandidateRef,
+  AgentMemoryExperimentComparisonRef,
   AgentMemoryExperimentRankingRow,
   AgentMemoryExperimentRunLease,
+  AgentMemoryForgettingComparison,
+  AgentMemoryLearningArmOrder,
+  AgentMemoryLearningCandidateSummary,
+  AgentMemoryLearningCellComparison,
+  AgentMemoryLearningComparison,
+  AgentMemoryMode,
   AgentMemorySequence,
   AgentMemorySequenceArtifact,
   AgentMemorySequenceProbe,
   AgentMemorySequenceProbeResult,
   AgentMemorySequenceScenario,
   AgentMemorySequenceStep,
+  AgentMemoryTransferCellComparison,
+  AgentMemoryTransferStepSummary,
   BuildAgentMemorySequencesFromBenchmarkCasesOptions,
+  CompareAgentMemoryLearningOptions,
   RunAgentMemoryExperimentOptions,
   RunAgentMemoryExperimentResult,
+  RunAgentMemoryLearningExperimentOptions,
+  RunAgentMemoryLearningExperimentResult,
 } from './experiment/types'

src/memory/experiment/cases.ts

@@ -32,7 +32,7 @@ export function buildAgentMemorySequencesFromBenchmarkCases(
         scope: compactScope(
           options.eventScope?.({ event, case: testCase, eventIndex }) ?? {
             agentId: memoryAgentId,
-            sessionId: testCase.id,
+            sessionId: 'benchmark-session',
           },
         ),
         writes: [
@@ -56,7 +56,7 @@ export function buildAgentMemorySequencesFromBenchmarkCases(
         scope: compactScope(
           options.probeScope?.(testCase) ?? {
             agentId: memoryAgentId,
-            sessionId: testCase.id,
+            sessionId: 'benchmark-session',
           },
         ),
         probes: [