feat(agent): durable working-notes block on the session envelope (plan #938) - #940
Conversation
…#938, source #550) - NEW cap WORKING_NOTES_MAX_BYTES = 32 KiB + sanitizeWorkingNotes (length-only freeform text, poison drop-to-unset, never truncates) - reserved meta.workingNotes on the store + worker overlays (drop-to-unset) - always-on working_notes_get/_update/_clear tools: envelope-seam reads, over-cap reject, best-effort persist at tool-execute via the worker copy-forward overlay (clock max(stored, wall)+1 + one bounded LWW retry), honest store-down, no auto-extraction - resolveSystem notesPreamble + workingNotesBlock() frame: persona -> notes -> skills, unverified agent-authored working memory, never standing orders - assembled in buildToolWorld after meta_*; durable in-step fold (resolveInStepPreambles widened) + legacy /api/agent parity fold - host mirror: SessionSnapshot.workingNotes, localStorage re-sanitize, cloudMetaFor emit, parse/overlay restore - route soft-path guard excludes working_notes_* from the 403 deferral surface (like meta_*) - docs: session-model, harness-limits, feature-divide, agent-stream, AGENTS.md - tests: caps sanitize, reserved keys, worker overlay, 14-row tools matrix, system fold, buildToolWorld, host round-trip (full suite 3309 green) Refs #938 Refs #550
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
btipling
left a comment
There was a problem hiding this comment.
Adversarial review — PR #940
Verdict: CONCERNS
Repo: btipling/invincible
Scope: main ← plan/working-memory-notes · 23 files · durable meta.workingNotes + always-on working_notes_* tools
Lenses run: L1, L2, L3, L5, L6, L7, L8 (skip: L4 — no workflow/artifact/harness-SHA change; L9 — no UI/palette)
AGENTS.md read: yes · docs/feature-divide.md read: yes
Findings
| Sev | Lens | Finding | Break scenario | Refutation attempt | Confidence |
|---|---|---|---|---|---|
| Major | L1+L6 | cloudMetaFor emits workingNotes from the host snapshot (lib/sessionRepository.ts · cloudMetaFor). Envelope PUT is full-replace (absent = clear). Host snapshot is not updated when working_notes_update PATCHes the envelope (no SSE carrier; HarnessHost.tsx never mentions workingNotes). persist() then putEnvelopeOnce sends meta without the key (or with a stale GET-overlaid value) at updatedAt: Date.now() — strictly newer than the tool's max(stored, wall)+1 clock — and LWW-stomps the worker write. Same class as adversarial-review #937 (modelMessagesPointer). The PR's own test locks the wrong contract: "Unset mirror → meta omitted entirely (absent = clear intact)." turnPersistSeam correctly omits the key (copy-forward keeps it); the host flatten does not. |
1. Session has no notes (or notes from a prior GET). 2. Agent working_notes_update persists a finding (worker overlay, clock ≈ now+1). 3. Host paints the tool_result and persist()s (HarnessHost.tsx:871 → repo.put → putEnvelopeOnce with meta: body.meta). 4. Snapshot has no / stale workingNotes; host clock is Date.now() > worker clock. 5. Envelope PUT replaces meta; notes wiped (first write) or rolled back (update). Next turn's notesPreamble is empty. The feature's only payload does not survive the rest of the same turn's host persist. |
Defender: "unlike modelMessagesPointer this is a host mirror that round-trips GET honestly, so emit is safe beyond ordinary LWW." Fail: GET overlay is not in the persist path; cwd/usage survive host PUT because SSE updates the snapshot before persist — notes have no such event. keepObservedModelMessagesPointer exists exactly because persist snapshots omit worker-authored fields; nothing equivalent exists for notes. Copy-forward-on-omit inside upsertEnvelope cannot be copied blindly: working_notes_clear currently DELETEs the key (patchWorkerMeta poison → delete out[key]), which is indistinguishable from host omit and would make clear restore the block. Fix: (1) cloudMetaFor never emits workingNotes (host snapshot is always possibly stale). (2) upsertEnvelope copy-forwards stored notes on omit, same site as copyForwardModelMessagesPointer. (3) Worker clear must be a present marker (workingNotes: '') so copy-forward does not restore; sanitizeWorkingNotes('') then drops it. (4) Lock it with a test the current suite does not have. |
high |
| Minor | L1 | working_notes_update / _clear claim "updated/cleared in-turn" when persist fails (workingNotesTools.ts · persistNotesPatch fail paths). There is no in-turn notes cache — the fold is envelope-at-modelGenerateStep / once-at-/api/agent. The code comment says "the in-turn value exists but nothing persisted." It does not. |
Store down or LWW retry exhausted. Tool returns working notes updated in-turn; persistence unavailable. Model treats the block as written (or cleared). Next round/turn the envelope is unchanged. False-clear is the worse face: user asked to forget, agent reported cleared, notes reappear in the next system fold. |
Defender: "the tool-call args still hold the text this turn, and tests assert no working notes updated (." Fail: that assertion is a prefix trick around the lie. The module header promises "the tool never claims a persistence that did not happen." "Updated in-turn" is a persistence claim. |
high |
| Minor | L6 | Claimed coverage that is not in the file: workingNotesTools.test.ts header lists "LWW retry"; no test drives first-attempt lww_conflict then a successful retry. Soft-path 403 guard (app/api/agent/route.ts · isWorkingNotesToolName) has no route.test.ts row — the PR body says this defect was found during impl, then left unpinned. |
A future edit drops the !isWorkingNotesToolName filter. working_notes_* become nonSkillToolCount > 0. A notes-only turn on a workspace-required 403 proceeds and hides the unavailable sandbox. |
Defender: buildToolWorld.test.ts asserts the three tools exist; the 14-row tools matrix is real. Fail: assembly ≠ the 403-deferral surface; the retry helper is untested against a one-conflict overlay. |
high |
Residual risk
The notes block is folded verbatim into the system prompt with a prose disclaimer, not a closed XML tag. A prior-turn working_notes_update can smuggle standing-order-shaped text (<persona_standing_orders>…). The plan's honesty bar accepts this (framed unverified; agency belongs to the agent). 32 KiB is a standing per-round inference cost by design. /api/agent resolves notesPreamble once per request (fold not hot on the legacy path); durable modelGenerateStep re-reads the envelope each round — so a successful persist is hot on /api/turns. Live Redis/host persist was not executed here.
Merge guidance
- CONCERNS: do not merge until the Major is fixed. The #937 copy-forward exception exists because worker-authored envelope keys cannot ride a stale host snapshot through full-replace PUT.
workingNotesis that class. - Minors should land in the same pass (honest fail-soft copy; LWW-retry + soft-path tests).
What was not attacked
Live durable turn + host repo.put against Redis; production Gateway; Wasm paint (none claimed); DO runner; Vercel Production deploy of this branch.
Host cloudMetaFor never emits the worker-authored notes block. Envelope PUT copy-forwards the stored value when incoming omits the key, matching modelMessagesPointer (#937). Worker clear is a present empty string so copy-forward does not restore. Honest fail-soft copy on store-down (no false in-turn claim). Tests lock omit-keep, explicit-clear, LWW retry, and the route 403 soft-path filter. Refs #938 #550
btipling
left a comment
There was a problem hiding this comment.
Adversarial review — PR #940
Verdict: PASS WITH NOTES
Repo: btipling/invincible
Scope: main ← plan/working-memory-notes · 26 files · durable meta.workingNotes + always-on working_notes_* tools (re-review of 7da40f6 after the copy-forward fix)
Lenses run: L1, L2, L3, L5, L6, L7, L8 (skip: L4 — no workflow/artifact/harness-SHA change; L9 — no UI/palette)
AGENTS.md read: yes · docs/feature-divide.md read: yes
Prior pass (CONCERNS) is closed on this head:
- Host
cloudMetaFornever emitsworkingNotes;upsertEnvelopecopy-forwards on omit; worker clear is a present''(copyForwardWorkingNotes). Same class as #937, now locked insessionStore.test.ts+sessionRepository.test.ts. - Fail-soft copy is honest (
not persisted/not cleared) — no in-turn persistence claim. - LWW retry is driven (
workingNotesTools.test.ts); the/api/agent403 soft-path filter is pinned by the existingWORKSPACE_INSTANCE_REQUIREDrow (realbuildToolWorldassemblesworking_notes_*; dropping!isWorkingNotesToolNamewould 200).
Findings
| Sev | Lens | Finding | Break scenario | Refutation attempt | Confidence |
|---|---|---|---|---|---|
| Minor | L6 | Plan DoD rows 6–7 are not pinned. resolveInStepPreambles (lib/workflows/modelGenerateStep.ts) is the durable fold: notes must be read even when persona/skills stores are absent (the retired return {} guard). lib/workflows/ has zero notesPreamble / workingNotes assertions. /api/agent has a persona-preamble route test and no notes analogue — runAgent is never asserted to receive notesPreamble. |
1. Revert the widened guard to if (!userPersonas && !userSkills) return {}. 2. Durable turn, envelope has notes, persona/skills stores absent (fail-open). 3. In-step resolver returns {}. 4. Model system has no working-notes block. The feature's only payload is dropped on the exact path the plan called the highest-risk seam. Same silent miss if the /api/agent envelope read is deleted — resolveSystem tests still pass because they inject notesPreamble by hand. |
Defender: agentSystem.test.ts locks the frame, buildToolWorld.test.ts locks assembly, tools matrix locks persist. Fail: assembly ≠ fold. The fold is a separate read of meta.workingNotes in two callers (route.ts, resolveInStepPreambles). Neither caller is asserted. |
high |
Residual risk
Notes are folded verbatim after a prose disclaimer, not a closed tag (<persona_standing_orders> / <attached_skills> wrap their bodies). A working_notes_update can still plant a fake <attached_skills> block ahead of the real catalog. Plan-review #938 finding 2 accepted this (framed unverified; agency belongs to the agent) — not re-raised. 32 KiB is a standing per-round inference cost by design. Wrap-up disableTools rounds use STEP_BUDGET_WRAPUP_SYSTEM / TURN_WALL_CLOCK_WRAPUP_SYSTEM and skip the notes fold (history still carries the tool result). Live Redis/host persist was not executed here. Default host carrier is still roll-forward (NEXT_PUBLIC_HARNESS_CARRIER_ENVELOPE); worker overlay writes the envelope key, so the durable fold still sees notes when that key exists.
Merge guidance
- PASS WITH NOTES: safe to merge from this attack. The Minor should land as a pin on
resolveInStepPreambles(stores-absent notes read) and an/api/agentnotesPreambleroute assertion mirroring the persona row — not a merge blocker.
What was not attacked
Live durable turn + host repo.put against Redis; production Gateway; Wasm paint (none claimed); DO runner; Vercel Production deploy of this branch; origin NEXT_PUBLIC_HARNESS_CARRIER_ENVELOPE value.
|
Follow-up to the PASS WITH NOTES re-review: the Minor (unpinned DoD rows 6–7) is landed in 6547560.
Both files green locally (51 tests in those two files). |
btipling
left a comment
There was a problem hiding this comment.
Adversarial review — PR #940
Verdict: PASS WITH NOTES
Repo: btipling/invincible
Scope: main ← plan/working-memory-notes · 27 files · durable meta.workingNotes + always-on working_notes_* tools (re-review of 6547560 after the DoD 6–7 pin)
Lenses run: L1, L2, L3, L5, L6, L7, L8 (skip: L4 — no workflow/artifact/harness-SHA change; L9 — no UI/palette)
AGENTS.md read: yes · docs/feature-divide.md read: yes · SECURITY.md read: yes (API/session surface)
Prior passes are closed on this head:
f3ebb82CONCERNS (Major L1+L6):cloudMetaForemit + host flatten LWW-stomp. Fixed in7da40f6— host never emits;upsertEnvelopecopy-forwards on omit; worker clear is a present''. Locked insessionStore.test.ts+sessionRepository.test.ts.7da40f6PASS WITH NOTES (Minor L6): DoD rows 6–7 unpinned. Fixed in6547560—resolveInStepPreamblesexported and pinned (modelGenerateStep.preambles.test.ts: stores-absent read, empty/over-cap/store-down fail-open);/api/agentassertsrunAgentreceivesnotesPreamblefrom the envelope without persona/skills.
Re-attacked on 6547560 (not hunk-only): tools persist + LWW retry + honest fail-soft; overlay poison-as-'' vs copy-forward clear order (copyForward then assertValidSessionEnvelope); Redis/Memory upsertEnvelope both apply the helper against LWW existing; cloudMetaFor omit; GET overlayEnvelopeMeta restore; New/Clear = DELETE + mint (fresh id, not same-id PUT-omit); runAgent/runAgentStream both pass params into resolveSystem; durable modelGenerateStep folds after assemble (wrap-up disableTools uses the cap system, not notes); /api/agent 403 filter (isWorkingNotesToolName); buildToolWorld always-on assembly; DI/cost (MemorySessionStore / mocks, no extra PGlite); no Wasm/DOM notes UI.
Findings
No finding survived self-refutation on this head.
Residual risk
Notes fold verbatim after a prose disclaimer, not a closed tag (<persona_standing_orders> / <attached_skills> wrap their bodies). A working_notes_update can still plant a fake <attached_skills> block ahead of the real catalog. Plan-review #938 finding 2 accepted this (framed unverified; agency belongs to the agent) — not re-raised. 32 KiB is a standing per-round inference cost by design. Wrap-up disableTools rounds use STEP_BUDGET_WRAPUP_SYSTEM / TURN_WALL_CLOCK_WRAPUP_SYSTEM and skip the notes fold (history still carries the tool result). /api/agent resolves notesPreamble once per request (not hot inside generateText steps); durable modelGenerateStep re-reads the envelope each round, so a successful persist is hot on /api/turns next round — both match the tool copy (“next model round/turn”). Live Redis/host persist was not executed here. Default host carrier is still roll-forward (NEXT_PUBLIC_HARNESS_CARRIER_ENVELOPE); worker overlay writes the envelope key, so the durable fold still sees notes when that key exists.
Merge guidance
- PASS WITH NOTES: safe to merge from this attack. No remaining code change from this pass.
What was not attacked
Live durable turn + host repo.put against Redis; production Gateway; Wasm paint (none claimed); DO runner; Vercel Production deploy of this branch; origin NEXT_PUBLIC_HARNESS_CARRIER_ENVELOPE value; npm run test:int plan row 9 (unit pins cover write and fold separately on the same key/sanitizer).
Plan #938 — durable agent working notes (session memory across turns)
Implements the HANDOFF-READY plan issue #938 (source issue #550, parent #548).
Branch cut from post-A1
main(a6adb08— PR #937 merged, the plan's Blocker precondition).What lands
WORKING_NOTES_MAX_BYTES= 32 KiB (lib/sessionCloudCaps.ts) +sanitizeWorkingNotes— length-only freeform text, poison drop-to-unset, never truncates. Only a NEW cap; no existing cap value changed → no human gate.meta.workingNotesonRESERVED_META_KEYS(lib/sessions/sessionStore.ts) andWORKER_META_KEYS(lib/agent/workerMetaOverlay.ts), drop-to-unset sanitize on both.working_notes_get/working_notes_update/working_notes_cleartools (lib/agent/workingNotesTools.ts): envelope-seam reads, over-cap writes rejected with an explicit error, best-effort persist at tool-execute via the worker copy-forward overlay (overlayWorkerMeta, clockmax(stored, wall) + 1+ one bounded LWW retry), honest store-down text, no auto-extraction — the agency to persist belongs to the agent.lib/agent/agentSystem.ts):notesPreamble+workingNotesBlock()— persona → notes → skills, framed as unverified agent-authored working memory, never standing orders / established fact.buildToolWorld(aftermeta_*, before the FS merge — both routes inherit) · durable in-step fold (resolveInStepPreambleswidened guard; reads the notes even when persona/skills stores are absent) · legacy/api/agentparity fold (runAgent.notesPreamble).SessionSnapshot.workingNotes, localStorage re-sanitize on load,cloudMetaForemit (absent = clear),parseCloudSessionSnapshot+overlayEnvelopeMetarestore on the reserved-meta replace contract.working_notes_*excluded from the 403-deferral legitimate surface (likemeta_*) — a notes-only turn must not hide an unavailable workspace.Caps
WORKING_NOTES_MAX_BYTES= 32 KiB is a NEW generous cap — no existing cap value changed (no human gate). It bounds the standing per-round inference cost far under the 1 MiB whole-meta budget and the 4.5 MB Function wire.Living docs
docs/session-model.md(reserved key table + fold contract) ·docs/harness-limits.md(caps row) ·docs/feature-divide.md·docs/agent-stream.md·AGENTS.md.Tests
Full suite green: 3309 passed (168 files),
tsc --noEmitclean. New/updated: caps sanitizer, reserved-key locks, worker overlay rows, 14-row tools matrix (lib/agent/workingNotesTools.test.ts),resolveSystemfold order + no-standing-orders framing,buildToolWorldassembly, host fold→parse→overlay round-trip,/api/agentroute parity (46/46).Two defects found and fixed during implementation: the persist clock now strictly advances (an equal-timestamp PATCH would always LWW-conflict), and the notes family is excluded from the route's soft-surface guard so a notes-only turn can never mask a 403 workspace-required deferral.
Refs #938
Refs #550