feat(agent): durable working-notes block on the session envelope (plan #938)

AGENTS.md

@@ -344,6 +344,7 @@ invincible/
 | Durable-turn wall clock (plan #923 — hard 1-hour cap) | `lib/sessionCloudCaps.ts` (`TURN_WALL_CLOCK_MAX_MS` = 3 600 000 — **code**, human-authorized; `TURN_WALL_CLOCK_WRAPUP_MAX_MS` = 300 000 — **wall** wrap-up substitute bound only (`wrapUp === 'wall'`), 1h-exempt but not unbounded; the 512-step wrap-up does not inherit it (it still carries the 1h `deadlineAt` signal); `TURN_WALL_CLOCK_DEADLINE_TTL_MS` = 60 000 and `TURN_WALL_CLOCK_PROBE_EVERY_MS` = 2 000 are **env-comment-only cache/probe seams, never enforcement** — an env override can never shorten the cap), `lib/workflows/turnDeadline.ts` (directive-free `deadlineSignal` / `isDeadlineElapsed` / `wrapUpDeadlineAt` / `combineAbortSignals`), `lib/workflows/turnWorkflow.ts` (derives `deadlineAt` from `getWorkflowMetadata().workflowStartedAt` + the cap — the only runtime-pinned, replay-deterministic clock; never a live `Date.now()` in the workflow body; never import `getWorkflowMetadata` into a step file), `lib/workflows/turnLoop.ts` (boundary checks + `'wall_clock'` routing + wall wrap-up; **when both caps fire, wall wins** — the 512-step wrap-up is still subject to the 1h `deadlineAt` signal (not the 5-min substitute); wrap-up persist is the first `completed` overlay — a pre-wrap terminal persist would release C15's live-only 409), `lib/workflows/modelGenerateStep.ts` + `lib/agent/generateOneRound.ts` (per-attempt `AbortSignal.timeout(remaining)` from the serialized `deadlineAt`; tools-on assemble also gets that signal; wrap-up uses `deadlineAt + TURN_WALL_CLOCK_WRAPUP_MAX_MS` + `reasoning: 'none'` only when `wrapUp === 'wall'` (`wrapUp === 'steps'` still carries the 1h `deadlineAt` signal); a deadline abort returns the dedicated `'wall_clock'` step code — a genuine user Stop stays `'cancelled'`; no signal/closure ever crosses the step boundary), `lib/workflows/toolExecuteStep.ts` (whole-batch deadline gate + per-attempt deadline `AbortSignal` into `executeTool` / assembled world + between-wave skip + `'wall_clock'` batch code; in-wave aborts stay best-effort) |
 | Durable agent system prompt (same resolver as `/api/agent`) | `lib/agent/agentSystem.ts` (`resolveSystem` + `DEFAULT_AGENT_SYSTEM` / HTTP / skill / meta-only strings), `lib/workflows/modelGenerateStep.ts` (in-step after assemble, from the assembled registry; persona snapshot + sticky/always-on skills fail-open independently — persona via the envelope seam, never legacy `get`/`put`; slash-command attach is **not** handled in the step; cap wrap-up `disableTools` skips assemble and uses `STEP_BUDGET_WRAPUP_SYSTEM`, never `DEFAULT_AGENT_SYSTEM`), `lib/agent/runAgent.ts` (same helper). Do **not** add `maxOutputTokens`. |
 | Structured model-messages projection (durable-turn cross-turn LLM context — plan #936, source #549) | `lib/agent/modelMessages.ts` (pure projection builder: persisted deltas → typed `user` / `assistant`(+`tool-call`) / `tool-result` rows; `persist`/`error` skipped, reasoning dropped, **orphan tool-results dropped with a marker** so strict providers accept the seeded array; `tool-result` bodies truncated to `MODEL_MSG_TOOL_RESULT_MAX_CHARS` = 2000; whole object row/byte-capped at `MODEL_MSG_CHECKPOINT_MAX_ROWS` = 4096 / `MODEL_MSG_CHECKPOINT_MAX_BYTES` = 8 MiB — 3 NEW caps in `lib/sessionCloudCaps.ts`, no existing cap touched), `lib/agent/turnPersistSeam.ts` (writes the projection as its **own Blob object** + sets `meta.modelMessagesPointer`, fail-closed `model_messages_write_failed`), `lib/sessions/sessionStore.ts` + `lib/agent/workerMetaOverlay.ts` (reserved `modelMessagesPointer` key + drop-to-unset sanitize), `lib/workflows/persistStep.ts` + `lib/workflows/turnLoop.ts` (`derivePersistFold` derives the `modelMessages` sibling; `TurnLoopInput.priorMessages`; loop seeds `messages = [...priorMessages, {role:'user'}]`), `app/api/turns/route.ts` (pre-start seed: reads the bound `modelMessagesPointer` Blob with confused-deputy `isObjectIdBoundTo`, fail-closed to the legacy fold; `userMessage` = raw prompt when seeded, else `promptHistory ?? prompt`), `lib/agent/agentBody.ts` (`promptHistory` bounded by `PROMPT_BODY_MAX_CHARS` — no cap change), `lib/agentApi.ts` + `lib/turnApi.ts` + `lib/harnessChat.ts` (host sends raw `prompt` + `promptHistory` only when no local pointer; legacy injected path unchanged), `lib/sessionStore.ts` + `lib/sessionRepository.ts` (`modelMessagesPointer` carrier via `meta`; host `cloudMetaFor` never emits it — GET overlay is local sidecar-stop; envelope PUT copy-forwards the stored worker id). The LLM payload is **not** the display checkpoint and **not** the transcript paint rows; `formatPromptWithHistory` is demoted to a one-shot legacy roll-forward. docs: [docs/session-model.md](docs/session-model.md), [docs/feature-divide.md](docs/feature-divide.md), [docs/harness-limits.md](docs/harness-limits.md), [docs/agent-stream.md](docs/agent-stream.md) |
+| Durable agent working notes (session memory across turns — plan #938, source #550) | `lib/sessionCloudCaps.ts` (`WORKING_NOTES_MAX_BYTES` = 32 KiB NEW cap + `sanitizeWorkingNotes` — length-only freeform text, poison drop-to-unset, never truncates; no charset restriction), `lib/sessions/sessionStore.ts` + `lib/agent/workerMetaOverlay.ts` (reserved `workingNotes` key on both overlays; drop-to-unset), `lib/agent/workingNotesTools.ts` (always-on `working_notes_get` / `working_notes_update` / `working_notes_clear` — envelope-seam reads, over-cap writes rejected with an explicit error, best-effort persist at tool-execute via the worker copy-forward overlay with clock `max(stored, wall) + 1` + one bounded LWW retry, honest store-down text, **no auto-extraction** — the agency to persist belongs to the agent), `lib/agent/agentSystem.ts` (`notesPreamble` param + `workingNotesBlock()` frame — persona → notes → skills, framed as **unverified agent-authored working memory, never standing orders**), `lib/agent/buildToolWorld.ts` (assembles the family after `meta_*`, before the FS merge — both routes inherit), `lib/workflows/modelGenerateStep.ts` (`resolveInStepPreambles` widened guard + envelope notes fold; runs even when persona/skills stores are absent), `lib/agent/runAgent.ts` + `app/api/agent/route.ts` (legacy-path parity fold), `lib/sessionStore.ts` + `lib/sessionRepository.ts` (host GET overlay restores `SessionSnapshot.workingNotes`; `cloudMetaFor` **never emits** the key — a stale/absent host snapshot at `Date.now()` would LWW-stomp the tool write, adversarial-review #940; envelope PUT copy-forwards stored notes on omit; worker clear is a present empty string). The block folds into every later model round between persona and skills; a note survives a cancelled/wall-clocked/errored turn **and** the same-turn host flatten; the fold is not hot. docs: [docs/session-model.md](docs/session-model.md), [docs/feature-divide.md](docs/feature-divide.md), [docs/harness-limits.md](docs/harness-limits.md), [docs/agent-stream.md](docs/agent-stream.md) |
 | Agent read-before-edit / file freshness | `lib/agent/fileFreshness.ts`, `lib/agent/pathLock.ts` (per-path apply serialization), `lib/agent/tools.ts`, `lib/agent/runAgent.ts`, [docs/sandbox.md](docs/sandbox.md) |
 | Logical agent cwd + workspace-root↔abs canonicalization (`change_dir` / session / default env; **`sandbox_info`** is the bind/cwd/caps/env introspector — do not `exec env`; `canonicalizePath(R, p)` / `workspaceAbsToRel(R, abs)` / `resolvePathForTool(R, cwd, p)` / `rewriteExecRootToRel(R, text)` in `lib/agent/workPath.ts`) + **`search`** (read-grant-only code-grep via `rg`; `lib/agent/tools.ts`) | `lib/agent/workPath.ts`, `lib/agent/tools.ts`, `lib/agent/runAgent.ts`, `lib/agent/agentBody.ts`, `lib/sandbox/config.ts`, `lib/sessionStore.ts`, `lib/harnessChat.ts`, `lib/agentApi.ts`, `lib/sessionCloudCaps.ts` (shared client-safe `sanitizeSessionCwd` + Redis-safe opaque id predicate), [docs/sandbox.md](docs/sandbox.md), [docs/session-model.md](docs/session-model.md), [docs/agent-stream.md](docs/agent-stream.md). Tool paths accept **in-jail absolute paths** on all FS tools + `change_dir` + `exec` cwd: an absolute under the per-binding jail root R (`resolved.value.workspaceRoot` → `RunAgentParams.workspaceRoot` → `createAgentTools`) is canonicalized to the same workspace-relative freshness key as its relative form (BYO + Vercel parity); out-of-jail absolutes and `..`/symlink escapes fail closed. Absolute paths under `R` that **appear in `exec` stdout/stderr** are likewise rewritten to workspace-relative (`rewriteExecRootToRel` in `lib/agent/workPath.ts`, applied to `result.stdout`/`result.stderr` separately) so `exec pwd` ≡ `pwd`/annotations; when `R` is unresolvable the exec output passes through byte-for-byte (fail-open), and rewrites are capped and never throw. When R is unresolvable (BYO daemon down/pre-v2 — `workspaceRoot === null`) absolute is rejected (“root unavailable — use workspace-relative”) while relative + cwd still work. Initial request/session `cwd` stays relative-only; `.` is the workspace-root default session start (there is no `SANDBOX_DEFAULT_CWD` env knob), `..` walks up toward the workspace root and errors only past it, and an **exact ancestor** of cwd re-roots cleanly (`change_dir invincible` from `cwd=invincible/docs` → `invincible`, not the phantom `invincible/docs/invincible`) while a name-prefix sibling is never re-rooted. P1/GAP-1 (#452/#330): `cwd` + `activeSandboxId` are **session-owned** and ride the Redis record (`meta.{logicalCwd,activeSandboxId}`). `activeSandboxId` is now **server-resolved** (routing override via `lib/tenancy/resolveSandbox.ts` `requestedSandboxId`), not carry-only. A **confirmed successful `change_dir`** is persisted as the session cwd even when the turn later cancels / times out / hard-errors (`lib/harnessChat.ts` host-side `liveCwd`); the success path still prefers the authoritative `agentResult.cwd`, and only a confirmed `change_dir` (never an errored one) is stored on a failed/aborted turn. The **`exec` tool** returns a **compact summary**, not a raw dump: first `EXEC_LOG_HEAD_LINES` (10) + last `EXEC_LOG_TAIL_LINES` (10) lines per stdout/stderr with line/byte counts and `... (N lines truncated)`, each shown line byte-clipped (`EXEC_SUMMARY_LINE_MAX_BYTES`=4096) so a single fat stdio line can't inline the stream or truncate the `log:` pointer off; and when either stream is non-empty writes the full redacted output to `<workspace>/.invincible/logs/exec-<ts>-<seq>.log` via `client.write_file(..., mkdir: true)` (a brand-new hidden workspace dir; backends never auto-create parents; the `-<seq>` monotonic counter keeps same-ms parallel execs from overwriting), reporting two `read_file` pointers — `log: <rel path>` (cwd-relative, from nested cwd `../.invincible/logs/…`) and `log (root): <root path>` (workspace-root-relative, read from the workspace root `cwd .`, so a depth-changing `change_dir` can never strand the full output); the write stays workspace-root, and `.invincible/` is gitignored; both pointers ride immediately after `exit=`/`TIMED_OUT` — empty output (`exec true`) writes no file, and a log-write failure fails soft with a `⚠ log write failed` note whose reason is **sanitized** (a backend/jail path never surfaces) (caps `EXEC_LOG_HEAD_LINES`/`EXEC_LOG_TAIL_LINES`=10 and `EXEC_LOG_MAX_BYTES`=8 MiB in `lib/sandbox/config.ts`) |
 | Cloud multi-device harness session (Redis multi-session, `/api/sessions*`, hybrid local+cloud; **phase 0 #515 envelope + Blob transcript carrier**) | `app/api/sessions/*` (+ `app/api/sessions/[id]/envelope/*`, `[id]/transcript/*`), `lib/sessionRepository.ts`, `lib/sessionCloudCaps.ts`, `lib/sessions/*` (+ `lib/sessions/blobStore.ts`, `blobStores.ts`), `lib/tenancy/harnessSessionsRedis.ts`, `lib/tenancy/harnessSessions.ts` (archive read + shared validator), `lib/di/index.ts` (root), `app/harness/HarnessHost.tsx`, `middleware.ts`, [docs/session-model.md](docs/session-model.md), [docs/bring-your-own.md](docs/bring-your-own.md), [SECURITY.md](SECURITY.md) — one-shot Postgres→Redis backfill: GHA **`sessions-redis-backfill`** (idempotent per-user marker); Postgres `harness_sessions` is a read-only archive. P1/GAP-1 (#452): session-carrier `meta.{logicalCwd,activeSandboxId}` folds into the PUT body and restores on pull/adopt; **plan #616 (source #610)** adds the reserved `meta.selectedModel` session carrier for the selected model pick (restore by id after the model catalog push; server **drops a poisoned value to unset**, never a 400). **Phase 0 (#515):** the transcript lives in **Vercel Blob** (`BLOB_READ_WRITE_TOKEN` / BYO S3-R2 seam) pointed to by `meta.transcriptPointer` on the small Redis envelope (`harness:envelope:*`); server mints short-lived scoped upload URLs for **client→Blob** uploads; legacy full-record GET stays for roll-forward while old blobs stay small. Envelope upsert/read: `PUT`/`GET /api/sessions/:id/envelope`; mint/read: `POST`/`GET /api/sessions/:id/transcript` |

app/api/agent/route.test.ts

@@ -691,6 +691,9 @@ describe('POST /api/agent', () => {
     });
     expect(runAgent).not.toHaveBeenCalled();
     expect(mcp.buildUserMcpTools).toHaveBeenCalled();
+    // Plan #938 / adversarial #940: working_notes_* are always-on like meta_*
+    // and must not substitute for FS/MCP/http on this 403. If the filter
+    // dropped, this test would go 200 (notes-only turn hiding the workspace).
   });
 
   it('softContinue from resolve skips FS tools and still runs agent when MCP tools exist', async () => {
@@ -1655,6 +1658,63 @@ describe('POST /api/agent', () => {
     expect(userPersonas).not.toHaveBeenCalled();
   });
 
+  it('plan #938 / adversarial #940 — folds notesPreamble from the envelope (stores-absent; persona/skills not required)', async () => {
+    mockAuthedSession();
+    mockMcpEmpty();
+    mockByokOk();
+    mockGithubToken();
+    mockResolveSandboxOk();
+    process.env.AI_GATEWAY_API_KEY = 'gw-key';
+    const fakeSessionStore = {
+      get: vi.fn(),
+      put: vi.fn(),
+      list: vi.fn(),
+      remove: vi.fn(),
+      readEnvelope: vi.fn(async () => ({
+        id: 'sess_notes',
+        tenantId: 'tenant-1',
+        userId: 'user-1',
+        createdAt: 1,
+        updatedAt: 1,
+        meta: { workingNotes: 'finding: fold even without persona/skills stores' },
+      })),
+      upsertEnvelope: vi.fn(),
+    };
+    vi.doMock('../../../lib/tenancy/harnessSessionsRedis', () => ({
+      resolveSessionStore: async () => ({ ok: true as const, value: fakeSessionStore }),
+      sessionKeyFor: (t: string, u: string, s: string) => ({
+        tenantId: t,
+        userId: u,
+        sessionId: s,
+      }),
+    }));
+    servicesState.harnessSessionsRedis = {
+      resolveTenantIdForUser: vi.fn(async () => ({ ok: true as const, value: 'tenant-1' })),
+    };
+    type RunArg = { notesPreamble?: string; personaPreamble?: string; skillsPreamble?: string };
+    const runAgent = vi.fn(async (_arg: RunArg) => ({ text: 'ok', toolTrace: [] }));
+    vi.doMock('../../../lib/agent/runAgent', () => ({
+      runAgent,
+      runAgentStream: vi.fn(),
+    }));
+
+    const { POST } = await loadRoute();
+    const res = await POST(
+      new Request('http://localhost/api/agent', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ prompt: 'what did we conclude?', sessionId: 'sess_notes' }),
+      }),
+    );
+    expect(res.status).toBe(200);
+    expect(runAgent).toHaveBeenCalledTimes(1);
+    const arg = runAgent.mock.calls[0]?.[0] as RunArg;
+    expect(arg.notesPreamble).toBe('finding: fold even without persona/skills stores');
+    expect(arg.personaPreamble).toBeUndefined();
+    expect(arg.skillsPreamble).toBeUndefined();
+    expect(fakeSessionStore.readEnvelope).toHaveBeenCalled();
+  });
+
   it('strips /slug and folds the catalog skillsPreamble for an attach-with-prose prompt (phase 2 #517, plan #557/#931)', async () => {
     mockAuthedSession();
     mockMcpEmpty();

app/api/agent/route.ts

@@ -37,7 +37,8 @@ import {
 } from '../../../lib/tenancy/skillInject';
 import { isEnvelopeStore } from '../../../lib/sessions/sessionStore';
 import { isMetaToolName } from '../../../lib/agent/metaTools';
-import { parseAttachedSkills } from '../../../lib/sessionCloudCaps';
+import { isWorkingNotesToolName } from '../../../lib/agent/workingNotesTools';
+import { parseAttachedSkills, sanitizeWorkingNotes } from '../../../lib/sessionCloudCaps';
 
 export const runtime = 'nodejs';
 // Vercel Pro/Enterprise Fluid extended max is 1800s (30m). 3600s is not offered.
@@ -285,6 +286,37 @@ export async function POST(req: Request): Promise<Response> {
       runParams.prompt = `${modelPrompt}\n\n<reminder>Your persona standing orders (in the <persona_standing_orders> block above) are already in context. Follow them before any tool use.</reminder>`;
     }
 
+    // Plan #938 — session working-notes fold (legacy `/api/agent` parity with
+    // the durable in-step resolver): read `meta.workingNotes` off the session
+    // envelope when a sessionId is present and sanitize through the shared
+    // client-safe predicate (poison → unset). Fail-open: any store problem →
+    // no notes block (the turn proceeds exactly as today). The fold is NOT
+    // hot: a note written mid-turn lands on a later model round/turn.
+    let notesPreamble: string | undefined;
+    if (parsed.sessionId) {
+      try {
+        const tenantRes =
+          await services.harnessSessionsRedis.resolveTenantIdForUser(userId);
+        if (tenantRes.ok) {
+          const storeRes = await resolveSessionStore();
+          const store =
+            storeRes.ok && isEnvelopeStore(storeRes.value)
+              ? storeRes.value
+              : undefined;
+          if (store) {
+            const envelope = await store.readEnvelope(
+              sessionKeyFor(tenantRes.value, userId, parsed.sessionId),
+            );
+            notesPreamble = sanitizeWorkingNotes(
+              envelope?.meta?.workingNotes,
+            );
+          }
+        }
+      } catch {
+        notesPreamble = undefined;
+      }
+    }
+
     // Phase 2 (#517) — resolve attached skills (sticky re-read from
     // `meta.attachedSkills` + the current `/slug` attach or `/unskill` detach).
     // Modeled on personaInject but WITHOUT the snapshot lock: skills are
@@ -611,6 +643,7 @@ export async function POST(req: Request): Promise<Response> {
       ...runParams,
       modelId: runParams.modelId,
       ...(personaPreamble ? { personaPreamble } : {}),
+      ...(notesPreamble ? { notesPreamble } : {}),
       ...(skills?.preamble ? { skillsPreamble: skills.preamble } : {}),
       ...(reasoning !== undefined ? { reasoning } : {}),
     };
@@ -628,7 +661,14 @@ export async function POST(req: Request): Promise<Response> {
       k.startsWith('meta_sandbox_'),
     );
     const nonSkillToolCount = Object.keys(extraTools).filter(
-      (k) => k !== 'find_skill' && k !== 'fetch_skill' && !isMetaToolName(k),
+      (k) =>
+        k !== 'find_skill' &&
+        k !== 'fetch_skill' &&
+        !isMetaToolName(k) &&
+        // Plan #938: the working-notes family is always-on like `meta_*` and
+        // must NOT substitute for a real FS/MCP/http surface on a deferred 403
+        // — a notes-only turn would hide the unavailable workspace.
+        !isWorkingNotesToolName(k),
     ).length;
     if (deferredNoFsResponse && !sandboxClient) {
       const canProceed = metaSelectionDeferred