FE-1574: Let Voice and the typed panel share one canonical Brunch conversation

[lunelson]

🌟 What is the purpose of this PR?

Make the mounted Flue conversation route the only product door into a Brunch conversation, and make Voice a faithful audio projection of that one canonical conversation. In the Petrinaut Brunch panel you can type or speak to Brunch in one conversation; what you hear is exactly what Brunch wrote; Stop asks Brunch to stop rather than only hiding its answer; and reopening the panel shows the same conversation without re-sending or replaying anything.

The production throughline is the local Petrinaut surface under yarn dev:brunch: the typed panel's useChat and the Voice control both reach one browser Flue client at the same-origin-proxied /agents/chat/:instanceId route, behind the single ownership guard, into the mounted ChatAgent. The former Brunch /api/chat handler, projector, and proxy are deleted.

What the proof establishes: one typed turn and one finalized spoken turn each cross the Flue route exactly once, the typed turn returns as one finite AI SDK stream, the Voice turn returns as visible text and TTS of the same canonical words, Stop uses Flue's durable abort, and reopen rehydrates from Flue observation. What it does not claim: trusted production authentication or authorization, remote deployment, structured brunch_ask questions (that capability stays suspended), Petrinaut mutation tools, or product acceptance. Acceptance waits on the human Voice witness in proof leaf 8.

🔗 Related links

• Linear: FE-1574 (internal)
• Mission authority: libs/@hashintel/brunch-agent/MISSION.md
• Evidence record: docs/evidence/implementations/mission-5-direct-voice-flue/README.md
• Read-only Voice source evidence: FE-1570: Optimize spoken Brunch responses for Voice mode #9496, H-6763: Add temporary Brunch ask shim for Voice preview #9507, H-6800: Improve Petrinaut Voice turn-taking and answer provenance #9512

🚫 Blocked by

• FE-1563: Establish Brunch elicitation skill boundaries #9468 (Mission 4). This branch is stacked on it, so this diff against main includes Mission 4's commits until FE-1563: Establish Brunch elicitation skill boundaries #9468 merges or this PR is rebased onto it.
• Proof leaf 8: the human Voice witness and retained evidence bundle have not been run.

🔍 What does this change?

• @hashintel/brunch-agent-transport-aisdk becomes the browser-side adapter over public @flue/sdk: a ChatTransport that admits one send() per turn, projects the wait() event stream into UIMessageChunks, and projects Flue history snapshots into AI SDK messages. It no longer contains a server handler.
• The Brunch app stops mounting /api/chat; the local launcher proxies /agents/chat/* instead. Ownership is derived from principal plus logical conversation id in one guard.
• The Petrinaut website's Brunch demo supplies that transport to the panel, tracks admissions and response messages per conversation, hydrates canonical history through SDK observation, and exposes a durable requestStop.
• The Voice control submits a finalized Realtime answer through the panel's transport, waits for the real Flue admission, correlates the reply by server submissionId, and speaks exactly the canonical text.
• Petrinaut's AI panel accepts a host requestStop so Stop can become durable before the local stream is cancelled (changeset: .changeset/durable-brunch-stop.md).
• Plugin packages move to source-authored skills with flue.ts entrypoints; YAML plugin machinery is removed.

🏗️ Agent notes

Mission authority is libs/@hashintel/brunch-agent/MISSION.md (live as of 2026-09-03). The six sections, condensed:

Imperative. One Flue route is the only product door into a Brunch conversation; Voice is an audio projection of that conversation. Now, because Mission 4 fixed the canonical agent composition while two transports still reached the same conversation.

Throughline. Petrinaut typed panel (useChat with a host-supplied browser ChatTransport) and Petrinaut microphone (OpenAI Realtime, one validated finalized continue_interview answer) → one browser createFlueClient() per principal + logical conversation id → one send() at the same-origin-proxied /agents/chat/:instanceId → agentOwnershipGuard → createAgentRouter(ChatAgent) with useBrunchAgent() + useSdcpnPlugin() → wait(admission, { onEvent }) for the finite per-turn stream, observe({ live: "sse" }) for canonical state and reopen → parts correlated by submissionId → panel projector terminated on submission-settled; Voice speaks canonical completed text unchanged.

Proof. Nine leaves with named oracles: (1) typed panel over the browser transport, (2) direct finalized admission, (3) no server-side AI SDK door, (4) canonical visible and TTS output, (5) cancellation and abort stay distinct, (6) reopen resumes without replay, (7) architecture and boundary integrity, (8) real Voice witness and retained proof bundle, (9) focused repository verification and truthful docs. Leaves 1 to 7 and 9 have automated evidence recorded in the evidence README; leaf 8 is open.

Constraints. Flue history is the sole canonical record; @flue/sdk is used directly with no re-specified offsets, retries, or recovery; one product route; the panel stays on useChat; only the validated finalized answer is submitted and never automatically retried; Brunch owns response content; the four cancellation kinds stay distinguishable; the local principal is a discriminator, not authentication; the transport package depends only on ai and @flue/sdk; brunch_ask is not mounted.

Fog-line. How Voice enters Flue (shape B, via the panel transport, is what shipped); the home of the identity and header contract; whether wait() alone gives a clean per-turn stream or reconnect needs observe(); whether reconnectToStream resumes; which parts of #9496/#9507/#9512 to port; speech selection for multi-block responses; whether one Stop affordance can express local interruption and durable abort; whether the same-origin proxy carries every SDK route.

Stop or reorient. Stop on a second conversation authority, any non-Flue turn route, provisional STT submission, automatic retry of ambiguous admission, canonical text rewritten by another model, hand-rolled recovery, restored stub agent, or activated brunch_ask; stop if client-tool resume semantics need a server helper; stop if local cancellation aborts durable work or Stop only cancels the browser request.

Deferred items are recorded in MISSION.md §Deferred and MISSION.next.md.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• modifies an npm-publishable library and I have added a changeset file(s) (@hashintel/petrinaut: .changeset/durable-brunch-stop.md and .changeset/voice-input-withdrawal-and-stop-correlation.md; the @hashintel/brunch-agent* packages are private)

📜 Does this require a change to the docs?

The changes in this PR:

• require changes to docs which are made as part of this PR (libs/@hashintel/petrinaut/docs/ai-assistant.md, Brunch evidence and mission records)

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• affected the execution graph, and the turbo.json's have been updated to reflect this (plugin packages gained build, lint:*, and test:unit scripts; apps/brunch-agent, core, and plugin-dafny turbo.json updated)

⚠️ Known issues

The ten review-bot findings on this PR were read as samples of three underlying faults and all are addressed in the review-fix commit, with a failing-first test for each:

• Stop was not correlated to the turn it targeted. A Stop pressed while send() was still in flight now waits for that admission before Flue's conversation-wide abort, so aborted: false no longer reads as "already settled" while the admitted turn keeps running. A durable Stop result or failure that lands after a newer turn started is ignored. A Voice turn whose submission settles aborted closes its Realtime function call without speaking, driven by Flue's settlement index rather than chat status, because the AI SDK reports ready between a step and its automatic client-tool follow-up.
• Live-stream and snapshot projections encoded the same rules twice. The snapshot projector keeps providerExecuted on a server tool that is still running and folds a client-tool continuation back into the assistant message it resumed, matching the live stream. The transport reports the resumed assistant id to the host, and the tracker records every submission that wrote a message, so Voice correlates by membership and speaks a reply whether the continuation was admitted by the panel after a readPetrinautDoc call or by Voice answering a pending brunch_ask. One catalog (brunch-client-tools.ts) now feeds the panel transport and the history projection, and a test asserts every registered interactive widget answers a tool in it. Host-owned history is hydrated only once the snapshot carries every locally streamed reply.
• A withdrawn Voice input could still be sent. A retained voice input carries an AbortSignal; Petrinaut drops it from the queue and rejects it when the signal fires.
• Smaller fixes: a consumer stream.cancel() no longer enqueues on a closed controller, and first-canonical-text latency is recorded when the first completed canonical text block appears rather than at settlement.

brunch_ask remains suspended on the server. Its browser paths are now consistent but carried as latent code, not as a claim.

🐾 Next steps

• Run the proof leaf 8 human witness and retain the evidence bundle; the Stop step of the demo script exercises the settlement-driven Voice stop for the first time on the real route.
• Consider an equivalence test between the live projector and the snapshot projector over one shared fixture, so the two cannot drift again silently.

🛡 What tests cover this?

• libs/@hashintel/brunch-agent/packages/transport-aisdk/test/chat-transport.test.ts, transcript.test.ts, ui-stream.test.ts
• apps/petrinaut-website/src/main/app/voice-interview/realtime-brunch-bridge.test.ts, voice-interview-control.test.tsx, voice-turn-controller.test.ts, canonical-speech.test.ts
• apps/petrinaut-website/src/main/app/local-storage-demo/brunch-panel-transport.test.ts, local-storage-demo-app.test.tsx
• libs/@hashintel/petrinaut/src/ui/views/Editor/panels/ai-assistant-panel.test.tsx
• apps/brunch-agent boundary and ownership tests
• Focused command (recorded in the evidence README): yarn exec turbo run lint:tsc lint:eslint test:unit build --filter @apps/brunch-agent --filter @apps/petrinaut-website --filter @hashintel/petrinaut --filter @hashintel/brunch-agent --filter @hashintel/brunch-agent-plugin-sdcpn --filter @hashintel/brunch-agent-transport-aisdk

❓ How to test this?

1. Run yarn dev:brunch and open the Petrinaut Brunch preview.
2. Type one message and read the reply.
3. Start Voice mode and speak one answer. Confirm exactly one new user message appears, then Brunch's reply appears as text and is read aloud in the same words.
4. Speak over the playback once. Playback stops and the text stays.
5. Ask a second question and press Stop while Brunch is still working. Confirm the conversation shows that turn as stopped, or the documented already-settled race, rather than a browser-only cancellation.
6. Close and reopen the panel on the same conversation. Confirm nothing is re-sent or replayed.

📹 Demo

Pending the proof leaf 8 witness recording.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hash	Ready	Preview	Sep 4, 2026 9:55am UTC
petrinaut	Ready	Preview	Sep 4, 2026 9:55am UTC
petrinaut-docs	Ready	Preview	Sep 4, 2026 9:55am UTC

1 Skipped Deployment

Project	Deployment	Actions	Updated
hashdotdesign-tokens	Ignored	Preview	Sep 4, 2026 9:55am UTC

[codspeed-hq]

Merging this PR will not alter performance

✅ 98 untouched benchmarks

---

_{Comparing ln/fe-1574-direct-voice-flue (eecbe99) with ln/fe-1563-redesign-runbook-workpiece (51bec22)}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit eecbe99. Configure here.}

[cursor]

Abort race leaves unhandled rejection

Medium Severity

submitVoiceInputWithAdmission races Promise.all against the input AbortSignal. When abort wins, the composer promise can still reject afterward, and that rejection is no longer awaited. Withdrawing a queued Voice answer therefore surfaces an unhandled rejection even though the helper already failed the caller.

 

^{Reviewed by Cursor Bugbot for commit eecbe99. Configure here.}

[cursor]

Empty snapshot can latch hydration

Medium Severity

The panel treats a host message list of [] as complete history and latches hydratedConversationIdRef. useFlueChatHistory publishes that empty list whenever observation is absent. If Flue later becomes live with the real conversation, hydration never runs again, so reopen can show a blank panel instead of the canonical transcript.

Additional Locations (1)

• apps/petrinaut-website/src/main/app/local-storage-demo/use-flue-chat-history.ts#L98-L109

 

^{Reviewed by Cursor Bugbot for commit eecbe99. Configure here.}