FE-1580: Reconcile Voice turn behavior on the shared Brunch conversation - #9531
Draft
kostandinang wants to merge 52 commits into
Draft
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Author
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
5 tasks
lunelson
changed the base branch from
ln/fe-1574-direct-voice-flue
to
graphite-base/9531
September 4, 2026 09:38
4 tasks
Amp-Thread-ID: https://ampcode.com/threads/T-01a06941-3ee7-7482-a82e-8a16c7141785 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a06941-3ee7-7482-a82e-8a16c7141785
Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a06941-3ee7-7482-a82e-8a16c7141785
Amp-Thread-ID: https://ampcode.com/threads/T-01a06941-3ee7-7482-a82e-8a16c7141785 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06cf5-75d4-7763-addd-4a881479b20f Co-authored-by: Amp <amp@ampcode.com>
Merging this PR will not alter performance
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
as_constant |
< 1 ns | < 1 ns | N/A | |
constant_equal |
< 1 ns | < 1 ns | N/A | |
constant_not_equal |
< 1 ns | < 1 ns | N/A | |
access |
< 1 ns | < 1 ns | N/A | |
runtime_equal |
< 1 ns | < 1 ns | N/A | |
runtime_not_equal |
< 1 ns | < 1 ns | N/A |
Comparing kostandin/fe-1580-harden-voice-safety-and-ux-on-the-unified-flue-route (756ed4f) with ln/fe-1574-direct-voice-flue (eecbe99)
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d1b-91ad-741d-ae26-ca27b1ba20c5 Co-authored-by: Amp <amp@ampcode.com>
Contributor
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d1b-91ad-741d-ae26-ca27b1ba20c5 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d1b-91ad-741d-ae26-ca27b1ba20c5 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d1b-91ad-741d-ae26-ca27b1ba20c5 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d1b-91ad-741d-ae26-ca27b1ba20c5 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d1b-91ad-741d-ae26-ca27b1ba20c5 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d67-bcd6-77dc-b3c4-4b51537a64c8 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a06d67-bcd6-77dc-b3c4-4b51537a64c8 Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

🌟 What is the purpose of this PR?
This PR builds on #9528's unified Flue conversation route and restores the Voice safety and UX behavior needed on top of it.
A completed spoken transcript now follows the same path as typed input:
Voice → shared panel useChat → browser Flue transport → /agents/chat/:instanceIdThe response shown in the panel is the same canonical Brunch text passed to TTS. Initial speech can now start from a durably completed, submission-correlated canonical segment while the rest of the Flue submission continues; settlement still gates replay and the next input turn. This PR does not restore Brunch
/api/chat, add a direct VoiceFlueClient.send()path, rewrite responses with a simplifier, mountbrunch_ask, or include Petri-net generation.🔗 Related links
756ed4f63e756ed4f63eeecbe99e201f51bec22bb374🚫 Blocked by
🔍 What does this change?
gpt-4o-transcribeoutput the sole authoritative spoken answer.message-startedandmessage-completedboundaries with stream position and submission identity, then schedules only new finalized canonical segments in stable canonical order.brunch_askbehavior from the production Voice path.Observed timing diagnostic
These are single real microphone-to-audible-TTS diagnostic turns, not the controlled 10+10 comparative acceptance sample. The post-change target gap was only 0.7 ms, so this change did not meaningfully improve that turn. Its dominant 5.10 s admission-to-canonical delay belongs to the tool/construction work in #9538, not Mission 5.
Acceptance ledger
local-storage-demo-app.tsx,brunch-panel-transport.ts, and panelsubmitVoiceInput; repository audit finds only the OpenAI data-channel.send()in Voice sourcesubmitVoiceInputWithAdmission→ bridge preserves rejection, 409 originalsubmissionId, ambiguity, and local abort; every failure path callssend()once and never retriesFlueClient.abort()interactiveTools: []; canonical speech ignoresbrunch_ask; dormant source is unmounted756ed4f63e; the prior seven-workspace gate passed atf3f5c4ebc7; current CI is pendingErrorcauses, strings, and plain-object JSON across both Flue failure paths; AI errors persist with wrapped detail, copy, and close actions; browser-tool rejections enter AI SDKoutput-errorstate and failed tool cards lead with the inline errorbrunch_askshim unmounted and unrecognizedHow donor PRs are used
The existing Voice PRs are reference implementations, not branches merged into this stack. Their useful behavior was reimplemented against #9528's Flue architecture:
brunch_askshim is no longer part of the architecture.The donor PRs remain open and unchanged until this replacement is accepted and their owners approve retirement.
Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
@hashintel/petrinautpackage and includes a patch changeset.📜 Does this require a change to the docs?
🕸️ Does this require a change to the Turbo Graph?
🐾 Next steps
🛡 What tests cover this?
756ed4f63epasses the complete transport suite (32/32) and Petrinaut website suite (289/289), both production builds, both TypeScript checks, and both ESLint checks with zero errors. One pre-existing React effect warning remains.f3f5c4ebc7ran the full required Turbo command locally with exit 0, 39/39 tasks and 1,123/1,123 tests — Brunch app 80, core 86, binding 18, SDCPN 8, transport 32, Petrinaut 617, website 282.submissionId, keeping local abort distinct from durable abort, and callingsend()once per path.yarn lint:formataccepts all 5,527 matched files, andgit diff --checkpasses.[object Object]; browser-tool rejection entersoutput-error; and error toasts expose persistent detail, copy, and Ark close actions. Chrome inspection confirms the compact and expanded consent renders, ghost collapse control, wrapped persistent error detail, and inline tool-card failure.9d753b2068d3:yarn workspace @apps/brunch-agent vitest run test/local-dev-origins.test.tspasses 4/4; pre-commit formatting and task-dependency checks pass.d64f809c2233has a 60-file diff. Its code-push GitHub check set completed with 70 successful and 8 intentionally skipped checks, no failures or cancellations: Preflight, CodSpeed, Bench, Semgrep, Lint, Test (including Brunch units and Playwright), and Deploy all pass. Subsequent PR-metadata Preflight checks also pass.d64f809c2233CI. The current compact-dock refinement is pushed and locally verified; CI is pending.❓ How to test this?
📹 Demo
One real before/after microphone-to-audible timing diagnostic is recorded above. The complete retained browser/microphone/handoff/Stop/reload witness remains pending.