H-6763: Migrate voice interviews to GPT-Realtime-2

[kostandinang]

🌟 What is the purpose of this PR?

Migrate the H-6763 voice interview from a transcription-plus-TTS chain to gpt-realtime-2 as an ephemeral duplex media shell, while keeping Brunch the sole durable authority for interview state and business logic. Present canonical Brunch output through a warm, professionally neutral expert-interviewer voice.

🔗 Related links

• H-6763
• Parent: H-6763: Add the voice interview experience #9380

🚫 Blocked by

• H-6763: Add the voice interview experience #9380

🔍 What does this change?

• Configures a server-owned gpt-realtime-2 WebRTC session with audio input/output, low-effort reasoning, low-eagerness semantic VAD, automatic interruption, and one required continue_interview function.
• Adds a focused Realtime–Brunch bridge that validates and serializes tool calls, derives stable submission identity, preserves pending brunch_ask correlation, and rejects duplicate, overlapping, or stale work.
• Lets the first complete spoken description start Brunch when no question is pending, while later turns remain correlated with the active question.
• Emits brunch_ask as a dynamic AI SDK tool so Petrinaut recognizes and renders each returned question.
• Keeps the microphone active during Brunch processing and assistant playback, receives remote audio over WebRTC, and exposes orthogonal connection/input/output state to the existing minimal voice UI.
• Clears provisional transcript text after failed turns and surfaces specific correlation, response, and submission diagnostics.
• Gives the Realtime session and canonical response requests a consistent expert-interviewer delivery policy without allowing OpenAI to invent, alter, or answer interview questions.
• Returns canonical Brunch segments through the function result and requests tool-free audio without treating generated speech as the visible source of truth.
• Removes the separate Speech API route, MP3 playback controller, speech queue, and manual done/interrupt controls.
• Updates the voice ADR, Petrinaut website setup, and Petrinaut user guide. Transient implementation plans and design artifacts are excluded from the final diff.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

• require changes to docs which are made as part of this PR

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph

⚠️ Known issues

• Voice remains a non-production preview and is still disabled by server policy in production.
• The live OpenAI preflight used synthetic browser media in the orb; a physical microphone and real acoustic environment still need preview-deployment validation.

🐾 Next steps

• Exercise microphone quality, thinking pauses, expert-interviewer delivery, and barge-in with a physical device in the preview deployment.

🛡 What tests cover this?

• Focused website tests cover Realtime session policy, initial interview startup, pending-question correlation, canonical response delivery, transcript cleanup, diagnostics, reconnect, and cleanup.
• Brunch transport tests cover dynamic brunch_ask tool streaming and return-path handling.
• Integration coverage exercises remote media, semantic turn events, automatic interruption, malformed events, failures, and continuous capture.
• TypeScript checks, ESLint, formatting, architecture docs validation, and the production build pass.
• A live gpt-realtime-2 WebRTC preflight confirmed remote RTP audio, function calling, semantic VAD, and automatic interruption with the configured key.

❓ How to test this?

1. Configure a preview or local environment with PETRINAUT_OPENAI_VOICE_ENABLED=true, OPENAI_VOICE_API_KEY, and a working Brunch endpoint.
2. Start an empty interview, speak a process description, and confirm Brunch starts and returns the first recognized question without an unknown-tool error.
3. Answer the pending question and confirm the answer reaches that brunch_ask call exactly once.
4. Confirm the canonical Brunch response remains visible while Realtime speaks it with the expert-interviewer delivery style.
5. Speak during interviewer audio and confirm playback stops automatically while the microphone remains active.
6. Exercise failed submission, pause, reconnect, end, and text fallback; confirm provisional text clears after failure and old media/events do not survive reconnect or end.

📹 Demo

No screenshot replacement is required; the affected documentation contains no screenshots of this surface.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
petrinaut	Ready	Preview	Sep 2, 2026 9:39am UTC
petrinaut-docs	Ready	Preview	Sep 2, 2026 9:39am UTC

2 Skipped Deployments

Project	Deployment	Actions	Updated
hash	Ignored	Preview	Sep 2, 2026 9:39am UTC
hashdotdesign-tokens	Ignored	Preview	Sep 2, 2026 9:39am UTC

[CLAassistant]

All committers have signed the CLA.

[kostandinang]

• FE-1576: Ground Brunch interviews in the current Petrinaut net #9523
• H-6800: Improve Petrinaut Voice turn-taking and answer provenance #9512
• H-6763: Add temporary Brunch ask shim for Voice preview #9507
• H-6763: Submit only completed Voice transcripts as interview answers #9500
• FE-1570: Optimize spoken Brunch responses for Voice mode #9496
• H-6763: Integrate Voice mode into the AI assistant #9467
• H-6763: Migrate voice interviews to GPT-Realtime-2 #9399 👈 (View in Graphite)
• H-6763: Add the voice interview experience #9380
• H-6763: Improve voice preview reliability diagnostics #9378
• H-6763: Speak finalized Brunch responses with OpenAI #9357
• H-6763: Add OpenAI Realtime voice input to Brunch #9356
• H-6763: Add a generic Petrinaut composer submission API #9355
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cursor]

PR Summary

High Risk
Large real-time voice and interview-turn correlation change (WebRTC, tool bridging, Brunch authority); mitigated by preview-only production disable but still high complexity and regression surface.

Overview
Replaces the transcription-only Realtime + Speech API (MP3) pipeline with a duplex gpt-realtime-2 WebRTC session: remote interviewer audio plays in-browser, the mic stays open through Brunch work and playback, and barge-in is automatic (no manual “Done speaking” or “Interrupt”).

OpenAIRealtimeSession now queues canonical Brunch text as tool-free Realtime audio responses, parses continue_interview tool streams, authorizes only those responses, and fails closed on stray provider audio or malformed tool output.

Adds RealtimeBrunchBridge to validate tool arguments, submit answers through the existing composer with stable IDs and brunch_ask correlation, then return canonical segments via completeFunctionCall before speaking.

Removes /api/voice/speech, SpeechPlaybackController, and related tests; updates the voice UI snapshot to connection / input / output axes and refreshes README voice-preview documentation.

^{Reviewed by Cursor Bugbot for commit b930ef9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 66f7a87. Configure here.}