(feat): interim transcript send and minmal token response architecture: - #986
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdds an opt-in speculative turn gate for scripted voice templates. The change introduces constrained interim classification, final-turn commitment, scripted function dispatch, no-LLM pipeline wiring, new template models and flows, supporting benchmarks, and logging safeguards. ChangesSpeculative scripted voice flow
Logging hardening
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
fe5ce2a to
db47adf
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a “speculative” scripted-response path for Breeze Buddy voice calls, where interim transcripts can be classified by the main LLM into a constrained output (line-id / “.” / function call) and then committed on the final transcript to reduce perceived latency. It also adds offline tests and benchmarking/spike scripts around the constrained/speculative modes, plus a small logging safety improvement.
Changes:
- Add
SpeculativeTurnGate+ constrained completer/dispatcher to run main-LLM speculation on interims and commit on final. - Extend template configuration types to support phrase tables, scripted functions, and interim-send cadence controls.
- Add tests + benchmark/spike artifacts, and harden logging (websocket exception logging, redaction of sensitive live config values).
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_speculative_turn_gate.py | Offline unit tests for speculative hold/commit, interruption, cadence, and finalize behaviors. |
| tests/test_speculative_constrained.py | Offline tests for constrained completer/dispatcher (number / “.” / tool-call) and arg recovery. |
| scripts/speculative_modes_bench.py | Benchmark harness comparing constrained-output strategies across models/providers. |
| scripts/speculative_classify_spike.py | Spike script to evaluate interim classification accuracy/TTFT on partial transcripts. |
| redbus-customer-trip-feedback-realtime.json | Template JSON updated to include functions and constrained behavior expectations for realtime usage. |
| new_template.json | Example template updated with phrase table + interim LLM send config + scripted functions. |
| docs/SCRIPTED_RESPONSES_PLAN.md | Updated implementation plan and spike results documentation for speculative scripted responses. |
| bench_results/speculative_modes.json | Captured benchmark output JSON. |
| app/services/live_config/store.py | Redact sensitive config values in logs to prevent credential leakage. |
| app/api/routers/breeze_buddy/websocket.py | Fix Loguru exception logging usage (and related formatting changes). |
| app/ai/voice/agents/breeze_buddy/template/types.py | Add PhraseEntry, InterimLLMSendConfig, ScriptedFunction and config fields on ConfigurationModel. |
| app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py | New processor implementing speculative interim completion + commit-on-final. |
| app/ai/voice/agents/breeze_buddy/processors/init.py | Export SpeculativeTurnGate. |
| app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py | New constrained prompt/tooling implementation for speculative gate. |
| app/ai/voice/agents/breeze_buddy/handlers/internal/end_conversation.py | Fall back to transcript collector when no LLMContext exists (gate/stream modes). |
| app/ai/voice/agents/breeze_buddy/agent/pipeline.py | Wire scripted/no-LLM pipeline variant and insert speculative processor + transcript collector. |
| app/ai/voice/agents/breeze_buddy/agent/init.py | Add gate-mode wiring, system-prompt seeding, and speculative processor construction. |
Suppressed comments (1)
app/api/routers/breeze_buddy/websocket.py:61
- Same as above: this warning uses f-strings, which are formatted even if WARNING logs are filtered out in some environments. Loguru supports lazy
{}formatting here too.
logger.warning(
f"Could not close websocket v2 (likely already closed): "
f"{close_error}"
)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 18
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
app/ai/voice/agents/breeze_buddy/agent/pipeline.py (1)
280-285: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winScripted mode silently discards
user_idle_configuration.
no_llmnow disables user-idle detection for scripted templates.new_template.jsonsetsuser_idle_configuration.enabled: truewith a 20-second timeout and ships anidle_checkphrase (line 366-370) plus anIDLE CHECKinstruction in the system prompt. In scripted mode neither the idle timer nor that phrase can ever fire, and no warning tells the template author.If idle handling is out of scope for scripted mode, log a warning when a scripted template enables it, so the gap is visible.
🛡️ Proposed warning
user_idle_enabled = ( user_idle_config is not None and getattr(user_idle_config, "enabled", False) and not no_llm ) + if ( + is_scripted + and user_idle_config is not None + and getattr(user_idle_config, "enabled", False) + ): + logger.warning( + "[GATE] user_idle_configuration is enabled but scripted mode has " + "no idle detection; the idle message will never fire" + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/agent/pipeline.py` around lines 280 - 285, Update the user-idle configuration handling near user_idle_config and user_idle_enabled to detect when a scripted template (no_llm) enables user_idle_configuration. Log a warning in that case explaining that user-idle detection is disabled for scripted mode, while preserving the existing disabled behavior and normal idle handling for LLM-enabled templates.
🧹 Nitpick comments (17)
scripts/speculative_modes_bench.py (4)
216-217: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove the no-op assignment.
line = linedoes nothing. Delete the branch or add an explicit comment-onlypass.♻️ Proposed refactor
- elif name == "silence": - line = line # null + elif name == "silence": + pass # silence leaves `line` unset🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/speculative_modes_bench.py` around lines 216 - 217, Remove the no-op `line = line` assignment from the `name == "silence"` branch in the surrounding mode-handling logic; delete the branch if no behavior is needed, or replace its body with an explicit comment-only `pass` while preserving silence behavior.Source: Linters/SAST tools
173-173: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRename
land drop the unusedremainingalias.Ruff flags
las ambiguous (E741). Alsoremainingis assignedtat line 178 and never reassigned, so it duplicatest.♻️ Proposed refactor
- lines = [l for l in t.splitlines() if l.strip()] + lines = [ln for ln in t.splitlines() if ln.strip()]- if '"' in remaining: + if '"' in t: try: - feedback = remaining.split('"', 2)[1] + feedback = t.split('"', 2)[1] except Exception: feedback = NoneAlso applies to: 197-201
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/speculative_modes_bench.py` at line 173, Update the line-processing comprehensions in the affected sections to use a descriptive name instead of the ambiguous variable l, and remove the unused remaining alias where it merely duplicates t. Ensure all subsequent references use the renamed line variable or t directly without changing behavior.Source: Linters/SAST tools
32-35: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winBoth spike scripts read
os.environdirectly instead of using the central config module. The shared root cause is that neither script routes credentials throughapp/core/config/static.py, which the guidelines require for all Python files.
scripts/speculative_modes_bench.py#L32-L35: replace the three__import__("os").environ.get(...)calls withget_required_env()fromapp/core/config/static.py, or importosnormally and document the exemption in the module docstring.scripts/speculative_classify_spike.py#L411-L413: replace theos.environ.getreads forAZURE_OPENAI_API_KEY,AZURE_OPENAI_ENDPOINT, andGEMINI_API_KEYwith the same central accessor, or document the same exemption.As per coding guidelines: "Load ALL configuration from
app/core/config/static.pyusingget_required_env()for mandatory variables; never import directly fromos.environelsewhere".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/speculative_modes_bench.py` around lines 32 - 35, Route all required Azure and Gemini configuration through get_required_env() from app/core/config/static.py: update scripts/speculative_modes_bench.py lines 32-35 and scripts/speculative_classify_spike.py lines 411-413 to replace direct os.environ reads, preserving each variable’s existing key and required behavior; do not add a direct os.environ exemption.Source: Coding guidelines
34-34: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRun Black on this file.
Many lines exceed 88 characters and use manual continuation styles that Black rewrites. Ruff also reports E702 on lines 392 and 395. The CI lint gate runs Black and isort. Format the file before merge.
As per coding guidelines: "Format code using Black with line-length=88 and isort with profile=black".
Also applies to: 65-65, 107-108, 121-122, 206-206, 229-229, 231-235, 239-241, 342-350, 454-459
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/speculative_modes_bench.py` at line 34, Run Black with line length 88 and isort using the Black profile on scripts/speculative_modes_bench.py, including the cited long-line and continuation sections. Resolve Ruff E702 findings around lines 392 and 395 while preserving behavior, and ensure the file passes the CI formatting and lint checks.Sources: Coding guidelines, Linters/SAST tools
scripts/speculative_classify_spike.py (2)
178-178: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd type hints to the function signatures.
azure_call,gemini_call,ms,run_model, andmainhave untyped parameters and return types. The coding guidelines require type hints on all function signatures.♻️ Example for `azure_call` and `ms`
-async def azure_call(client, endpoint, key, deployment, user_text): +async def azure_call( + client: httpx.AsyncClient, + endpoint: str, + key: str, + deployment: str, + user_text: str, +) -> dict[str, Any]:-def ms(x): +def ms(x: float | None) -> str:As per coding guidelines: "Include required type hints on all function signatures".
Also applies to: 259-259, 321-321, 325-325, 410-410
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/speculative_classify_spike.py` at line 178, Add complete parameter and return type annotations to azure_call, gemini_call, ms, run_model, and main, using appropriate existing client, argument, and result types while preserving their current behavior.Source: Coding guidelines
422-429: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueReplace the boolean list multiplication.
["gpt-4o-automatic", "gpt-5.4-mini-2"] * have_azurerelies onboolacting as0/1. The intent is a conditional include. Use explicit conditionals.♻️ Proposed refactor
+ model_labels: list[str] = [] + if have_azure: + model_labels += ["gpt-4o-automatic", "gpt-5.4-mini-2"] + if have_gemini: + model_labels.append("gemini-3.5-flash-lite") print( f"phrase table: {len(PHRASES)} ids + '.' | turns: {len(TURNS)} | " - f"models: " - + ", ".join( - ["gpt-4o-automatic", "gpt-5.4-mini-2"] * have_azure - + ["gemini-3.5-flash-lite"] * have_gemini - ) + f"models: {', '.join(model_labels)}" )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/speculative_classify_spike.py` around lines 422 - 429, Update the model list construction in the summary print statement to use explicit conditionals for Azure and Gemini model inclusion instead of multiplying lists by the boolean flags have_azure and have_gemini. Preserve the existing model order and conditional inclusion behavior.docs/SCRIPTED_RESPONSES_PLAN.md (1)
24-29: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAdd a language to the fenced blocks.
markdownlint reports MD040 for both blocks. Use
textto silence it.📝 Proposed fix
-``` +```text user speech ─► Soniox interims ─► [classifier LLM, ~1 token] ─► held intent-``` +```text TODAY (scripted, final-only): stop → endpoint ≤500ms → fire LLM → TTFT ~315ms → TTS ~100ms ≈ 915msAlso applies to: 37-40
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/SCRIPTED_RESPONSES_PLAN.md` around lines 24 - 29, Add the text language identifier to both fenced code blocks in SCRIPTED_RESPONSES_PLAN.md, including the block containing the speech-flow diagram and the separately referenced block around the scripted latency flow, without changing their contents.Source: Linters/SAST tools
app/services/live_config/store.py (1)
211-229: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winNormalize dynamic secret-key names before matching.
_redact_value_for_log()uppercaseskeybut does not normalize separators or camelCase. It redactsAPI_KEY, but notapiKey,api-key, orprivateKey.The Azure path passes dynamic
llm_config.api_key_nametoget_config()inapp/ai/voice/agents/breeze_buddy/llm/__init__.py. If that field accepts these formats,str(value)can still write a credential to the debug log.Normalize key names before matching, or use explicit secret-key metadata. Add tests for every key format accepted by the configuration schema.
As per coding guidelines: configuration constants use SCREAMING_SNAKE_CASE, but dynamic keys require schema validation or normalization before redaction.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/services/live_config/store.py` around lines 211 - 229, Update _redact_value_for_log to normalize dynamic key names by handling separators and camelCase before comparing against _SENSITIVE_KEY_HINTS, so apiKey, api-key, privateKey, and existing SCREAMING_SNAKE_CASE variants are redacted. Add coverage for every accepted key format, including the Azure llm_config.api_key_name path.Source: Coding guidelines
app/api/routers/breeze_buddy/websocket.py (1)
47-60: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winPass exception data to Loguru as arguments.
The f-strings place
e!r,e.args, andclose_errordirectly into Loguru messages. Use{}placeholders and pass values as arguments. This avoids treating exception text as message-format data when wrappers or colorized sinks are enabled.Also verify that production sinks set
diagnose=False. Exception diagnostics can expose local variable values.Suggested logging shape
- logger.exception( - f"An error occurred in the WebSocket v2 handler - " - f"Type: {type(e).__name__}, Message: {e!r}, Args: {e.args}" - ) + logger.exception( + "An error occurred in the WebSocket v2 handler - " + "Type: {}, Message: {!r}, Args: {!r}", + type(e).__name__, + e, + e.args, + ) - logger.warning( - f"Could not close websocket v2 (likely already closed): " - f"{close_error}" - ) + logger.warning( + "Could not close websocket v2 (likely already closed): {!r}", + close_error, + )Loguru documents argument-based formatting and recommends
diagnose=Falsein production to avoid leaking variable values. (raw.githubusercontent.com)Based on learnings: use
logger.opt(exception=e).error(...)or parameterizedlogger.exception(...), and do not interpolate exception objects into Loguru messages.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/api/routers/breeze_buddy/websocket.py` around lines 47 - 60, Update the WebSocket v2 error logging around the logger.exception call to use Loguru "{}" placeholders with e’s type, repr, and args supplied as arguments rather than interpolated into the message; likewise update the logger.warning call in the close-error handler to pass close_error as an argument. Configure the production Loguru sinks with diagnose=False, while preserving traceback logging for the original exception.Sources: Learnings, MCP tools
app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py (2)
333-342: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winLog the swallowed exception from the in-flight speculation.
The bare
except Exception: passhides every failure of the awaited speculation task. When the held result is then missing, the regenerate path reports "no held result" and the real cause is lost.♻️ Proposed fix
except asyncio.TimeoutError: logger.info("[spec-gate] freshest spec late on final") - except Exception: # noqa: BLE001 - pass + except Exception as e: # noqa: BLE001 + logger.opt(exception=e).debug( + "[spec-gate] in-flight speculation raised on final" + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py` around lines 333 - 342, Update the exception handler around awaiting self._inflight in the finalization path to log the caught exception with logger, while preserving the existing timeout-specific message and subsequent self._inflight cleanup.Source: Linters/SAST tools
464-478: 🩺 Stability & Availability | 🔵 Trivial | ⚖️ Poor tradeoffThe awaited dispatch keeps
_busyset for the whole handler.
_commitawaitsself._dispatch_fn(...)on the finalize task, so_busystaysTrueuntil the builtin handler returns. Forend_conversationthat covers the database write, every end-of-call callback, and theEndFramequeue. During that window_on_interimdrops every interim and_on_finaldrops every final. The inlineNOTEstates that the wiring step will move this to a tracked background task, but this pull request contains the wiring step and the change is not present.For a terminal handler the impact is small, because the call is ending. For a non-terminal handler such as
update_outcomeorget_current_timethe user's next turn is lost.Do you want me to open an issue to track moving the dispatch to a tracked background task?
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py` around lines 464 - 478, Update the speak-then-call path in _commit so self._dispatch_fn is launched as a tracked background task instead of being awaited inline, allowing _busy to be released while the handler runs. Preserve the existing function name, arguments, history snapshot, and exception logging, and remove or update the stale NOTE once the wiring is implemented.new_template.json (1)
1-7: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMove this template dump out of the repository root and give it a descriptive name.
new_template.jsonis a database template export (it carriesid,created_at,updated_at,telephony_number_id, and reseller/merchant identifiers) committed at the repository root under a generic name. Two problems follow. First, the name says nothing about the template, so it will collide with the next export. Second, a root-level export is easy to mistake for a seedable artifact. Place it beside the other template assets (for example atemplates/ordocs/fixture directory) and name it after the campaign, asredbus-customer-trip-feedback-realtime.jsondoes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@new_template.json` around lines 1 - 7, Move the database export currently named new_template.json out of the repository root into the existing template-assets or documentation fixture directory, and rename it to redbus-customer-trip-feedback-realtime.json. Preserve the export contents while ensuring its location clearly distinguishes it from seedable artifacts.tests/test_speculative_turn_gate.py (1)
56-62: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse the public
set_taskand add coverage for two untested paths.
_make_gateassignsgate._taskdirectly.SpeculativeTurnGate.set_taskexists for exactly this purpose, and using it keeps the test honest about the wiring the agent performs.Two behaviors have no test:
_close(), reached throughEndFrameandCancelFrame, which must cancel and drain both tasks and clear_busy.- The
_history_captrim in_record_turn, which drops the oldest turns once the buffer exceeds the cap.♻️ Proposed change to the helper
- gate._task = FakeTask(order) + gate.set_task(FakeTask(order)) return gate, clf_calls, dispatched, orderDo you want me to generate the two missing tests?
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_speculative_turn_gate.py` around lines 56 - 62, Update _make_gate to assign the fake task through SpeculativeTurnGate.set_task instead of accessing _task directly, then add tests covering EndFrame and CancelFrame closure that cancel and drain both tasks while clearing _busy, plus a _record_turn test verifying the oldest turns are removed when the history exceeds _history_cap.app/ai/voice/agents/breeze_buddy/template/types.py (2)
1642-1647: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAdd type hints to the validator signature.
_type_positivehas no annotations. The coding guidelines require type hints on all function signatures.♻️ Proposed fix
`@field_validator`("type") `@classmethod` - def _type_positive(cls, v): + def _type_positive( + cls, v: Union[Literal["every"], int] + ) -> Union[Literal["every"], int]: if isinstance(v, int) and v <= 0: raise ValueError("interval type must be a positive int (ms) or 'every'") return vAs per coding guidelines: "Include required type hints on all function signatures".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/template/types.py` around lines 1642 - 1647, Add type annotations to the _type_positive validator’s cls and v parameters and its return type, using types consistent with the accepted interval values and the surrounding model definitions while preserving the existing validation behavior.Source: Coding guidelines
1937-1961: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winConsider validating
scripted_functions[*].lineagainstphrasesat parse time.
ScriptedFunction.lineis a 1-based index intophrases._resolve_lineinapp/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py(lines 131-139) returnsNonefor an out-of-range id, so a mistyped id silently removes the closing line before a hang-up. Amodel_validator(mode="after")onConfigurationModelcan reject the template at save time instead. The same validator can check thatrequiredis a subset ofpropertieskeys, which the field description already states.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/template/types.py` around lines 1937 - 1961, Add a model_validator(mode="after") to ConfigurationModel that validates every scripted_functions[*].line is a valid 1-based index into phrases, rejecting non-null out-of-range references at parse time, and verifies each scripted function’s required entries are present in its properties keys. Reuse the existing ConfigurationModel, phrases, and scripted_functions symbols and preserve valid configurations unchanged.app/ai/voice/agents/breeze_buddy/agent/pipeline.py (1)
231-253: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUpdate the docstring for the new scripted mode.
Three statements are now stale:
- The
Argsblock does not documentspeculative_processor.- The
Returnsblock still saystranscript_collectoris "stream mode only"; line 502 now creates it for every no-LLM mode.- The prose describes only agent, stream, and realtime shapes. Scripted mode is a fourth shape and is documented only in the inline comment at lines 264-267.
♻️ Proposed additions
kb_processor: KnowledgeRetrievalProcessor for per-turn KB retrieval (auto_retrieve mode). Agent mode only — inserted between the user aggregator and the LLM; ignored in stream/realtime modes + speculative_processor: SpeculativeTurnGate that owns the turn in + scripted mode. When set, the pipeline is built without an LLM + (same shape as stream mode) and the processor is inserted after + the transcription gate. None for every other mode. Returns: 6-tuple of (pipeline, context, context_aggregator, user_idle_callback_handler, transcription_gate, transcript_collector) @@ - - transcript_collector: TranscriptCollectorProcessor instance (stream mode only, None in agent mode) + - transcript_collector: TranscriptCollectorProcessor instance (stream + and scripted modes, None in agent and realtime modes)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/agent/pipeline.py` around lines 231 - 253, Update the pipeline-building function docstring to document speculative_processor in Args, describe transcript_collector as available for every no-LLM mode rather than stream mode only, and include scripted mode alongside agent, stream, and realtime in the mode/shape description. Align the wording with the existing inline scripted-mode comment and current transcript_collector creation behavior.app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py (1)
514-517: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse exception-aware Loguru logging in these handlers.
Both handlers interpolate the exception text into the message string. Exception text can contain formatting braces, which Loguru then tries to parse.
♻️ Proposed fix
- except Exception as e: # noqa: BLE001 - logger.error( - f"[spec-constrained] hooks for {name!r} failed: {type(e).__name__}: {e}" - ) + except Exception as e: # noqa: BLE001 + logger.opt(exception=e).error( + f"[spec-constrained] hooks for {name!r} failed" + )Apply the same change to the handler failure log at lines 528-531.
Based on learnings: "use Loguru's exception-aware logging in exception handlers, such as
logger.opt(exception=e).error(...)... do not interpolate exception objects directly into Loguru message strings, because exception text may contain formatting braces".Also applies to: 528-531
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py` around lines 514 - 517, Update both exception handlers around the hook failure logs to use Loguru’s exception-aware logging, such as logger.opt(exception=e).error(...), and remove direct interpolation of e or its text from the message. Apply this consistently to the handlers associated with the first failure log and the handler failure log near lines 528-531, while retaining the existing context and hook name.Source: Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/ai/voice/agents/breeze_buddy/agent/__init__.py`:
- Around line 1389-1398: Declare _active_llm, _speculative_processor, and
_speculative_system_holder in the class __init__ alongside the other runtime
attributes, using their appropriate initial values and types. Confirm whether
_active_llm has any readers; if it is only assigned and never consumed, remove
the unnecessary assignments or otherwise retain it only if an existing read
requires it.
- Around line 1101-1118: Update the system-message assembly in the
prepare_initial_node block to exclude assistant-role greeting messages before
populating _speculative_system_holder, so the greeting is only retained as
assistant history. In the exception handler, use Loguru’s exception-aware
logging rather than interpolating the exception into the message, then re-raise
or otherwise propagate the render failure so the cached call cannot proceed with
an empty prompt or hide the error from the lead.
- Around line 1414-1421: Update the scripted-mode decision before
create_services so is_scripted is enabled only when both
enable_interim_llm_send.enable and a non-empty configurations.phrases value are
present. This must make empty phrases use the normal LLM flow with the
appropriate services and flow manager; then remove the redundant empty-phrases
warning/check in the gate setup while preserving the existing gate-building else
branch.
In `@app/ai/voice/agents/breeze_buddy/handlers/internal/end_conversation.py`:
- Around line 150-157: Update the no-LLM transcript handling around the existing
transcription seed so the collector messages are appended with extend rather
than rebinding transcription, preserving prior_generation_messages. In the same
collector branch, populate filtered_transcript consistently with the LLM path
before the hold-transfer publish consumes it, including scripted
hold_and_consult handlers.
In `@app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py`:
- Around line 213-222: Update _recover_missing_int_args to recover ratings only
from the most recent user turn, or from user turns after the latest assistant
prompt requesting that rating; do not scan an arbitrary eight-turn history. Keep
_extract_rating_value’s token parsing unchanged, and ensure unrelated numbers
from older conversation turns cannot be injected into call_args or persisted
through _execute_hooks_async.
- Around line 426-435: Update the _prompt function’s caching logic so it stores
and reuses the generated prompt only when _system_prompt returns a non-empty
value. If system_messages_fn produces no messages and the resulting prompt is
empty, leave prompt_cache unset so later turns retry after
_handle_client_connected populates the holder; preserve normal caching once a
non-empty prompt is available.
In `@app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py`:
- Around line 293-296: Remove transcript and captured argument values from
speculative-path logs: in
app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py at lines
230, 246, 250, 257, 287, 293-296, and 318, log transcript lengths instead of
text; in app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py at
lines 328, 336, 357, 382, 387, and 409, replace user_text excerpts with
transcript lengths, and at lines 500-503 log sorted call-argument keys instead
of argument values. Ensure all affected logs expose only sizes or keys.
In `@app/ai/voice/agents/breeze_buddy/template/field_reference.json`:
- Around line 461-471: Update the scripted_functions example in
field_reference.json to remove the unsupported outcome field and demonstrate
recording the outcome through the supported hooks and expected_fields structure.
Also revise its description to remove the stale claim that each function has an
optional outcome. Apply the same wording correction to
ConfigurationModel.scripted_functions in the ScriptedFunction configuration
documentation.
In `@bench_results/speculative_modes.json`:
- Around line 35-56: Remove the failed Gemini entries from
bench_results/speculative_modes.json and document that Vertex was unreachable,
or regenerate the artifact only after fixing exception handling in run_cell
within scripts/speculative_modes_bench.py. Do not publish all-zero metrics with
nonzero n_calls as benchmark measurements; keep regenerated results limited to
successful runs.
In `@docs/SCRIPTED_RESPONSES_PLAN.md`:
- Line 5: Update docs/SCRIPTED_RESPONSES_PLAN.md to reference branch
interim-transcript-to-llm-based-architecture and the implemented symbols/files
app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py and
app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py instead of
the outdated names. In §6, mark Q1 and Q2 as decided according to the existing
pipeline wiring and template models rather than leaving them open.
In `@new_template.json`:
- Around line 397-400: Confirm the production intent of the
enable_interim_llm_send configuration in new_template.json; if production
traffic should be throttled, replace type "every" with the intended integer
cadence, such as 180 milliseconds, while preserving the existing enable setting.
In `@scripts/speculative_classify_spike.py`:
- Around line 355-356: Update the token collection condition in the surrounding
classification flow to append `res["tokens"]` whenever the value is not `None`,
including a valid count of 0; only omit missing token counts.
- Around line 211-255: Update the api-version loop around the streaming request
so retryable HTTP client errors do not return immediately; record the error and
continue to the next version, allowing 2024-06-01 to be attempted after an
unsupported 2024-10-21 response. Preserve immediate success handling and
exception behavior, and retain the final exhausted-api-versions result when all
versions fail.
In `@scripts/speculative_modes_bench.py`:
- Around line 248-249: Update azure_call to accept a shared httpx.AsyncClient
argument and remove its per-request AsyncClient creation. Create the client once
outside the measured call, then pass it into each azure_call invocation so
connection setup and reuse are handled consistently with the Gemini benchmark.
- Around line 461-464: Update the write_text call in the benchmark output flow
to explicitly use UTF-8 encoding when writing the JSON generated by json.dumps,
preserving the existing output path and serialization options.
- Around line 48-50: Update the ROLE construction to join the role and task
message contents as one combined sequence, ensuring a "\n\n" separator also
appears between the final role message and the first task message while
preserving the existing content order.
- Around line 46-47: Update the module-level template loading around _T and
_NODE to resolve new_template.json from the repository root using
Path(__file__).resolve().parents[1], matching the approach in
speculative_classify_spike.py, so imports work regardless of the current working
directory.
- Around line 389-394: Update the exception handler around call_fn in the
benchmark flow to log the caught exception, remove the zero-value appends and
valid_total increment, and continue without recording a sample so failed calls
do not affect latency statistics. Also split the semicolon-separated statements
in this handler to satisfy Ruff E702.
---
Outside diff comments:
In `@app/ai/voice/agents/breeze_buddy/agent/pipeline.py`:
- Around line 280-285: Update the user-idle configuration handling near
user_idle_config and user_idle_enabled to detect when a scripted template
(no_llm) enables user_idle_configuration. Log a warning in that case explaining
that user-idle detection is disabled for scripted mode, while preserving the
existing disabled behavior and normal idle handling for LLM-enabled templates.
---
Nitpick comments:
In `@app/ai/voice/agents/breeze_buddy/agent/pipeline.py`:
- Around line 231-253: Update the pipeline-building function docstring to
document speculative_processor in Args, describe transcript_collector as
available for every no-LLM mode rather than stream mode only, and include
scripted mode alongside agent, stream, and realtime in the mode/shape
description. Align the wording with the existing inline scripted-mode comment
and current transcript_collector creation behavior.
In `@app/ai/voice/agents/breeze_buddy/llm/speculative_constrained.py`:
- Around line 514-517: Update both exception handlers around the hook failure
logs to use Loguru’s exception-aware logging, such as
logger.opt(exception=e).error(...), and remove direct interpolation of e or its
text from the message. Apply this consistently to the handlers associated with
the first failure log and the handler failure log near lines 528-531, while
retaining the existing context and hook name.
In `@app/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.py`:
- Around line 333-342: Update the exception handler around awaiting
self._inflight in the finalization path to log the caught exception with logger,
while preserving the existing timeout-specific message and subsequent
self._inflight cleanup.
- Around line 464-478: Update the speak-then-call path in _commit so
self._dispatch_fn is launched as a tracked background task instead of being
awaited inline, allowing _busy to be released while the handler runs. Preserve
the existing function name, arguments, history snapshot, and exception logging,
and remove or update the stale NOTE once the wiring is implemented.
In `@app/ai/voice/agents/breeze_buddy/template/types.py`:
- Around line 1642-1647: Add type annotations to the _type_positive validator’s
cls and v parameters and its return type, using types consistent with the
accepted interval values and the surrounding model definitions while preserving
the existing validation behavior.
- Around line 1937-1961: Add a model_validator(mode="after") to
ConfigurationModel that validates every scripted_functions[*].line is a valid
1-based index into phrases, rejecting non-null out-of-range references at parse
time, and verifies each scripted function’s required entries are present in its
properties keys. Reuse the existing ConfigurationModel, phrases, and
scripted_functions symbols and preserve valid configurations unchanged.
In `@app/api/routers/breeze_buddy/websocket.py`:
- Around line 47-60: Update the WebSocket v2 error logging around the
logger.exception call to use Loguru "{}" placeholders with e’s type, repr, and
args supplied as arguments rather than interpolated into the message; likewise
update the logger.warning call in the close-error handler to pass close_error as
an argument. Configure the production Loguru sinks with diagnose=False, while
preserving traceback logging for the original exception.
In `@app/services/live_config/store.py`:
- Around line 211-229: Update _redact_value_for_log to normalize dynamic key
names by handling separators and camelCase before comparing against
_SENSITIVE_KEY_HINTS, so apiKey, api-key, privateKey, and existing
SCREAMING_SNAKE_CASE variants are redacted. Add coverage for every accepted key
format, including the Azure llm_config.api_key_name path.
In `@docs/SCRIPTED_RESPONSES_PLAN.md`:
- Around line 24-29: Add the text language identifier to both fenced code blocks
in SCRIPTED_RESPONSES_PLAN.md, including the block containing the speech-flow
diagram and the separately referenced block around the scripted latency flow,
without changing their contents.
In `@new_template.json`:
- Around line 1-7: Move the database export currently named new_template.json
out of the repository root into the existing template-assets or documentation
fixture directory, and rename it to redbus-customer-trip-feedback-realtime.json.
Preserve the export contents while ensuring its location clearly distinguishes
it from seedable artifacts.
In `@scripts/speculative_classify_spike.py`:
- Line 178: Add complete parameter and return type annotations to azure_call,
gemini_call, ms, run_model, and main, using appropriate existing client,
argument, and result types while preserving their current behavior.
- Around line 422-429: Update the model list construction in the summary print
statement to use explicit conditionals for Azure and Gemini model inclusion
instead of multiplying lists by the boolean flags have_azure and have_gemini.
Preserve the existing model order and conditional inclusion behavior.
In `@scripts/speculative_modes_bench.py`:
- Around line 216-217: Remove the no-op `line = line` assignment from the `name
== "silence"` branch in the surrounding mode-handling logic; delete the branch
if no behavior is needed, or replace its body with an explicit comment-only
`pass` while preserving silence behavior.
- Line 173: Update the line-processing comprehensions in the affected sections
to use a descriptive name instead of the ambiguous variable l, and remove the
unused remaining alias where it merely duplicates t. Ensure all subsequent
references use the renamed line variable or t directly without changing
behavior.
- Around line 32-35: Route all required Azure and Gemini configuration through
get_required_env() from app/core/config/static.py: update
scripts/speculative_modes_bench.py lines 32-35 and
scripts/speculative_classify_spike.py lines 411-413 to replace direct os.environ
reads, preserving each variable’s existing key and required behavior; do not add
a direct os.environ exemption.
- Line 34: Run Black with line length 88 and isort using the Black profile on
scripts/speculative_modes_bench.py, including the cited long-line and
continuation sections. Resolve Ruff E702 findings around lines 392 and 395 while
preserving behavior, and ensure the file passes the CI formatting and lint
checks.
In `@tests/test_speculative_turn_gate.py`:
- Around line 56-62: Update _make_gate to assign the fake task through
SpeculativeTurnGate.set_task instead of accessing _task directly, then add tests
covering EndFrame and CancelFrame closure that cancel and drain both tasks while
clearing _busy, plus a _record_turn test verifying the oldest turns are removed
when the history exceeds _history_cap.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 318e3365-d822-4780-9241-600e97ecc175
📒 Files selected for processing (18)
app/ai/voice/agents/breeze_buddy/agent/__init__.pyapp/ai/voice/agents/breeze_buddy/agent/pipeline.pyapp/ai/voice/agents/breeze_buddy/handlers/internal/end_conversation.pyapp/ai/voice/agents/breeze_buddy/llm/speculative_constrained.pyapp/ai/voice/agents/breeze_buddy/processors/__init__.pyapp/ai/voice/agents/breeze_buddy/processors/speculative_turn_gate.pyapp/ai/voice/agents/breeze_buddy/template/field_reference.jsonapp/ai/voice/agents/breeze_buddy/template/types.pyapp/api/routers/breeze_buddy/websocket.pyapp/services/live_config/store.pybench_results/speculative_modes.jsondocs/SCRIPTED_RESPONSES_PLAN.mdnew_template.jsonredbus-customer-trip-feedback-realtime.jsonscripts/speculative_classify_spike.pyscripts/speculative_modes_bench.pytests/test_speculative_constrained.pytests/test_speculative_turn_gate.py
d8bb083 to
ea77b8a
Compare
ea77b8a to
3c7b890
Compare
Summary by CodeRabbit