Conversation
There was a problem hiding this comment.
Pull request overview
Adds configurable ASR support to the oracle server, making OpenAI Realtime transcription the default while keeping Google Speech-to-Text available as an alternative provider.
Changes:
- Introduces shared ASR processor implementations for OpenAI Realtime transcription and Google Cloud Speech-to-Text.
- Makes ASR provider/model/language configurable via
--asr-provider,--asr-model, and--asr-languageinkame.server_oracle. - Updates README runtime notes and quickstart guidance for the new default ASR setup.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| src/kame/server_oracle.py | Replaces the inline Google-only ASR implementation with a configurable ASR provider abstraction and new CLI flags. |
| src/kame/asr_processors.py | Adds the new ASR provider implementations + factory/initialization gate used by the oracle server. |
| README.md | Updates runtime notes and setup instructions to reflect OpenAI ASR default and optional Google STT usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def create_asr_processor( | ||
| provider: ASRProvider, | ||
| sample_rate: int, | ||
| model: str = DEFAULT_OPENAI_ASR_MODEL, | ||
| language: str | None = None, | ||
| ) -> ASRProcessor: | ||
| if provider == "openai": | ||
| return OpenAIRealtimeASRProcessor( | ||
| sample_rate=sample_rate, | ||
| model=model, | ||
| language=language or DEFAULT_OPENAI_ASR_LANGUAGE, | ||
| ) | ||
| if provider == "google": | ||
| return GoogleASRProcessor( | ||
| sample_rate=sample_rate, | ||
| language_code=language or DEFAULT_GOOGLE_ASR_LANGUAGE, | ||
| ) | ||
| raise ValueError(f"Unsupported ASR provider: {provider}") | ||
|
|
||
|
|
||
| def require_initialized_asr(enable_asr: bool, asr_processor: ASRProcessor | None) -> None: | ||
| if not enable_asr: | ||
| return | ||
|
|
||
| if asr_processor is not None and asr_processor.asr_enabled: | ||
| return | ||
|
|
||
| reason = "unknown error" | ||
| if asr_processor is not None and asr_processor.init_error: | ||
| reason = asr_processor.init_error | ||
| raise RuntimeError( | ||
| "ASR is enabled but the selected ASR provider could not be initialized. " | ||
| f"{reason} " | ||
| "Set the required credentials for the selected provider or rerun with --no-enable-asr." | ||
| ) |
There was a problem hiding this comment.
This PR introduces a new ASR abstraction layer (provider selection, processor creation, and initialization gating) but there are no unit tests covering the deterministic parts (e.g., create_asr_processor returning the expected implementation, require_initialized_asr raising with the processor's init_error, OpenAI processor setting init_error when OPENAI_API_KEY is missing). Adding a small test module for these cases would help prevent regressions without requiring network calls.
There was a problem hiding this comment.
Added ASR processor tests for OpenAI/Google provider creation, missing credential handling, and ASR initialization error behavior.
| self._process_responses(responses) | ||
| except Exception as e: | ||
| if self.running: | ||
| raise e |
There was a problem hiding this comment.
In this exception handler, raise e will discard the original traceback context. Use a bare raise to preserve the full stack trace (and make debugging streaming failures easier).
| raise e | |
| raise |
| @staticmethod | ||
| def _linear_resample_int16(x_int16: np.ndarray, src_hz: int, dst_hz: int) -> np.ndarray: | ||
| """Very lightweight linear resample to reduce aliasing vs index stepping.""" | ||
| if src_hz == dst_hz: | ||
| return x_int16 | ||
| n_src = len(x_int16) | ||
| n_dst = int(n_src * dst_hz / src_hz) | ||
| if n_dst <= 0: | ||
| return np.zeros(0, dtype=np.int16) | ||
| src_idx = np.arange(n_src, dtype=np.float64) | ||
| dst_pos = np.linspace(0, n_src - 1, n_dst, endpoint=True) | ||
| y = np.interp(dst_pos, src_idx, x_int16.astype(np.float64)) | ||
| y = np.clip(y, -32768, 32767).astype(np.int16) | ||
| return y |
There was a problem hiding this comment.
_linear_resample_int16 is duplicated in both the Google and OpenAI ASR processors. Consider extracting it into a shared helper to avoid divergence if resampling behavior needs to change (e.g., scaling/clipping rules) and to reduce maintenance overhead.
| error = event.get("error", event) | ||
| raise RuntimeError(f"OpenAI Realtime error event: {error}") | ||
|
|
||
| if event_type == "conversation.item.input_audio_transcription.failed": |
There was a problem hiding this comment.
On conversation.item.input_audio_transcription.failed, the code logs but does not clear any buffered partial transcript for the associated item_id. If failures occur, _partial_transcripts can retain entries indefinitely. Consider extracting item_id (if present) and removing it from _partial_transcripts, similar to the completed path.
| if event_type == "conversation.item.input_audio_transcription.failed": | |
| if event_type == "conversation.item.input_audio_transcription.failed": | |
| item_id = event.get("item_id") | |
| if item_id is not None: | |
| self._partial_transcripts.pop(str(item_id), None) |
Summary
This PR adds OpenAI Realtime transcription support for the oracle-guided server and makes the ASR provider selectable from
server_oracle.py.OpenAI ASR is now the default provider, so the quick start path only requires
OPENAI_API_KEY. Google Cloud Speech-to-Text remains available via--asr-provider google.Changes
--asr-provider,--asr-model, and--asr-languageoptions tokame.server_oraclegpt-4o-mini-transcribe--asr-provider googleserver_oracle.pyVerification
GoogleASRProcessorwith--asr-provider googleuv run python -m kame.server_oracle --helpuv run kame-server-oracle --helpuv run python -m py_compile src/kame/asr_processors.py src/kame/server_oracle.pyuv run ruff check src/kame/asr_processors.py src/kame/server_oracle.py pyproject.tomluv run pytest tests/test_server_oracle_logging.pygit diff --check