diff --git a/.claude/agents/swift6-concurrency-reviewer.md b/.claude/agents/swift6-concurrency-reviewer.md index 93164667..612f6fd3 100644 --- a/.claude/agents/swift6-concurrency-reviewer.md +++ b/.claude/agents/swift6-concurrency-reviewer.md @@ -34,11 +34,19 @@ Look at the changes (default to the working diff via `git diff` and ## Project invariants (treat violations as findings) - **Do not reintroduce `AVAudioEngine`/`installTap`.** `MicCapture` deliberately - uses `AVAudioRecorder` with a **fresh recorder per session** to survive input - device switches (`-10868` / all-zero buffers). Flag any move back to a - long-lived engine or tap. + uses an `AVCaptureSession` (`CaptureSessionRecorder`) with a **fresh recorder + per session** to survive input device switches (`-10868` / all-zero buffers). + Flag any move back to a long-lived engine or tap. Session control — building + and `startRunning()` — belongs off the actor on the recorder's serial + `controlQueue`; flag a blocking hardware call made inline on `MicCapture`. +- **Do not reintroduce a warm/prepared recorder.** `warmUp()` is stateless by + measurement (see its doc comment): a warm recorder pre-pays nothing and brings + back a device-identity check, a pin check and a bring-up flag. - No streaming STT, no local models, no separate LLM cleanup pass — cleanup - rides in the Sync STT `prompt`. Flag reintroductions. + rides in the dictation request's `llm` block, as its `instruction` + (`CleanupInstruction`). There is **no** `config.prompt`; the steering field is + `config.conversation_context` (`ConversationContext`). Flag reintroductions of + any of these, `config.prompt` included. - Unit tests use **Swift Testing** (`@Suite`/`@Test`/`#expect`), not XCTest (the `BlurtUITests` XCUITest bundle is the exception), and must never touch the real Keychain (`APIKeyStore`) — use an isolated service. diff --git a/.claude/skills/project-guardrails/SKILL.md b/.claude/skills/project-guardrails/SKILL.md index aecb1a23..7446a3fb 100644 --- a/.claude/skills/project-guardrails/SKILL.md +++ b/.claude/skills/project-guardrails/SKILL.md @@ -24,12 +24,23 @@ genuinely correct, and reaching for it means it's time to stop and ask. ## Audio -- **No `AVAudioEngine` / `installTap` capture path.** `MicCapture` uses - `AVAudioRecorder` with a **fresh recorder per session** on purpose — a - long-lived engine bound its input graph to one device and went stale on a - mic↔built-in switch (`-10868`, all-zero buffers). Keep recording to a 16 kHz - mono 16-bit WAV (the Sync API's geometry) so `stop()` reads it back with no - resample pass. +- **No `AVAudioEngine` / `installTap` capture path.** `MicCapture` builds a + **fresh `AVCaptureSession` recorder per capture** (`CaptureSessionRecorder`; + owner-directed move from `AVAudioRecorder`, 2026-08-25) — a long-lived engine + bound its input graph to one device and went stale on a mic↔built-in switch + (`-10868`, all-zero buffers), so no recorder survives across a device change. + Keep the data output converting to 16 kHz mono 16-bit S16LE (the Sync API's + geometry), captured straight to memory so `stop()` returns upload-ready bytes + with no resample pass. +- **Don't pre-open the mic to make presses feel faster.** `MicCapture.warmUp()` + is stateless on purpose — build a session, drop it — and there is no warm or + prepared recorder to reuse. Measured on hardware: building a session (or the + retired `prepareToRecord()`) leaves the device closed and costs ~15 ms, while + `record()`'s `startRunning()` opens it and costs 180–600 ms, so nothing can + pre-pay the bring-up. A warm-recorder lifecycle was tried, and its + device-identity check, pin check, 60 s expiry and bring-up flag all existed to + protect ~15 ms. If you want the press to feel faster, the liveness gate's + polling is where the remaining latency actually is. ## Transcription pipeline diff --git a/AGENTS.md b/AGENTS.md index 659a2b5d..1f1cad37 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -51,9 +51,13 @@ workflow and the _why_ behind the design; the engine's README covers the _what_ ```text Sources/BlurtEngine/ the engine (dependency-free Swift package) README.md the engine's developer guide (quick start, seams, error table) - Audio/ MicCapture (+meter/+warm), MicLiveness (mic bring-up gate), AudioRoute - (+Monitor)/AudioTransport — CoreAudio routing, SoundPack/Catalog/Store - (the voice *descriptors*; the voices themselves are app-side) + Audio/ MicCapture (+meter) over CaptureSessionRecorder (the + AVCaptureSession backend), MicLiveness (mic bring-up gate), + AudioRoute (+Monitor) — the CoreAudio output route, AudioTransport + — what a transport type means, + AudioInputDevices + MicDeviceStore (microphone selection), + SoundPack/Catalog/Store (the voice *descriptors*; the voices + themselves are app-side) HostIdentity.swift the host's Keychain service, log subsystem, defaults prefix, log directory, product name and release feed — one overridable value Config/ Keychain-backed API key, key terms, developer mode, style profiles, @@ -364,25 +368,26 @@ a rule that outlives its row keeps firing and keeps citing this table while enfo project has already reversed. Reworded the row? Update the anchor. Reversed the decision? Delete the rule along with the row. -| Don't | Because | -| ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | -| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` uses a fresh `AVAudioRecorder` per session. | -| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | -| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `ConversationContext`. | -| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | -| Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_code`** either — the API documents it as defaulting to `en` and as ignored while a custom `prompt` is set, so dropping the prompt un-ignored it; detection was then measured to work with neither field set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included). Setting one would only take that away. `KeytermsWireTests` asserts the absence. | -| Bring back `config.prompt` | Replaced by `config.conversation_context` (`ConversationContext`), which is the structured field for the same job. A custom `prompt` also replaces the service's managed default and makes the API ignore `language_code`, so re-adding one silently gives up both. | -| Widen the request's context past history + the prior chunk | `ConversationContext.turns` reads exactly two fields of `TranscriptionContext` — `recentTranscripts` and `priorText`. The app name, window title, field label and selected text are captured for the paste path and the developer-mode log and stay on the machine; the app/window/field hints and the `Selected text:` block were removed, not gated. Don't add one back, and don't route that context onto the request by another path. The user's key terms are the exception that proves the rule: they _are_ sent, as the request's own `word_boost` list (`KeytermsBoost`) — never folded back into the context turns. | -| Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | -| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | -| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | -| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | -| Add a self-replacing install or background auto-updater | Updates are download-only; `mxcl/AppUpdater` and its in-place updater were removed. The once-a-day launch _check_ (`AutomaticUpdateCheck`) is fine; installing for the user, or polling, is not. Extend `UpdateCheckModel`. | -| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | -| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | -| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | -| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | +| Don't | Because | +| --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | +| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` builds a fresh `AVCaptureSession` recorder per capture (owner-directed move from `AVAudioRecorder`, 2026-08-25). | +| Pre-open the mic to shave bring-up latency (a warm/prepared recorder) | Measured against `kAudioDevicePropertyDeviceIsRunningSomewhere`: neither building an `AVCaptureSession` nor the retired `AVAudioRecorder.prepareToRecord()` opens the device, so neither pre-pays the 180–600 ms route activation `record()` costs. A warm-recorder lifecycle bought ~15 ms and cost a device-identity check, a pin check, an expiry and a bring-up flag. `MicCapture.warmUp()` is stateless: it absorbs the process's first-touch cost and holds nothing. | +| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | +| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `ConversationContext`. | +| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | +| Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_code`** either — the API documents it as defaulting to `en` and as ignored while a custom `prompt` is set, so dropping the prompt un-ignored it; detection was then measured to work with neither field set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included). Setting one would only take that away. `KeytermsWireTests` asserts the absence. | +| Bring back `config.prompt` | Replaced by `config.conversation_context` (`ConversationContext`), which is the structured field for the same job. A custom `prompt` also replaces the service's managed default and makes the API ignore `language_code`, so re-adding one silently gives up both. | +| Widen the request's context past history + the prior chunk | `ConversationContext.turns` reads exactly two fields of `TranscriptionContext` — `recentTranscripts` and `priorText`. The app name, window title, field label and selected text are captured for the paste path and the developer-mode log and stay on the machine; the app/window/field hints and the `Selected text:` block were removed, not gated. Don't add one back, and don't route that context onto the request by another path. The user's key terms are the exception that proves the rule: they _are_ sent, as the request's own `word_boost` list (`KeytermsBoost`) — never folded back into the context turns. | +| Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | +| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | +| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | +| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | +| Add a self-replacing install or background auto-updater | Updates are download-only; `mxcl/AppUpdater` and its in-place updater were removed. The once-a-day launch _check_ (`AutomaticUpdateCheck`) is fine; installing for the user, or polling, is not. Extend `UpdateCheckModel`. | +| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | +| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | +| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | +| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | Release-side invariants (hardened runtime and a secure timestamp on every nested mach-o and embedded framework, or notarization rejects the build; roll-forward-only for a bad release) live in @@ -435,26 +440,41 @@ DictationKeyTap (CGEventTap + DictationKeyGate) → AppCoordinator → Dictation ### `MicCapture` — `Sources/BlurtEngine/Audio/MicCapture.swift` -An actor implementing `MicCaptureProtocol`. It captures with **`AVAudioRecorder`**, recording -straight to a temp 16 kHz / mono / 16-bit PCM WAV — the exact geometry the dictation API wants, so -`stop()` reads the file back as raw S16LE bytes (`Data`, via `AVAudioFile`'s int16 common format) -with no resampling or float-conversion pass. That blob is what the dictation request uploads, byte -for byte. - -A **fresh recorder per session** resolves the current default input device at `record()` time; this -is deliberate — see [Settled decisions](#settled-decisions--dont-reintroduce-these) for the -`AVAudioEngine` failure it replaced. - -**Bluetooth inputs are the reason for four of this actor's moving parts.** Opening the mic on +An actor implementing `MicCaptureProtocol`. It captures with an **`AVCaptureSession`** recorder +(`CaptureSessionRecorder`, used directly — the `CaptureRecorder` protocol that once abstracted two +backends went with the second one; the owner-directed move from `AVAudioRecorder`, 2026-08-25): the +session's audio data output converts to 16 kHz / mono / 16-bit +LPCM — the exact geometry the dictation API wants — and the delegate accumulates the raw S16LE +bytes in memory, so `stop()` returns the blob the dictation request uploads byte for byte, with no +temp file, no read-back, and no resampling or float-conversion pass. + +A **fresh recorder per session**, built around the _current_ resolution of the user's selection at +press time; this is deliberate — see +[Settled decisions](#settled-decisions--dont-reintroduce-these) for the `AVAudioEngine` failure it +replaced. + +**Microphone selection** is that resolution: a device pinned in Settings (`MicDeviceStore`, +persisted as the device UID — `AVCaptureDevice(uniqueID:)` takes the same string) or the system +default input. The transport-keyed policies (liveness cap, tail linger) key off the **pinned** +device, and a pinned device that isn't connected falls back to the system default per press +(`MicDeviceSelection.effective` — pure and unit-tested; the pin stays stored). Enumeration, naming +and presence live in `AudioInputDevices`, on `AVCaptureDevice` — the same API the recorder opens the +device with, so the picker can't offer something the capture path would then call missing. So does +the transport read, once `AVCaptureDevice.transportType` was confirmed on hardware to report the same +four-character codes CoreAudio does for every case the policies turn on — `blue` on AirPods most of +all, since a transport that fails to read as Bluetooth silently costs the 2.5 s liveness cap and the +220 ms tail linger. Hardware-bound and coverage-excluded. + +**Bluetooth inputs are the reason for three of this actor's moving parts.** Opening the mic on AirPods (or any Bluetooth headset) makes the system renegotiate the link into its mic-capable mode — one to two seconds, during which the OS receives no audio at all — and that link then buffers audio in both directions. So: - **`start()` does not return until the input is live.** `record()` returning `true` only means the - AudioQueue started, not that frames are arriving, so `start()` polls the recorder until **both** - its clock has advanced past 0 **and** its meter reads above `MicLiveness.silenceFloorDB`. The + session started running, not that frames are arriving, so `start()` polls the recorder until **both** + it has delivered at least one frame **and** its meter reads above `MicLiveness.silenceFloorDB`. The clock alone is not enough: macOS can deliver all-zero buffers from a stale or not-yet-switched - device, and frames of digital silence advance the clock exactly like real audio — so the gate was + device, and frames of digital silence arrive and count exactly like real audio — so the gate was satisfied on its first ~1 ms poll while the AirPods link was still renegotiating. The floor discriminates _digital_ silence from any real analog input (a live mic in a quiet room still reads its own self-noise, far above it); it is deliberately **not** a speech threshold, which would hang @@ -464,11 +484,35 @@ in both directions. So: and `start()` throws `.audioCaptureFailed`, so the press ends in the same error pill as any other capture failure rather than recording an utterance the mic never heard. (The gate originally failed open, proceeding as if live — which cued the user to speak into a dead mic anyway.) Audio spoken - during the switch cannot be recovered by anything, because nothing ever receives it. `stopGeneration` covers the one suspension - this introduces — a teardown landing mid-wait wins, and the recorder is torn down rather than - installed. `bringingUpCapture` covers the other consequence: across the wait both `activeRecorder` - and `warm` are nil, so the warm-up paths can't infer "no capture in flight" from them (see - `canPrepareWarmRecorder`) or they'd open a second recorder onto the live input. + during the switch cannot be recovered by anything, because nothing ever receives it. Where that cost lands differs by + transport, measured: on a USB interface `startRunning()` blocks for ~600 ms and the first frame + follows ~8 ms later, so the gate confirms almost at once; on AirPods `startRunning()` returns in + ~80 ms and the first frame arrives ~410 ms after that, so the gate is doing the waiting. Both end + in the same place because the wait's clock starts _after_ `record()` returns — a slow open eats + none of the frame-arrival budget, which is what lets the caps be this tight. + + **The bring-up is off-actor.** Building the session and opening the device are both `async`, run + on the recorder's serial `controlQueue`, so neither blocks the actor: the ~600 ms open, and the + build before it (~5 ms warm, ~185 ms on a process's first touch of the capture stack). Only the + selection lookup stays inline, at microseconds once warm. Run inline it did both: a teardown arriving during the open measured **578 ms** of + waiting on a 635 ms open, against **24 µs** once suspended (`MicCaptureBringUpTests`, which warms + the stack first so it measures this rather than a process's first touch, then asserts an absolute + budget that holds on every input). It + also parked a cooperative-pool thread for the duration — the same pool the overlapping context + capture and the transcriber's connection warm-up run on, and starving it is how covering the + retired backend in CI deadlocked the whole test run, which is why the hop is a `DispatchQueue` + rather than a detached `Task`. Through `DictationSession` the win is latent rather than visible: + its serial command queue already runs release and cancel only after the press turn completes, and + `cancel()` claims `.cancelled` synchronously without touching the mic. But `MicCapture` is a + public actor and the contract has to hold for a host that doesn't serialize its own commands. + `stopRunning()` stays inline, at 19–41 ms against the open's ~600. + + The suspension is also why the teardown snapshot moved: `stopGeneration` is now read before the + open rather than before the liveness wait, and checked at both ends, because a stop or cancel can + land while the input is still coming up. `stopGeneration` + covers the suspensions this introduces — a teardown landing anywhere in the bring-up wins, and the + recorder is torn down rather than installed. The check is written at each suspension; the teardown + itself is one `defer`, so a new suspension point inherits it. **A cancel preempts the bring-up rather than queueing behind it**, which took two pieces. The press publishes its task handle (`inFlightPress`) exactly as the pipeline publishes @@ -483,37 +527,35 @@ in both directions. So: `performPress` still consumes the flag before claiming `.recording`, for the narrow window where the cancel lands after the wait returned and there is nothing left to interrupt. -- **The warm recorder is re-armed after every capture**, not just at launch. The cost above is paid - at `prepareToRecord()`, i.e. per session, so warming only the first one hid it for one dictation - out of N. `stop()`/`cancelCapture()` schedule a re-warm; `start()` consumes it. -- **A warm recorder is validated before reuse.** `AVAudioRecorder` resolves its device once and - never re-resolves, so `MicCapture` records the default input's identity (`AudioRoute.currentInput()`) - alongside the warm recorder and discards it when the device has changed — otherwise a recorder - warmed before the user connected their AirPods would keep recording the built-in mic. Unknown - counts as changed. -- **The warm recorder expires** (`preparedRecorderLifetime`, 60 s). A prepared recorder holds the - input device open, which is exactly what pins AirPods in the profile where _output_ audio is - degraded — so it is not held indefinitely. Back-to-back dictations land inside the window; a press - past it just prepares lazily, which is the pre-re-warm behavior. - -The re-warm and the liveness gate are complements, not alternatives: the re-warm shortens how _often_ -the profile switch is paid (a warm recorder has already held the route open), and the gate is what -keeps the app honest on the presses that pay it anyway. +- **There is no warm recorder** — the bring-up cost is not pre-payable, and that is a + measurement, not a preference. Timed against CoreAudio's own + `kAudioDevicePropertyDeviceIsRunningSomewhere` (the bit behind the input indicator): building an + `AVCaptureSession` leaves the device closed and costs ~15 ms, while `record()`'s `startRunning()` + opens it and costs 180 ms on the built-in mic and ~600 ms on a USB interface. The retired + `AVAudioRecorder.prepareToRecord()` measured the same way — ~3 ms, device still closed, and a cold + `record()` after it cost 614 ms versus 585 ms with no prepare at all. **Neither API ever pre-paid + the route activation**, so the machinery that assumed one did (a recorder re-warmed after every + capture, validated against the resolved device identity and pin, a 60 s expiry to un-pin AirPods + from their degraded output profile, and a `bringingUpCapture` flag to stop a re-warm racing a live + press) was buying ~15 ms of a ~600 ms bring-up. It was deleted in favour of a stateless + `warmUp()`: build one session at launch and drop it, which absorbs the ~185 ms a process pays the + _first_ time it touches AVFoundation's capture stack (~90–125 ms of that the first device query + alone) and holds nothing afterwards. The live + suite pins the load-bearing half — building, and warming, must leave the microphone closed. `stop()` also waits out `AudioTransport.tailLinger(forTransportType:)` (220 ms on Bluetooth, `.zero` otherwise — the policy sits beside `MicLiveness`'s wait cap so both are unit-tested) before ending the recording **when the -session's input is Bluetooth**, so speech still travelling over the link lands in the file instead of -being truncated — the missing last word. It runs after `.transcribing` is claimed, so it delays the -transcript, never the "it heard me" cue. Cancels take `cancelCapture()` instead, which skips both the -linger and the file read-back: the audio is being discarded, so neither is worth delaying the user's -cancel for. - -The routing facts behind all of that live in **`AudioRoute`** (`Audio/AudioRoute.swift`, internal): -which device is the default input, and its raw CoreAudio transport type. **Raw reads only** — what a -transport _means_ is **`AudioTransport.isBluetooth`** and **`MicLiveness.timeout`**, which are pure -and unit-tested, because `AudioRoute` itself needs real hardware and is excluded from the coverage -gate. Don't let a decision drift into it. Its sibling +session's input is Bluetooth**, so speech still travelling over the link lands in the recording +instead of being truncated — the missing last word. It runs after `.transcribing` is claimed, so it +delays the transcript, never the "it heard me" cue. Cancels take `cancelCapture()` instead, which +skips the linger: the audio is being discarded, so it isn't worth delaying the user's cancel for. + +What a transport _means_ is **`AudioTransport.isBluetooth`** and **`MicLiveness.timeout`**, which +are pure and unit-tested; the reads that feed them are hardware-bound and coverage-excluded, so don't +let a decision drift into them. **`AudioRoute`** (`Audio/AudioRoute.swift`, internal) is what's left +of the CoreAudio side after the input reads moved to `AVCaptureDevice`: the default _output_ device, +plus the property addressing its sibling shares. Its sibling **`AudioRouteMonitor`** (public) publishes output-route changes for the cue players — see [Settings, persistence, and cues](#settings-persistence-and-cues). Both are excluded from the coverage gate for the same reason @@ -837,6 +879,9 @@ Engine-side stores, all `UserDefaults`-backed value types with the same shape: Only the active profile is ever sent, so that budget is not divided between them. One of the three stores with a setter, for the encoded-on-write reason below: the settings sheet and the main window's switcher write through it rather than binding the raw slots), + **`MicDeviceStore`** (`BlurtMicDeviceUID`, the input device dictation is pinned to, as its + CoreAudio UID — empty/unset follows the system default; re-read at every press, and a pinned + device that isn't connected falls back to the system default without unpinning), **`OverlayOriginStore`** (the pill's dragged origin, x/y), **`LastUpdateCheckStore`** (`BlurtLastUpdateCheck`, the stamp throttling the automatic launch update check). - **`DefaultsKey`** (`Config/DefaultsKey.swift`) defines every key those stores write, one case each, diff --git a/App/Blurt/Blurt.xcodeproj/project.pbxproj b/App/Blurt/Blurt.xcodeproj/project.pbxproj index b9126c7a..267996bb 100644 --- a/App/Blurt/Blurt.xcodeproj/project.pbxproj +++ b/App/Blurt/Blurt.xcodeproj/project.pbxproj @@ -409,6 +409,7 @@ EA50EC15CDECF534464D9208 /* BlurtUITestSupport.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3A0683C1BE5AD05E4C724956 /* BlurtUITestSupport.swift */; }; EA9FBD50906FBE8ECC2DCB43 /* rom1a-2-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = DDD514790151F484A68127BB /* rom1a-2-stop.m4a */; }; EB1E111038C87156F2CF11B4 /* rom1b-24-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 87570A5DC1428AC2DE812BBE /* rom1b-24-stop.m4a */; }; + EC0B15FB6EAAF91396A8BF50 /* MicrophoneStepView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4BE9BA7182953D06AAD913AE /* MicrophoneStepView.swift */; }; EC3DAE9036AA1D57607157F8 /* juno-111-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 0088DE3FD3980CA727EB02FE /* juno-111-start.m4a */; }; EC49246782EDC2BE84223723 /* juno-2-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 958C5FDCD430D87BD61D048C /* juno-2-start.m4a */; }; EC6A4B9B4DB3A8DAF2388004 /* juno-71-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 156C3E8FF4B96A45913A97F0 /* juno-71-start.m4a */; }; @@ -600,6 +601,7 @@ 4AE2B597126C95C118FF9FDD /* rom1a-29-start.m4a */ = {isa = PBXFileReference; path = "rom1a-29-start.m4a"; sourceTree = ""; }; 4B9209100F5A6E8514AF6FA7 /* rom1a-9-stop.m4a */ = {isa = PBXFileReference; path = "rom1a-9-stop.m4a"; sourceTree = ""; }; 4B9A2A460519088D70893121 /* rom1a-25-start.m4a */ = {isa = PBXFileReference; path = "rom1a-25-start.m4a"; sourceTree = ""; }; + 4BE9BA7182953D06AAD913AE /* MicrophoneStepView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MicrophoneStepView.swift; sourceTree = ""; }; 4DB6065E9D1F64F3AF0FD03B /* juno-5-start.m4a */ = {isa = PBXFileReference; path = "juno-5-start.m4a"; sourceTree = ""; }; 4E25792AC8E46820FF270D64 /* rom1a-2-start.m4a */ = {isa = PBXFileReference; path = "rom1a-2-start.m4a"; sourceTree = ""; }; 4E414413BCDA4C613782919D /* juno-66-stop.m4a */ = {isa = PBXFileReference; path = "juno-66-stop.m4a"; sourceTree = ""; }; @@ -1450,6 +1452,7 @@ 3DA3A304528FD1C7FE6C2B94 /* APIKeyStepView.swift */, 50FFCDBD1FB2CF0A81A68D18 /* HotkeyStepView.swift */, C59092F9220BBE7465068325 /* KeyTermsStepView.swift */, + 4BE9BA7182953D06AAD913AE /* MicrophoneStepView.swift */, E8DE60178ABA2CC7F4EB6401 /* PermissionsStepView.swift */, 27E18FF6880EADB94F98C7C4 /* SoundStepView.swift */, ); @@ -2006,6 +2009,7 @@ E8AE03AA4DCD807D2BB8D660 /* LiquidGlass.swift in Sources */, 7F838018EA0ACE7170FC32C4 /* MainWindowRoot.swift in Sources */, AD32046355138298E7FC40D0 /* MenuBarScene.swift in Sources */, + EC0B15FB6EAAF91396A8BF50 /* MicrophoneStepView.swift in Sources */, 74BD28AA54484635D62FA797 /* OverlayPillContent.swift in Sources */, 9437C292A6D7FE533338073D /* OverlayView.swift in Sources */, 8FE4B9B139BB9557FBC5589B /* OverlayWindowController.swift in Sources */, diff --git a/App/Blurt/Blurt/AppCoordinator.swift b/App/Blurt/Blurt/AppCoordinator.swift index aadb6775..4d1da333 100644 --- a/App/Blurt/Blurt/AppCoordinator.swift +++ b/App/Blurt/Blurt/AppCoordinator.swift @@ -101,11 +101,13 @@ final class AppCoordinator { } func start() { - // Pre-warm the mic so the first dictation doesn't pay hardware-route - // discovery on the hot path — but only once microphone access is granted, so - // warming up never triggers the permission prompt at launch. Before the grant - // the user opts in via the setup screen's "Allow Microphone Access" button; - // the first dictation after that just prepares a recorder lazily. + // Absorb the one-off cost of this process's first touch of the capture + // stack, so the first dictation doesn't pay it on the hot path (~75 ms; see + // `MicCapture.warmUp()`, which holds no recorder and opens no device). Gated + // on the grant because building a capture input is what raises the + // microphone prompt, and launch is the wrong moment for it: before the grant + // the user opts in via the setup screen's "Allow Microphone Access" button, + // and the first dictation after that simply pays the build itself. if PermissionsChecker.check().microphone { let mic = mic Task { await mic.warmUp() } diff --git a/App/Blurt/Blurt/CueSoundPlayer.swift b/App/Blurt/Blurt/CueSoundPlayer.swift index 1e6b0e7b..f412da66 100644 --- a/App/Blurt/Blurt/CueSoundPlayer.swift +++ b/App/Blurt/Blurt/CueSoundPlayer.swift @@ -152,12 +152,16 @@ final class CueSoundPlayer { // A pending route re-prime is acted on at exactly two points. // // `.connecting` covers the re-prime that is *already pending when a press - // arrives*, which is the one the user hears: `MicCapture.warmUp()` opens the - // input at launch, flipping AirPods out of their output-only profile well - // before the first dictation, so the players `prime()` pre-rolled are stale - // and the very first start chime is the one that stalls or clips. Waiting for - // a terminal phase meant that reload landed *after* the dictation it should - // have protected. Reloading here is safe where reloading mid-press was not: + // arrives*, which is the one the user hears: opening the mic flips AirPods + // out of their output-only profile, which invalidates the players `prime()` + // pre-rolled, and the first start chime after such a flip is the one that + // stalls or clips. That flip lands at the *press* — `record()`'s + // `startRunning()` is what opens the device, and nothing before it does (see + // `MicCapture.warmUp()`, which deliberately holds no open input) — so a + // pending re-prime and the chime it has to protect are one phase apart. + // Waiting for a terminal phase meant that reload landed *after* the + // dictation it should have protected. Reloading here is safe where reloading + // mid-press was not: // no cue is in flight in this phase (the start chime rides the // connecting→recording edge), so nothing is swapped out from under a playing // `AVAudioPlayer`; the decode runs off the main actor, so it isn't competing diff --git a/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift b/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift index 9e403154..b34f7a27 100644 --- a/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift +++ b/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift @@ -65,7 +65,7 @@ private struct SettingsPane: View { } /// The everyday setup a user changes: the AssemblyAI key, the dictation -/// shortcut, the cue sound, and the transcription key terms. +/// shortcut, the microphone, the cue sound, and the transcription key terms. private struct GeneralSettingsTab: View { let coordinator: AppCoordinator @@ -73,6 +73,7 @@ private struct GeneralSettingsTab: View { SettingsPane { APIKeyStepView(apiKey: coordinator.apiKey) HotkeyStepView(coordinator: coordinator) + MicrophoneStepView() SoundStepView(coordinator: coordinator) KeyTermsStepView() } diff --git a/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift b/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift new file mode 100644 index 00000000..7b9fa283 --- /dev/null +++ b/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift @@ -0,0 +1,94 @@ +import BlurtEngine +import SwiftUI + +/// The microphone section of the settings screen: a menu picker that either +/// follows the system's default input (the default) or pins dictation to one +/// specific input device, persisted as the device's UID via `MicDeviceStore`. +/// `MicCapture` re-reads the selection at every press, so a change applies to +/// the next dictation with nothing to push — and a pinned device that isn't +/// connected gracefully falls back to the system default +/// (`MicDeviceSelection.effective`), so no choice here can break dictation. +struct MicrophoneStepView: View { + // Empty means "no device pinned" — the unset default belongs to + // `MicDeviceSelection.fromPersisted`, which reads it as "same as system". See + // `HotkeyStepView` for why the view must not restate it. + @AppStorage(MicDeviceStore.defaultsKey) private var micDeviceUID = "" + + /// The input devices present when the pane appeared, re-read each time it + /// does. A snapshot rather than a live listener: the picker's menu is built + /// when it opens, and a device plugged in mid-session shows up on the next + /// visit — the capture path resolves the UID fresh at every press regardless. + /// + @State private var devices: [AudioInputDevice] = [] + /// Whether `devices` has been read yet, as distinct from "read, and empty". + /// Without the distinction `missingPin` took the empty initial value as "the + /// pinned device is gone", so the picker showed "Disconnected microphone" for + /// the 150–500 ms of a cold read — on a mic plugged in the whole time. A flag + /// rather than an optional array because SwiftLint's + /// `discouraged_optional_collection` (opted into repo-wide) forbids the latter. + @State private var devicesLoaded = false + /// The current default input's name, for the "Same as system (…)" label; nil + /// (no device, or the read failed) drops the parenthetical. + @State private var systemDefaultName: String? + + /// The selection as the engine's own type, not the raw slot. Binding the + /// `String` meant spelling ""-means-system a second time (in a `.tag("")`), + /// which is exactly what `MicDeviceSelection` exists to own — the same shape + /// `SoundStepView` and `HotkeyStepView` already use for their pickers. + private var selection: Binding { + Binding( + get: { MicDeviceSelection.fromPersisted(micDeviceUID) }, + // Write through the store (see `HotkeyStepView` for why): the store owns + // the encoding, `@AppStorage` observes the key to re-render. + set: { MicDeviceStore().selection = $0 }) + } + + /// The stored pin when no connected device carries it — kept selectable so + /// the picker can render the persisted choice instead of a blank control, and + /// the user sees why dictation is currently using the system default. + private var missingPin: MicDeviceSelection? { + guard devicesLoaded, !micDeviceUID.isEmpty, + !devices.contains(where: { $0.uid == micDeviceUID }) + else { return nil } + return .pinned(uid: micDeviceUID) + } + + private var systemDefaultLabel: String { + systemDefaultName.map { "Same as system (\($0))" } ?? "Same as system" + } + + var body: some View { + Section { + PickerSettingRow( + title: "Input device", systemImage: "mic", + accessibilityID: UITestIdentifiers.micPicker, selection: selection + ) { + Text(systemDefaultLabel).tag(MicDeviceSelection.systemDefault) + ForEach(devices) { device in + Text(device.name).tag(MicDeviceSelection.pinned(uid: device.uid)) + } + if let missingPin { + Text("Disconnected microphone").tag(missingPin) + } + } + } header: { + Text("Microphone") + } footer: { + Text("Dictation records from this microphone. While it isn't connected, the system default is used.") + } + // Off the main actor, and `.task` rather than `.onAppear` to have somewhere + // to await: enumerating devices is the first thing to touch AVFoundation's + // capture stack in a process that hasn't dictated yet, measured at 150–500 ms + // cold (~0.02 ms once warm). On a first-run install — where `AppCoordinator` + // skips its launch warm-up because microphone access hasn't been granted — + // that ran inline while this window was being laid out. + .task { + let snapshot = await Task.detached { + (devices: AudioInputDevices.all(), defaultName: AudioInputDevices.systemDefaultInputName()) + }.value + devices = snapshot.devices + systemDefaultName = snapshot.defaultName + devicesLoaded = true + } + } +} diff --git a/App/Blurt/BlurtUITests/SettingsUITests.swift b/App/Blurt/BlurtUITests/SettingsUITests.swift index 61e68409..87540254 100644 --- a/App/Blurt/BlurtUITests/SettingsUITests.swift +++ b/App/Blurt/BlurtUITests/SettingsUITests.swift @@ -75,6 +75,22 @@ final class SettingsUITests: BlurtUITestCase { XCTAssertEqual(picker.value as? String, "right ⌥") } + /// The microphone picker exists and defaults to following the system input. + /// Only the first option is asserted: the rest of the menu lists whatever + /// input devices the machine happens to have (a CI runner may have none), and + /// the parenthetical default-device name varies with it — so the assertion is + /// a prefix, and nothing here opens the menu or pins a device. + func testMicrophonePickerDefaultsToSystemInput() { + let settings = openSettingsWindow() + + let picker = settings.popUpButtons[UITestIdentifiers.micPicker] + XCTAssertTrue(picker.waitForExistence(timeout: 10), "Microphone picker not found") + let value = picker.value as? String ?? "" + XCTAssertTrue( + value.hasPrefix("Same as system"), + "The picker should start on the system default (got: \(value))") + } + /// The sound-cue picker changes the persisted selection. Selecting "None" /// works regardless of the runner's persisted starting value. func testSoundPickerChangesSelection() { diff --git a/App/Blurt/Shared/UITestIdentifiers.swift b/App/Blurt/Shared/UITestIdentifiers.swift index f16cab73..93090502 100644 --- a/App/Blurt/Shared/UITestIdentifiers.swift +++ b/App/Blurt/Shared/UITestIdentifiers.swift @@ -61,6 +61,7 @@ enum UITestIdentifiers { static let apiKeyError = "settings.apiKey.error" static let keyTermsField = "settings.keyTerms.field" static let hotkeyPicker = "settings.hotkey.picker" + static let micPicker = "settings.mic.picker" static let soundPicker = "settings.sound.picker" static let developerToggle = "settings.developer.toggle" static let enhancedTranscriptsToggle = "settings.enhancedTranscripts.toggle" diff --git a/README.md b/README.md index 97428882..09e04382 100644 --- a/README.md +++ b/README.md @@ -185,7 +185,7 @@ what each script does, signing — and how changes land; ```text Sources/BlurtEngine/ Swift 6 package owning the pipeline — no external dependencies - Audio/ MicCapture: fresh AVAudioRecorder per session, 16 kHz mono PCM, + Audio/ MicCapture: fresh AVCaptureSession per session, 16 kHz mono PCM, live level meter; DX7/Juno-106 sound packs STT/ AssemblyAITranscriber: one POST to dictation.assemblyai.com/transcribe (STT + LLM rewrite); ConversationContext (contextual priming: diff --git a/Sources/BlurtEngine/Audio/AudioInputDevices.swift b/Sources/BlurtEngine/Audio/AudioInputDevices.swift new file mode 100644 index 00000000..7d681ceb --- /dev/null +++ b/Sources/BlurtEngine/Audio/AudioInputDevices.swift @@ -0,0 +1,108 @@ +@preconcurrency import AVFoundation +import Foundation + +/// One selectable input device: its persistent UID (what `MicDeviceStore` pins) +/// and its user-facing name (what the Settings picker shows). +public struct AudioInputDevice: Identifiable, Sendable { + public let uid: String + public let name: String + public var id: String { uid } +} + +/// Every question the capture path and the Settings picker ask about input +/// devices: what's available, what it's called, whether a pinned UID still names +/// something connected, and what transport it's on. +/// +/// All of it comes from **`AVCaptureDevice`**, which is the same API the recorder +/// opens the device with. `uniqueID` *is* the CoreAudio device UID string on +/// macOS, so the picker lists exactly the devices `CaptureSessionRecorder` can +/// pin to, and "the pin resolves" and "the recorder can open it" are one fact +/// rather than two that could disagree. +/// +/// `transportType` is the same four-character code CoreAudio's +/// `kAudioDevicePropertyTransportType` reports — confirmed against real hardware +/// for every case the policies care about: `blue` on AirPods, `bltn` on the +/// built-in mic, `usb ` on USB interfaces, `virt` on virtual devices and `grup` +/// on an aggregate. That confirmation is what retired the parallel CoreAudio +/// read this file used to keep (a device-list read, an input-stream filter, a +/// `CFString` property bridge and a UID→`AudioDeviceID` translation, ~60 lines). +/// The Bluetooth case was the one worth being slow about: a transport that +/// failed to read as Bluetooth silently costs the 2.5 s liveness cap and the +/// 220 ms tail linger, which is the missing-last-word bug both exist to fix. +/// +/// Raw reads only — the policy for a UID that no longer resolves is +/// `MicDeviceSelection.effective`, and what a transport *means* is +/// `AudioTransport`; both are pure and unit-tested. This file needs real +/// hardware to answer anything, so it is excluded from the coverage gate and +/// must not be where a decision hides. +public enum AudioInputDevices { + /// Every microphone the system offers, sorted by name for a stable picker + /// order. Empty when there are none. + /// + /// A discovery session rather than the deprecated `devices(for:)`, and no + /// input-stream filter: `mediaType: .audio` already means "can capture audio", + /// which is the filter the retired HAL path had to reconstruct by asking each + /// device for the size of its input-scope stream list. + public static func all() -> [AudioInputDevice] { + AVCaptureDevice.DiscoverySession( + deviceTypes: [.microphone], mediaType: .audio, position: .unspecified + ) + .devices + .map { AudioInputDevice(uid: $0.uniqueID, name: $0.localizedName) } + .sorted { $0.name.localizedCaseInsensitiveCompare($1.name) == .orderedAscending } + } + + /// The current default input device's name — what the picker's "Same as + /// system (…)" option shows — or nil when there is no input device (the picker + /// then says "Same as system" with no parenthetical). + public static func systemDefaultInputName() -> String? { + systemDefaultDevice?.localizedName + } + + /// The transport type of the device carrying this UID, or nil when no + /// connected device does. + /// + /// That nil does double duty, and deliberately: it is also the + /// missing-device signal `MicDeviceSelection.effective` falls back on. The two + /// used to be separate calls — an `isConnected(uid:)` and this — which meant + /// two lookups of the same device per press for one question, and two chances + /// to answer it differently. `transportType(of:)` cannot fail for a device + /// that exists, so "no transport" and "no device" are the same fact. + static func transportType(forUID uid: String) -> UInt32? { + transportType(of: device(forUID: uid)) + } + + /// The transport type of the system default input, or nil when there is no + /// input device at all. Nil is the conservative answer at both consumers: + /// `AudioTransport` reads it as not-Bluetooth, which means the middle liveness + /// cap and no tail linger. + static func systemDefaultTransportType() -> UInt32? { + transportType(of: systemDefaultDevice) + } + + /// The device carrying `uid`, or nil when none does. `isConnected` is checked + /// as well as the lookup succeeding: a device that has just gone away can + /// still be handed back, and every caller means "usable right now". + /// + /// Internal, not private, because `CaptureSessionRecorder` resolves the same + /// pin a moment later to attach it — one spelling of "resolve a pin to a + /// device", so the two layers can't apply different presence rules. + static func device(forUID uid: String) -> AVCaptureDevice? { + guard let device = AVCaptureDevice(uniqueID: uid), device.isConnected else { return nil } + return device + } + + /// The system default input device. One spelling, shared with the recorder's + /// fallback and the picker's "Same as system (…)" label. + static var systemDefaultDevice: AVCaptureDevice? { + AVCaptureDevice.default(for: .audio) + } + + /// `AVCaptureDevice.transportType` as the `UInt32` four-character code + /// `AudioTransport` matches against CoreAudio's `kAudioDeviceTransportType*` + /// constants. Bit-pattern converted rather than numerically: these are packed + /// ASCII, and the sign of the `Int32` spelling is an accident of the API. + private static func transportType(of device: AVCaptureDevice?) -> UInt32? { + device.map { UInt32(bitPattern: $0.transportType) } + } +} diff --git a/Sources/BlurtEngine/Audio/AudioRoute.swift b/Sources/BlurtEngine/Audio/AudioRoute.swift index bf7e8e98..142315ff 100644 --- a/Sources/BlurtEngine/Audio/AudioRoute.swift +++ b/Sources/BlurtEngine/Audio/AudioRoute.swift @@ -1,66 +1,28 @@ import CoreAudio -/// Read-only queries against the system's current audio routing — the two facts -/// about the mic that `AVFoundation` doesn't expose but the capture path needs: +/// The CoreAudio reads that are left: which device is the default *output*, and +/// the property addressing `AudioRouteMonitor` shares with it. /// -/// 1. **Which device is the default input**, so `MicCapture` can tell whether a -/// recorder it prepared earlier is still bound to the device the user is -/// about to speak into. `AVAudioRecorder` resolves the route at -/// `prepareToRecord()` time and never re-resolves it, so a recorder warmed -/// before the user connected their AirPods would silently record from the -/// built-in mic. -/// 2. **What transport that device is on**, since a Bluetooth link is both slow -/// to bring up (the wait `MicLiveness` caps) and buffered at the tail (the -/// linger `MicCapture.stop()` grants rather than truncating). +/// The input side used to live here too — which device is the default input, and +/// what transport it is on — and both have moved to `AVCaptureDevice` +/// (`AudioInputDevices`), which answers them off the same object the recorder +/// opens. That move waited on confirming `AVCaptureDevice.transportType` reports +/// `blue` for AirPods, because a transport that failed to read as Bluetooth +/// silently costs the 2.5 s liveness cap and the 220 ms tail linger — the +/// missing-last-word bug both exist to fix. It does; the parallel HAL read went. /// -/// Raw reads only — no policy. What a transport type *means* lives in -/// `AudioTransport` and `MicLiveness`, which are pure and unit-tested; this file -/// needs real hardware to answer anything, so it is excluded from the coverage -/// gate and must not be where a decision hides. +/// Output has no such replacement: `AVCaptureDevice` describes *capture* +/// devices, and what `AudioRouteMonitor` watches is the output route the cue +/// players render into. +/// +/// Raw reads only — no policy. Needs real hardware to answer anything, so it is +/// excluded from the coverage gate and must not be where a decision hides. /// /// Internal, not public: the app never asks these directly (it observes route /// *changes* through `AudioRouteMonitor`), and `.periphery.yml` runs with /// `retain_public: false`, so a `public` symbol only the engine reaches fails /// the unused-code scan. enum AudioRoute { - /// Identity plus link character of the default input device, read together in - /// one pass so the capture path makes a single trip through CoreAudio per - /// session rather than one per question. - struct InputSnapshot: Equatable, Sendable { - /// Which device this is. - /// - /// The `AudioDeviceID` rather than the device's persistent UID string, even - /// though IDs are in principle reusable across an unplug/replug while UIDs - /// are not. The only consumer is "is the warm recorder still bound to the - /// device about to be recorded from", and the two disagree in exactly one - /// case: the warmed device was removed and a new one took its ID inside the - /// 60 s warm window. That case fails *loudly* — the recorder is bound to a - /// device that no longer exists, so `record()` returns false and the press - /// surfaces `.audioCaptureFailed`. It cannot produce the failure the check - /// exists to prevent, which is silently recording the wrong mic. Reading the - /// UID instead would mean bridging a `CFString`, i.e. pulling Foundation - /// into a file that otherwise needs only CoreAudio, to buy a distinction - /// that changes a loud failure into a slightly louder one. - let deviceID: AudioDeviceID - /// The device's CoreAudio transport type, or nil when the read failed. - /// Interpreted by `AudioTransport.isBluetooth` and - /// `MicLiveness.timeout(forTransportType:)` — kept raw here so the policy - /// stays in the files `swift test` can reach. - let transportType: UInt32? - } - - /// The default input device as an `InputSnapshot`, or nil when there is no - /// input device (all of them unplugged or asleep) or CoreAudio refused the - /// read. Nil is the conservative answer everywhere it's consumed: an unknown - /// input invalidates a warm recorder rather than silently keeping one bound to - /// a device that may have gone away. - static func currentInput() -> InputSnapshot? { - guard let deviceID = defaultDeviceID(for: kAudioHardwarePropertyDefaultInputDevice) else { - return nil - } - return InputSnapshot(deviceID: deviceID, transportType: transportType(of: deviceID)) - } - /// The system's current default *output* device — what `AudioRouteMonitor` /// hangs its format listener on. Nil when there is none, or the read failed. static func defaultOutputDeviceID() -> AudioDeviceID? { @@ -85,17 +47,17 @@ enum AudioRoute { // MARK: - CoreAudio reads - // Spelled out per property rather than shared behind a generic - // `read(_:from:initial:)`. That reads better but doesn't compile: `&value` - // on an unconstrained `T` is "forming 'UnsafeMutableRawPointer' to a variable - // of type 'T'; this is likely incorrect because 'T' may contain an object - // reference". Making it work means constraining to `BitwiseCopyable` and going - // through `withUnsafeMutableBytes` — more machinery than two five-line reads - // are worth, in a file the coverage gate can't check anyway. - /// The device the system object reports for `selector` (a default-device /// property). Nil covers both a failed read and the "no such device" sentinel, /// which callers treat identically. + /// + /// Spelled out rather than shared behind a generic `read(_:from:initial:)`. + /// That reads better but doesn't compile: `&value` on an unconstrained `T` is + /// "forming 'UnsafeMutableRawPointer' to a variable of type 'T'; this is likely + /// incorrect because 'T' may contain an object reference". Making it work means + /// constraining to `BitwiseCopyable` and going through `withUnsafeMutableBytes` + /// — more machinery than one five-line read is worth, in a file the coverage + /// gate can't check anyway. private static func defaultDeviceID(for selector: AudioObjectPropertySelector) -> AudioDeviceID? { var address = globalAddress(selector) var deviceID = AudioDeviceID(0) @@ -106,16 +68,4 @@ enum AudioRoute { guard status == noErr, deviceID != 0 else { return nil } return deviceID } - - /// The device's transport type, or nil when the read failed — - /// `AudioTransport` and `MicLiveness` both treat nil as "not Bluetooth", which - /// is the conservative direction for each. - private static func transportType(of deviceID: AudioDeviceID) -> UInt32? { - var address = globalAddress(kAudioDevicePropertyTransportType) - var transport = UInt32(0) - var size = UInt32(MemoryLayout.size) - let status = AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, &transport) - guard status == noErr else { return nil } - return transport - } } diff --git a/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift b/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift new file mode 100644 index 00000000..dbbf9e3e --- /dev/null +++ b/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift @@ -0,0 +1,312 @@ +@preconcurrency import AVFoundation +import CoreMedia +import Dispatch +import Foundation +import Synchronization + +/// The capture backend (owner-directed move to `AVCaptureSession`, 2026-08-25): +/// a session built fresh per capture around a single audio device — the device +/// pinned in Settings when one is set, the system default otherwise — whose data +/// output converts to the dictation API's 16 kHz mono 16-bit LPCM on the fly and +/// delivers it here as sample buffers, accumulated in memory as the raw S16LE +/// blob `stopAndReadPCM` returns. No temp file, no resample pass, no decode on +/// the release path. +/// +/// Used directly by `MicCapture` rather than through a protocol. There *was* a +/// `CaptureRecorder` seam, and its whole job was to let the actor's machinery — +/// liveness gate, meter, tail linger — be written once across two backends (an +/// `AVAudioRecorder` WAV path and a device-pinned `AudioQueue`). Both are gone, +/// so the seam abstracted a single conformer with no test double behind it; the +/// contract it documented now lives on these members. `MicCaptureProtocol` is +/// the seam hosts and tests actually inject at. +/// +/// Fresh-per-session is load-bearing: the session's input is attached to one +/// device at build time and never re-resolves, so a session outliving a device +/// switch would record the wrong mic (or silence) rather than failing loudly. +/// `MicCapture` builds one per press and drops it at stop. +/// +/// Building the session does **not** engage the microphone: measured against +/// CoreAudio's own `kAudioDevicePropertyDeviceIsRunningSomewhere`, the bit +/// behind the input indicator, it stays clear until `record()`. Building is also +/// cheap next to opening. `MicCapture.warmUp()` holds the full measurement and +/// what follows from it (no warm-recorder lifecycle, and nothing worth +/// pre-paying but the process's first touch). +/// +/// Hardware-bound like `MicCapture`, and excluded from the coverage gate for +/// the same reason; its live test rides the `BLURT_LIVE_AUDIO_TESTS` gate in +/// `AudioInputDevicesTests`. +/// +/// `@unchecked Sendable`: `MicCapture.start()`'s liveness probes read +/// `deliveredFrames` and `meteredPowerDB()` off-actor, safe by confinement — +/// the polls run sequentially in one task while `start()` is suspended and +/// nothing else references the recorder in that window. The session/output +/// handles are configured in `init` and then only read, and everything the +/// delegate queue also touches lives behind the `Mutex`. +final class CaptureSessionRecorder: NSObject, @unchecked Sendable { + private struct Guarded { + /// Every byte the delegate has delivered, in arrival order — already the + /// raw S16LE blob `stopAndReadPCM` returns. + var captured = Data() + /// Frames delivered so far, summed off each sample buffer's own count. + /// Frames of digital silence count exactly like real audio; the liveness + /// gate's power term is what tells those apart. + var frameCount = 0 + } + + private let session = AVCaptureSession() + private let output = AVCaptureAudioDataOutput() + /// The serial queue sample buffers are delivered on; only the delegate + /// method runs here, and it touches nothing but `state`. + private let delegateQueue = DispatchQueue( + label: HostIdentity.current.queueLabel("MicCaptureSession")) + /// The serial queue session *control* runs on — building and starting — + /// deliberately not `delegateQueue`. Both steps block for as long as the + /// hardware takes, and blocking the delivery queue would stall the very frames + /// the liveness gate is then waiting for. A `DispatchQueue` rather than a + /// detached `Task`, so those waits park a Dispatch thread instead of one of the + /// cooperative pool's — the pool is what the rest of the press runs on (the + /// context capture that overlaps this, the transcriber's connection warm-up), + /// and starving it is how covering the retired backend in CI deadlocked the + /// whole test run. + /// + /// `static`, so it also serializes across recorders: only one capture session + /// is ever meant to be coming up at a time, and AVFoundation asks that session + /// control be serialized rather than concurrent. + private static let controlQueue = DispatchQueue( + label: HostIdentity.current.queueLabel("MicCaptureSessionControl")) + private let state = Mutex(Guarded()) + + /// Builds a recorder off the caller's executor, on `controlQueue`. + /// + /// The build is not free and not bounded: ~5 ms warm, but ~185 ms the first + /// time a process touches AVFoundation's capture stack (~90–125 ms of that the + /// first device query alone). Run inline on `MicCapture` it blocked the actor + /// for that whole window, which a teardown racing the bring-up then waited out + /// — the same defect as a blocking `record()`, and `MicCaptureBringUpTests` + /// caught it at 100 ms once its own warm-up probe was removed. Both hops run + /// on `controlQueue`. + /// + /// Deliberately still *separate* from `record()` rather than folded into one + /// "make and start" hop: that building leaves the microphone closed is the + /// invariant the whole warm-up design rests on, and the live suite can only + /// assert it while the two steps are observable apart. + static func make(pinnedUID: String?) async throws -> CaptureSessionRecorder { + try await withCheckedThrowingContinuation { continuation in + controlQueue.async { + continuation.resume(with: Result { try CaptureSessionRecorder(pinnedUID: pinnedUID) }) + } + } + } + + /// Builds and fully configures the session — device input, converted-format + /// data output, delegate — without starting it. Private: `make` is the entry + /// point, so no caller can put this back on its own executor. Throws only when + /// `AVCaptureDeviceInput` refuses the device (e.g. no microphone + /// authorization); "no device at all" instead leaves the session inputless, + /// so `record()` answers false and `MicCapture.start()` surfaces the same + /// `.audioCaptureFailed(noInputDevice)` it always has. + private init(pinnedUID: String?) throws { + super.init() + + session.beginConfiguration() + defer { session.commitConfiguration() } + if let device = Self.device(forPinnedUID: pinnedUID) { + let input = try AVCaptureDeviceInput(device: device) + if session.canAddInput(input) { + session.addInput(input) + } else { + // Leaves the session inputless, so `record()` answers false and the press + // surfaces `.audioCaptureFailed(noInputDevice)` — correct, but silent + // without this: "no usable input device" for a device that exists and was + // refused is the one case that log line can't explain on its own. + MicCapture.logger.error( + "session refused the input device \(device.localizedName, privacy: .public)") + } + } + output.audioSettings = Self.audioSettings() + output.setSampleBufferDelegate(self, queue: delegateQueue) + if session.canAddOutput(output) { + session.addOutput(output) + } else { + // Without an output nothing is ever delivered, so the liveness gate would + // fail closed at its cap with no clue why. + MicCapture.logger.error("session refused the audio data output") + } + } + + deinit { + // Backstop for a recorder dropped without a stop, so an orphaned instance + // can't keep the microphone engaged for the rest of the process. + if session.isRunning { + session.stopRunning() + } + } + + /// The device the session records from: the pinned device when its UID still + /// resolves (`AVCaptureDevice.uniqueID` is the CoreAudio device UID on + /// macOS, the same string `MicDeviceStore` persists), else the default input + /// — the same per-capture fallback `MicCapture.resolveInput` applies, kept + /// here too so the race where the device vanishes between resolution and + /// build degrades identically. Nil when the machine has no input at all. + private static func device(forPinnedUID pinnedUID: String?) -> AVCaptureDevice? { + if let pinnedUID, let pinned = AudioInputDevices.device(forUID: pinnedUID) { + return pinned + } + return AudioInputDevices.systemDefaultDevice + } + + /// The dictation API's geometry, converted by the output itself — the same six + /// keys the retired WAV recorder asked of its file, so capture still lands in + /// upload-ready S16LE with no resample pass anywhere. + /// + /// Every number comes from `SyncSTTLimits`, which also owns the byte math the + /// upload side applies to the result. They are one contract: a stereo or + /// 8-bit recorder would silently halve or double every duration the pipeline + /// computes. Device-free, so `MicCaptureFormatTests` can assert it despite + /// this file being coverage-excluded; a function rather than a stored static + /// because `[String: Any]` is not `Sendable`, and it is built once per + /// recorder regardless. + static func audioSettings() -> [String: Any] { + [ + AVFormatIDKey: kAudioFormatLinearPCM, + AVSampleRateKey: Double(SyncSTTLimits.sampleRate), + AVNumberOfChannelsKey: SyncSTTLimits.channelCount, + AVLinearPCMBitDepthKey: SyncSTTLimits.bitDepth, + AVLinearPCMIsFloatKey: false, + AVLinearPCMIsBigEndianKey: false, + ] + } + + /// Opens the device and starts capture, answering false when there is no + /// usable input: nothing was attached at build time, or the session refused to + /// run. + /// + /// This is where the route-activation cost lives, all of it — ~180 ms on the + /// built-in mic, ~600 ms on a USB interface, and on AirPods ~80 ms before + /// `startRunning()` returns plus ~400 ms before the first frame. It is `async` + /// and hops to `controlQueue` precisely because that cost is unbounded from the + /// caller's point of view: run inline it blocked `MicCapture` for the whole + /// open, and a teardown arriving in that window measured 578 ms of waiting on a + /// 635 ms open (`MicCaptureBringUpTests`). Suspending instead of blocking means + /// the actor stays available to the stop and cancel that may be racing this. + /// + /// The suspension is why `MicCapture.start()` snapshots `stopGeneration` + /// before the bring-up and re-checks after: a teardown can land during the + /// open, and it has to win. + func record() async -> Bool { + guard !session.inputs.isEmpty else { return false } + return await withCheckedContinuation { continuation in + Self.controlQueue.async { + self.session.startRunning() + continuation.resume(returning: self.session.isRunning) + } + } + } + + /// Frames the delegate has actually received. 0 until the first buffer lands, + /// which is the has-anything-arrived probe `MicLiveness` short-circuits on. + /// + /// A raw frame count rather than the seconds-since-`record()` the retired + /// `AVAudioRecorder` reported: the only consumer asks whether it has moved off + /// zero, so dividing by the sample rate manufactured a duration nothing read + /// as one. + var deliveredFrames: Int { + state.withLock { $0.frameCount } + } + + /// The loudest of the capture connection's channels, in dBFS + /// (`AVCaptureAudioChannel.averagePowerLevel`), feeding both the liveness + /// gate's silence-floor probe and the overlay meter. A missing connection or + /// channel answers -160 — reads as digital silence, so the gate keeps + /// waiting (and fails closed at its cap) and the meter rests, the + /// conservative direction for both. + /// + /// The **loudest**, not the first, and this is load-bearing: + /// `connection.audioChannels` describes the *device's* channels, not the mono + /// the data output converts to. Measured on this machine — a stereo interface, + /// an aggregate and two virtual devices all report `audioChannels=2` against + /// `outputChannels=1`, and a channel carrying nothing reads -758 dBFS. So + /// metering channel 0 alone reported silence for any device whose microphone + /// sits on input 2 (a 2-in interface with the mic in the second socket, or an + /// aggregate whose first sub-device is silent) while the recorded mono had full + /// signal: the liveness gate never confirmed and the press failed **closed** + /// with "The microphone didn't start." on a mic that records perfectly. The + /// retired `AVAudioRecorder.averagePower(forChannel: 0)` metered the recorded + /// mono downmix, so channel 0 was the whole picture there — this was a behavior + /// change that rode inside the backend swap unnoticed. + /// + /// A max slightly over-reads the true downmix (one loud channel of two averages + /// quieter once mixed), which is the harmless direction for a floor probe and + /// for bars. + /// + /// The channel itself has a second not-yet-ready value, measured on AirPods: + /// until its first update it reports `-Float.greatestFiniteMagnitude` + /// (~-3.4e38), which can arrive *after* the first frames do. Both consumers + /// take it as silence — `MicLiveness` keeps waiting, `linearLevel` floors — + /// so it needs no clamping here, but a caller that reads the meter the instant + /// frames appear will see it. + func meteredPowerDB() -> Float { + let channels = output.connection(with: .audio)?.audioChannels ?? [] + guard !channels.isEmpty else { return -160 } + return channels.reduce(-Float.infinity) { max($0, $1.averagePowerLevel) } + } + + /// End capture and hand back everything recorded as raw S16LE PCM at the + /// dictation API's rate, releasing the device. + /// + /// Non-throwing, unlike the seam this replaces: the bytes are already in + /// memory in the upload encoding, so there is no read-back, decode or temp + /// file left to fail at. + /// + /// Synchronous, unlike `make` and `record()`, on two grounds. `stopRunning()` + /// measures 19–41 ms against the open's ~600 ms, which is not worth another + /// suspension point on the release path the transcript waits behind; and it + /// cannot race the `controlQueue` work, because every caller runs on a recorder + /// whose `record()` has already resumed (an abandoned bring-up tears down only + /// after the open returns). If a stop ever does need to overlap an open, it + /// belongs on `controlQueue` too — that is what the queue is for. + func stopAndReadPCM() -> Data { + session.stopRunning() + return state.withLock { $0.captured } + } + + /// End capture and throw the audio away, releasing the device — the teardown + /// behind a failed `record()`, an aborted bring-up, and a cancel. + func stopAndDiscard() { + session.stopRunning() + state.withLock { + $0.captured = Data() + $0.frameCount = 0 + } + } +} + +extension CaptureSessionRecorder: AVCaptureAudioDataOutputSampleBufferDelegate { + /// Sample buffers, on `delegateQueue`: copy the converted S16LE bytes out of + /// the block buffer and account the frames, under the lock. A buffer whose + /// bytes can't be copied is dropped whole — better a short gap than a blob + /// whose byte count and frame count disagree. + func captureOutput( + _ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, + from connection: AVCaptureConnection + ) { + guard let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) else { return } + let length = CMBlockBufferGetDataLength(blockBuffer) + guard length > 0 else { return } + var chunk = Data(count: length) + let status = chunk.withUnsafeMutableBytes { raw -> OSStatus in + // Empty is excluded above, so a nil base can't happen; answered as a + // plain non-noErr status rather than trapping, since dropping the buffer + // is this method's failure mode for every other copy problem too. + guard let base = raw.baseAddress else { return OSStatus(-1) } + return CMBlockBufferCopyDataBytes( + blockBuffer, atOffset: 0, dataLength: length, destination: base) + } + guard status == kCMBlockBufferNoErr else { return } + let frames = CMSampleBufferGetNumSamples(sampleBuffer) + state.withLock { + $0.captured.append(chunk) + $0.frameCount += frames + } + } +} diff --git a/Sources/BlurtEngine/Audio/MicCapture+Meter.swift b/Sources/BlurtEngine/Audio/MicCapture+Meter.swift index 969a1548..4ffba37c 100644 --- a/Sources/BlurtEngine/Audio/MicCapture+Meter.swift +++ b/Sources/BlurtEngine/Audio/MicCapture+Meter.swift @@ -8,7 +8,7 @@ extension MicCapture { /// the overlay's voice bars read as empty at rest and only move for speech. static let meterFloorDB: Float = -50 - /// Convert `AVAudioRecorder`'s dBFS meter power into the `0...1` the overlay + /// Convert the capture meter's dBFS power into the `0...1` the overlay /// expects, mapped linearly across `[meterFloorDB, 0]`. (A raw `pow(10, db/20)` /// amplitude leaves ambient noise well above zero, so the bars never rested /// at empty.) diff --git a/Sources/BlurtEngine/Audio/MicCapture+Warm.swift b/Sources/BlurtEngine/Audio/MicCapture+Warm.swift deleted file mode 100644 index b5cc270f..00000000 --- a/Sources/BlurtEngine/Audio/MicCapture+Warm.swift +++ /dev/null @@ -1,109 +0,0 @@ -import AVFoundation -import Foundation - -// The warm-recorder lifecycle — prepare ahead of the press, validate it still -// matches the live input, and let it expire — split from `MicCapture.swift` to -// stay within the lint file-length budget, like `MicCapture+Meter`. Members it -// reaches (`warm`, `preparedGeneration`, `logger`, `removeFile`, `makeRecorder`) -// are internal rather than private for that reason: `private` is file-scoped and -// can't cross the split. -extension MicCapture { - /// Pre-create and prepare a recorder so the first `start()` skips first-time - /// hardware route discovery. Does NOT begin capture — no mic indicator. Safe to - /// call multiple times; a failure here just leaves `start()` to prepare lazily. - public func warmUp() { - guard canPrepareWarmRecorder else { return } - prepareWarmRecorder() - } - - /// Whether it is safe to open the input for a *warm* recorder right now: - /// nothing is capturing, nothing is mid-bring-up, and no warm recorder is - /// already held. A scheduled re-warm that fails this has been overtaken — - /// a press got there first — and has nothing to do. - /// - /// `bringingUpCapture` is the load-bearing term. Both `activeRecorder` and - /// `warm` are nil across `start()`'s liveness wait, so testing them alone reads - /// a live capture as "idle" and prepares a second recorder onto the open input. - var canPrepareWarmRecorder: Bool { - activeRecorder == nil && warm == nil && !bringingUpCapture - } - - /// The warm recorder if it is still bound to `input`, else nil — discarding - /// (and cleaning up after) one that isn't. - /// - /// Reuse requires *positively* confirming the device is unchanged: an - /// unreadable route on either side leaves us unable to tell, and a recorder - /// bound to the wrong device doesn't fail loudly — it records the wrong mic, or - /// silence. Paying route activation is the cheaper mistake, so unknown means - /// discard. - func takeWarmRecorder(matching input: AudioRoute.InputSnapshot?) -> AVAudioRecorder? { - guard let held = warm else { return nil } - held.expiry?.cancel() - warm = nil - guard let warmed = held.input, let input, warmed.deviceID == input.deviceID else { - Self.removeFile(at: held.recorder.url) - Self.logger.info("discarded warm recorder — input device changed since warm-up") - return nil - } - return held.recorder - } - - /// Queues a re-warm to run once the current actor turn finishes, so the caller - /// (`stop()` / `cancelCapture()`) returns before the input is re-opened. - /// - /// `warmUp()` rather than a separate re-warm entry point: the two differ only - /// in when they are called, and both answer the same question — is it safe to - /// open the input for a warm recorder right now — so they share the guard - /// rather than each keeping a copy of it. - func scheduleRewarm() { - Task { [weak self] in - await self?.warmUp() - } - } - - /// Builds a recorder, records the input it is bound to, and starts its idle - /// countdown. A failure is non-fatal: `start()` then prepares lazily, exactly - /// as it did before any warm recorder existed. - func prepareWarmRecorder() { - do { - preparedGeneration += 1 - warm = WarmRecorder( - recorder: try Self.makeRecorder(), - input: AudioRoute.currentInput(), - generation: preparedGeneration) - armPreparedRecorderExpiry(generation: preparedGeneration) - Self.logger.info("prepared a warm recorder") - } catch { - Self.logger.error("warm-up failed: \(error.localizedDescription, privacy: .public)") - } - } - - /// Arms the idle countdown for the warm recorder identified by `generation`. - /// The ticket is what makes a stale expiry harmless — see - /// `releasePreparedRecorder(generation:)`. - func armPreparedRecorderExpiry(generation: Int) { - warm?.expiry?.cancel() - warm?.expiry = Task { [weak self] in - try? await Task.sleep(for: Self.preparedRecorderLifetime) - guard !Task.isCancelled else { return } - await self?.releasePreparedRecorder(generation: generation) - } - } - - /// Tears down an idle warm recorder, freeing the input device — which is what - /// lets a Bluetooth output route return to its full-quality profile. See - /// `preparedRecorderLifetime`. - /// - /// A stale expiry — one whose recorder was consumed by a press, or replaced by - /// a later re-warm — must do nothing at all, which is what `generation` buys. - /// Cancellation alone doesn't cover it: an expiry that already passed its - /// `!Task.isCancelled` check still gets its actor turn, and would otherwise nil - /// out the *live* expiry's handle (leaving the current warm recorder with no - /// countdown at all) and tear down a recorder prepared a moment ago. - func releasePreparedRecorder(generation: Int) { - guard let held = warm, held.generation == generation else { return } - warm = nil - Self.removeFile(at: held.recorder.url) - Self.logger.info("released idle warm recorder") - } -} diff --git a/Sources/BlurtEngine/Audio/MicCapture.swift b/Sources/BlurtEngine/Audio/MicCapture.swift index eb3cc9a0..f821713a 100644 --- a/Sources/BlurtEngine/Audio/MicCapture.swift +++ b/Sources/BlurtEngine/Audio/MicCapture.swift @@ -1,15 +1,15 @@ -@preconcurrency import AVFoundation import Foundation import os -/// Captures mic audio with `AVAudioRecorder`, which records straight to the -/// 16 kHz / mono / 16-bit PCM the dictation API wants — so there's no manual tap, -/// sample-rate conversion, or PCM plumbing here. Each session uses a freshly -/// created recorder, which resolves the *current* default input device at -/// `record()` time. That's the whole reason this is no longer an `AVAudioEngine`: -/// the engine's input graph bound to one device and went stale on a device switch -/// (mic ↔ built-in), raising `-10868` (`kAudioUnitErr_FormatNotSupported`) or -/// quietly capturing all-zero buffers. A per-session recorder can't go stale. +/// Captures mic audio as the 16 kHz / mono / 16-bit PCM the dictation API wants +/// — so there's no manual tap, resample pass, or PCM plumbing here. Each +/// session uses a freshly built `CaptureSessionRecorder`: an `AVCaptureSession` +/// around the *current* resolution of the user's selection — the device pinned +/// in Settings (`MicDeviceSelection`), or the system default input. The fresh +/// recorder per session is the whole reason this is not a long-lived engine +/// graph: that graph bound its input to one device and went stale on a device +/// switch, raising `-10868` or quietly capturing all-zero buffers; per-session +/// recorders can't go stale. public actor MicCapture: MicCaptureProtocol { // Subsystem/category make these lines findable via: // log show --predicate 'subsystem == "dev.alex.blurt"' --last 1h @@ -20,91 +20,33 @@ public actor MicCapture: MicCaptureProtocol { public nonisolated let levels: AsyncStream private nonisolated let levelsContinuation: AsyncStream.Continuation - /// The geometry the recorder converts hardware audio to on the fly. The dictation - /// API's rate (`SyncSTTLimits.sampleRate`) — the same one the pipeline hands - /// the transcriber — so `stop()` returns bytes ready to upload with no - /// resampling or re-encoding pass. - private static let targetSampleRate = Double(SyncSTTLimits.sampleRate) - - /// A recorder prepared ahead of the press so `start()` doesn't pay hardware - /// route activation on the hot path, together with everything that belongs to - /// *that* recorder: the input it is bound to, its idle countdown, and the - /// ticket that countdown carries. - /// - /// One optional rather than four parallel fields, so "a recorder without its - /// input snapshot" and "an expiry without its recorder" are unrepresentable - /// instead of merely never written. - /// - /// Filled by `warmUp()` at launch and **re-filled after every capture** (see - /// `scheduleRewarm`), because that cost is paid per session, not once: - /// `prepareToRecord()` is where the route is resolved and opened, and on a - /// Bluetooth input that means renegotiating the link into its mic-capable - /// mode — hundreds of milliseconds, sometimes over a second, during which the - /// user has pressed the key and nothing has happened. Warming only the first - /// session (the previous behavior) hid that cost for one dictation out of N. - /// - /// Still a *fresh recorder per session*, which is the invariant the - /// `AVAudioEngine` rewrite bought: the warm recorder is validated against the - /// live default input before it is used (`takeWarmRecorder`) and discarded - /// rather than reused when the device has changed underneath it. - var warm: WarmRecorder? - - /// Bumped for each warm recorder prepared, so an expiry whose recorder has - /// since been consumed or replaced can recognise itself as stale. See - /// `releasePreparedRecorder(generation:)` for why cancelling the task isn't - /// sufficient on its own. - var preparedGeneration = 0 - - /// A prepared-but-not-started recorder and the state that only makes sense - /// alongside it. - struct WarmRecorder { - let recorder: AVAudioRecorder - /// The default input it was built against. `AVAudioRecorder` resolves its - /// device once, at `prepareToRecord()`, and never re-resolves — so without - /// this a recorder warmed while the built-in mic was default would keep - /// recording from it after the user connected their AirPods. See - /// `AudioRoute.InputSnapshot.deviceID` for why identity is the device ID. - let input: AudioRoute.InputSnapshot? - /// Releases this recorder once it has gone unused for - /// `preparedRecorderLifetime`. See that constant for why holding one open - /// forever is not an option. - var expiry: Task? - /// The ticket `expiry` carries, so a stale one can identify itself. - let generation: Int - } + /// Reads the persisted microphone selection, once per session bring-up (and + /// per warm-up). A closure so tests inject a fixed selection instead of the + /// process defaults; production reads `MicDeviceStore` per capture, so a + /// Settings change applies to the very next press — the same re-read-per-use + /// rule as the key terms. + let deviceSelection: @Sendable () -> MicDeviceSelection /// The recorder for the in-flight session; nil between `stop()` and `start()`. - var activeRecorder: AVAudioRecorder? - /// The in-flight session's input transport, sampled once at `start()`. Read by - /// `stop()` to size the tail linger — sampled at start rather than re-read at - /// stop so a device switch mid-utterance can't make the two halves of one - /// capture disagree. - private var activeTransportType: UInt32? + private var activeRecorder: CaptureSessionRecorder? + /// How long `stop()` keeps capturing past key-up for the in-flight session — + /// `.zero` off Bluetooth. Decided at `start()` from the resolved transport + /// rather than re-read at stop, so a device switch mid-utterance can't make + /// the two halves of one capture disagree; stored as the `Duration` itself + /// because the transport has no other reader once the cap is chosen. + private var activeTailLinger: Duration = .zero /// Incremented by every `stop()` / `cancelCapture()`. `start()` snapshots it /// before suspending in the liveness wait — its one internal suspension — and /// re-checks after, so a teardown that interleaves during the wait wins. /// Without it, a reentrant caller's stop saw `activeRecorder == nil`, returned /// an empty "clean stop", and the not-yet-installed recorder kept capturing - /// (mic indicator hot, temp WAV leaked) until `start()` resumed. Unreachable + /// (mic indicator hot) until `start()` resumed. Unreachable /// through `DictationSession` — its serial command queue runs release/cancel /// only after the press turn completes — but this is a public actor, and any /// host can call it unqueued. private var stopGeneration = 0 - /// True from `record()` succeeding until the capture is installed or torn - /// down — i.e. across the liveness wait. - /// - /// Needed because during that window **both** recorder slots are nil: - /// `activeRecorder` isn't installed until the wait returns (the recorder stays - /// confined to `start()` so nothing can touch it while the poll loop reads its - /// clock off-actor), and `warm` was consumed on the way in. Without this, - /// `warmUp()` — which every re-warm goes through — reads those two nils as "no - /// capture in flight" and prepares a *second* recorder onto the already-live - /// input, which is very reachable since `stop()` schedules a re-warm that can - /// land inside the next press's bring-up. - var bringingUpCapture = false - /// Polls the active recorder's meter and feeds `levels` while recording. private var meterTask: Task? /// The last value `emitLevel` put on the stream, so an unchanged tick can be @@ -126,20 +68,10 @@ public actor MicCapture: MicCaptureProtocol { /// `meterIntervalSeconds` as the `Duration` the meter task sleeps for. private static let meterInterval = Duration.seconds(meterIntervalSeconds) - /// How long a prepared-but-unused recorder is held before being torn down. - /// - /// The warm recorder is the fix for per-session route activation, but it is - /// not free to hold: a prepared recorder keeps the input device open, and an - /// open input is exactly what pins AirPods in their mic-capable profile, where - /// *output* audio is degraded. Holding one indefinitely would trade dictation - /// latency for permanently worse music. This bounds that: back-to-back - /// dictations — the case the warm recorder exists for — land well inside the - /// window, and a user who stops dictating gets their output route back shortly - /// after. The next press past the window simply prepares lazily, which is the - /// behavior that shipped before the re-warm existed. - static let preparedRecorderLifetime = Duration.seconds(60) - - public init() { + public init( + deviceSelection: @escaping @Sendable () -> MicDeviceSelection = { MicDeviceStore().selection } + ) { + self.deviceSelection = deviceSelection // The continuation is fed from a ~20 Hz meter timer; the levels stream is a // meter, not the captured signal — the consumer only renders the most recent // value — so cap it at the newest single element. @@ -148,83 +80,117 @@ public actor MicCapture: MicCaptureProtocol { self.levelsContinuation = continuation } - public func start() async throws { - // One CoreAudio read per session, answering both questions this capture has - // about its input: whether the warm recorder is still bound to it, and - // whether its link buffers a tail worth waiting for at stop. - let input = AudioRoute.currentInput() - // Reuse the warm recorder when it is still bound to the current default - // input; otherwise build a fresh one. `??` only evaluates `makeRecorder()` - // when nothing usable was warmed. - let recorder = try takeWarmRecorder(matching: input) ?? Self.makeRecorder() + /// Absorbs the one-off cost of this process's first touch of AVFoundation's + /// capture stack, by building a session for the current selection and dropping + /// it. Holds no state: there is no warm recorder, nothing to validate against + /// the live route at press time, and nothing to tear down. + /// + /// That is deliberate, and it is a measurement rather than a preference. This + /// actor used to keep a prepared recorder between presses — re-warmed after + /// every capture, validated against the resolved input's device identity and + /// pin, and guarded by a bring-up flag so a re-warm couldn't race a live + /// press. What that machinery bought, timed against + /// `kAudioDevicePropertyDeviceIsRunningSomewhere` on real hardware, was ~15 ms + /// of a ~600 ms bring-up: neither building an `AVCaptureSession` **nor** the + /// retired `AVAudioRecorder.prepareToRecord()` opens the device (the indicator + /// bit stays clear through both), so neither could pre-pay the route + /// activation the warm recorder existed to hide. `record()`'s `startRunning()` + /// pays all of it, inside the connecting window the liveness gate already + /// covers. Only the first build in a process is worth pre-paying — ~185 ms of + /// framework set-up (~90–125 ms of it the first device query, the rest the + /// session and its input), against ~5 ms for every build after it — and that + /// needs no state to hold. + /// + /// Reads the pin straight off the selection rather than running `start()`'s + /// input resolution: what is being pre-paid is process-level framework set-up, + /// which is the same whichever device is named, and resolving here would spend + /// two device lookups on a value the recorder resolves again anyway — and + /// would log `resolveInput`'s "pinned microphone not connected" line at launch, + /// where nothing is being recorded. Naming the pin still costs a microsecond, + /// so it stays: if a driver has its own first-load cost, this pre-pays it. + /// + /// Never throws (a failure just leaves `start()` to build the real one) and + /// never begins capture, so no input indicator appears. + public func warmUp() async { + do { + _ = try await CaptureSessionRecorder.make(pinnedUID: deviceSelection().pinnedUID) + Self.logger.info("warmed the capture stack") + } catch { + Self.logger.error("warm-up failed: \(error.localizedDescription, privacy: .public)") + } + } - // record() returns false when no usable input device is available (unplugged, + public func start() async throws { + // The selection, resolved once per press: which device to pin the session to + // (or nil to follow the system default, including the missing-pin fallback), + // and the transport the liveness cap and the tail linger key off. + let resolved = Self.resolveInput(selection: deviceSelection()) + + // Snapshotted before the first suspension, which is the *build* — both it + // and the open hop off-actor (see `CaptureSessionRecorder.make`), so a stop + // or cancel can land anywhere in the bring-up, and it has to win over a + // bring-up that is no longer wanted. + let generationBeforeBringUp = stopGeneration + let recorder = try await CaptureSessionRecorder.make(pinnedUID: resolved.pinnedUID) + // Building is a suspension too, so it needs the same check as the two below. + // It had none when the build moved off-actor, which is the failure this + // helper exists to stop repeating: a guard hand-written per suspension means + // the next suspension arrives unguarded. + try checkStillWanted(since: generationBeforeBringUp, stage: "the session build") + + // One teardown for every exit that doesn't install the recorder — a failed + // open, either abandonment check, the fail-closed timeout. Written once as a + // `defer` rather than at each `throw`, because forgetting it at any of them + // leaves a session capturing with nothing holding it: a hot input indicator + // and no owner, which is the failure `stopGeneration` exists to prevent. A + // future suspension point in this bring-up inherits the teardown for free + // and only has to add its own abandonment check. + var installed = false + defer { if !installed { recorder.stopAndDiscard() } } + + // record() answers false when no usable input device is available (unplugged, // asleep, route lost). Surface that as a thrown Swift error so // DictationSession.press() reports `.audioCaptureFailed` instead of recording - // nothing. (Unlike AVAudioEngine's installTap, no path here can raise an - // uncatchable Obj-C exception, so there's no degenerate-format guard to keep.) - guard recorder.record() else { + // nothing. (No path here can raise an uncatchable Obj-C exception, so there's + // no degenerate-format guard to keep.) + guard await recorder.record() else { Self.logger.error("recorder.record() returned false — no usable input device") - Self.removeFile(at: recorder.url) throw BlurtError.audioCaptureFailed(underlying: MicCaptureError.noInputDevice) } - // The input is open from here, so claim the bring-up window before the first - // suspension — see `bringingUpCapture`. `defer` clears it on every exit, - // including the abort throw below. - bringingUpCapture = true - defer { bringingUpCapture = false } - - // `record()` returning true only means the AudioQueue started — not that the - // input route is delivering frames. A Bluetooth mic spends up to a couple of - // seconds switching into its mic-capable profile first, and the OS captures - // nothing in that window, so returning here immediately cues the user to - // speak into a dead mic and the first words never reach the transcript. Hold - // — `DictationSession` keeps the pill in `.connecting` and the start chime - // waits — until the recorder is clocking *and* metering real samples, capped - // per transport and FAILING CLOSED on timeout (the throw below). `MicLiveness` - // owns both policies and the reasoning, including why the clock alone was - // not enough. The re-warm above is what usually makes this return at once; - // the gate is what makes it correct when it doesn't. - let timeout = MicLiveness.timeout(forTransportType: input?.transportType) - let generationBeforeWait = stopGeneration + // A teardown or cancel that landed during the open: the `defer` tears the + // recorder down rather than sitting through the liveness wait for a capture + // nobody wants. + try checkStillWanted(since: generationBeforeBringUp, stage: "the device open") + + // `record()` returning true only means the session started running — not + // that the input route is delivering frames. A Bluetooth mic spends up to a + // couple of seconds switching into its mic-capable profile first, and the OS + // captures nothing in that window, so returning here immediately cues the + // user to speak into a dead mic and the first words never reach the + // transcript. Hold — `DictationSession` keeps the pill in `.connecting` and + // the start chime waits — until the recorder is delivering frames *and* + // metering real samples, capped per transport and FAILING CLOSED on timeout + // (the throw below). `MicLiveness` owns both policies and the reasoning, + // including why frame arrival alone was not enough. + let timeout = MicLiveness.timeout(forTransportType: resolved.transportType) // Both probes read off-actor (`waitUntilLive` is nonisolated), safe by // confinement: the polls run sequentially in one task and nothing else // references this recorder while `start()` is suspended — it isn't - // `activeRecorder` yet, the warm slot was cleared above, and the meter task - // hasn't started. Meters read stale until refreshed, exactly as in `emitLevel`. + // `activeRecorder` yet, and the meter task hasn't started. Meters read stale + // until refreshed, exactly as in `emitLevel`. let gap = await MicLiveness.waitUntilLive( timeout: timeout, clock: ContinuousClock(), - currentTime: { recorder.currentTime }, - inputPowerDB: { - recorder.updateMeters() - return recorder.averagePower(forChannel: 0) - }) + deliveredFrames: { recorder.deliveredFrames }, + inputPowerDB: { recorder.meteredPowerDB() }) - // Two ways the bring-up can be abandoned while suspended, both ending the - // same way — tear the recorder down instead of installing it, so nothing is - // left capturing and no temp file is orphaned: - // - // - A teardown landed (`stopGeneration` moved), so the caller's stop has to - // stay a real stop rather than returning an empty "clean" one. - // - The task was cancelled, which is how a cancel preempts the wait: - // `waitUntilLive` returns as soon as it sees it, so this is the difference - // between an Escape acting now and acting in `bluetoothTimeout`. It must be - // distinguished from the timeout, which returns nil too but is a reported - // *failure* (the throw below) — a cancel is the user's own act, not a fault. - guard stopGeneration == generationBeforeWait, !Task.isCancelled else { - recorder.stop() - Self.removeFile(at: recorder.url) - Self.logger.info("start aborted — teardown or cancellation during the liveness wait") - throw CancellationError() - } + try checkStillWanted(since: generationBeforeBringUp, stage: "the liveness wait") // The gate's outcome, worded in `MicLiveness` so the line a field report is // read off is unit-tested. See `logSummary` for what it has to carry. - recorder.updateMeters() let summary = MicLiveness.logSummary( - gap: gap, timeout: timeout, transportType: input?.transportType, - powerDB: recorder.averagePower(forChannel: 0)) + gap: gap, timeout: timeout, transportType: resolved.transportType, + powerDB: recorder.meteredPowerDB()) Self.logger.log(level: gap == nil ? .error : .info, "\(summary, privacy: .public)") // No confirmed signal within the cap: fail CLOSED — tear the capture down and // surface the same `.audioCaptureFailed` presentation as the record()-false @@ -232,43 +198,30 @@ public actor MicCapture: MicCaptureProtocol { // "speak now" over a mic delivering nothing; a recording made anyway would // discard the user's words after the fact. The recorder was never installed, // so no stopGeneration bump — a concurrent stop() correctly sees no active - // capture — and `bringingUpCapture` is cleared by the defer above. + // capture. guard gap != nil else { - recorder.stop() - Self.removeFile(at: recorder.url) throw BlurtError.audioCaptureFailed(underlying: MicCaptureError.inputNeverDelivered) } activeRecorder = recorder - activeTransportType = input?.transportType + activeTailLinger = AudioTransport.tailLinger(forTransportType: resolved.transportType) lastEmittedLevel = nil - Self.logger.info("start recording to \(recorder.url.lastPathComponent, privacy: .public)") + installed = true + Self.logger.info( + "start recording from \(resolved.pinnedUID ?? "system default", privacy: .public)") startMeterTimer() } public func stop() async throws -> Data { - // Read before the suspension below, so this capture's decision can't be - // rewritten by whatever a later `start()` sets. - let linger = AudioTransport.tailLinger(forTransportType: activeTransportType) + let linger = activeTailLinger guard let recorder = detachActiveRecorder() else { return Data() } if linger > .zero { // Keep capturing for a moment past key-up so the audio still travelling - // over the link lands in the file instead of being truncated. See + // over the link lands in the recording instead of being truncated. See // `AudioTransport.tailLinger(forTransportType:)`. try? await Task.sleep(for: linger) } - recorder.stop() - - let url = recorder.url - defer { - Self.removeFile(at: url) - // Re-arm for the *next* press now that the device is free, so the route - // activation this session just paid for isn't paid again. Scheduled rather - // than done inline: preparing re-opens the input, which is the slow part, - // and `stop()` is on the release path the transcript waits behind. - scheduleRewarm() - } - let pcm = try Self.decodePCM(fromFileAt: url) + let pcm = recorder.stopAndReadPCM() let sampleCount = pcm.count / SyncSTTLimits.bytesPerSample let durationMs = SyncSTTLimits.durationMs(ofPCMBytes: pcm.count) @@ -281,10 +234,9 @@ public actor MicCapture: MicCaptureProtocol { /// `DictationSession`'s cancels. /// /// Deliberately *not* `stop()`-and-discard: the user asked for nothing to - /// happen, so neither of `stop()`'s costs is worth paying. The Bluetooth tail - /// linger would delay the `.cancelled` phase (and with it the pill's dismissal) - /// to preserve audio about to be deleted, and reading the whole recording back - /// off disk would decode a blob with no consumer. + /// happen, so `stop()`'s one cost isn't worth paying. The Bluetooth tail + /// linger would delay the `.cancelled` phase (and with it the pill's + /// dismissal) to preserve audio about to be deleted. /// /// Not marked `throws`, because nothing on this path can fail — a /// non-throwing implementation satisfies the `throws` requirement fine. The @@ -293,10 +245,30 @@ public actor MicCapture: MicCaptureProtocol { /// developer-mode failure log keeps working for hosts that take it. public func cancelCapture() { guard let recorder = detachActiveRecorder() else { return } - recorder.stop() - Self.removeFile(at: recorder.url) + recorder.stopAndDiscard() Self.logger.info("cancelled capture, discarded audio") - scheduleRewarm() + } + + /// Throws when this bring-up has been abandoned while suspended, in either of + /// the two ways that end the same — the caller's `defer` tears the recorder + /// down instead of installing it, so nothing is left capturing: + /// + /// - A teardown landed (`stopGeneration` moved), so the caller's stop has to + /// stay a real stop rather than returning an empty "clean" one. + /// - The task was cancelled, which is how a cancel preempts the bring-up: + /// `waitUntilLive` returns as soon as it sees it, so this is the difference + /// between an Escape acting now and acting in `bluetoothTimeout`. It must be + /// distinguished from the liveness timeout, which also abandons the press but + /// is a reported *failure* — a cancel is the user's own act, not a fault. + /// + /// One definition called after each of `start()`'s suspensions, rather than a + /// guard copied per suspension: the copies drifted the moment a third + /// suspension (the off-actor session build) arrived without one. + private func checkStillWanted(since generation: Int, stage: String) throws { + guard stopGeneration == generation, !Task.isCancelled else { + Self.logger.info("start aborted — teardown or cancellation during \(stage, privacy: .public)") + throw CancellationError() + } } /// Ends the current capture's claim on the actor's state and hands back the @@ -306,34 +278,56 @@ public actor MicCapture: MicCaptureProtocol { /// the generation bump in particular is what `start()` re-checks across its /// liveness wait to know a teardown landed, so a third exit that forgot it /// would let an abandoned bring-up install itself and keep capturing. - private func detachActiveRecorder() -> AVAudioRecorder? { + private func detachActiveRecorder() -> CaptureSessionRecorder? { stopGeneration += 1 meterTask?.cancel() meterTask = nil - defer { activeRecorder = nil } + defer { + activeRecorder = nil + activeTailLinger = .zero + } return activeRecorder } - // MARK: - Recorder construction + // MARK: - Input resolution - /// Build a recorder that writes mono 16-bit little-endian PCM at the target - /// rate into a unique temp file. `prepareToRecord()` does the heavy route/buffer - /// setup so the subsequent `record()` starts promptly. - static func makeRecorder() throws -> AVAudioRecorder { - let url = FileManager.default.temporaryDirectory - .appendingPathComponent("blurt-\(UUID().uuidString).wav") - let settings: [String: Any] = [ - AVFormatIDKey: kAudioFormatLinearPCM, - AVSampleRateKey: targetSampleRate, - AVNumberOfChannelsKey: 1, - AVLinearPCMBitDepthKey: 16, - AVLinearPCMIsFloatKey: false, - AVLinearPCMIsBigEndianKey: false, - ] - let recorder = try AVAudioRecorder(url: url, settings: settings) - recorder.isMeteringEnabled = true - recorder.prepareToRecord() - return recorder + /// What a press needs to know about its input, and nothing more: the UID the + /// session must pin to (nil for the system default — no pin, or the + /// missing-device fallback), and the transport the two transport-keyed + /// policies read (`MicLiveness.timeout`, `AudioTransport.tailLinger`). + /// + /// It used to carry the resolved device's `AudioDeviceID` as well, purely so a + /// warm recorder could be checked for still being bound to it. With no warm + /// recorder to validate, nothing asks which device this is — only how to open + /// it and how its link behaves. + struct ResolvedInput { + let pinnedUID: String? + let transportType: UInt32? + } + + /// One trip through the selection policy: check whether the pinned UID still + /// names a connected device, let `MicDeviceSelection.effective` — the pure, + /// unit-tested rule — pick the fallback, and read the transport of whichever + /// input won. Hardware-adjacent glue in a coverage-excluded file; the decision + /// itself stays testable. + static func resolveInput(selection: MicDeviceSelection) -> ResolvedInput { + // One device lookup, answering both questions: the transport is nil exactly + // when no connected device carries the pin (see + // `AudioInputDevices.transportType(forUID:)`), so presence is `!= nil` and + // the value is reused below instead of read a second time. + let pinnedTransport = selection.pinnedUID.flatMap(AudioInputDevices.transportType(forUID:)) + switch selection.effective(pinnedDevicePresent: pinnedTransport != nil) { + case .pinned(let uid): + return ResolvedInput(pinnedUID: uid, transportType: pinnedTransport) + case .systemDefault: + if case .pinned = selection { + // Graceful degradation, not an error: the pin stays stored, and this + // capture records what an un-pinned one would. + logger.info("pinned microphone not connected — recording the system default input") + } + return ResolvedInput( + pinnedUID: nil, transportType: AudioInputDevices.systemDefaultTransportType()) + } } // MARK: - Level metering @@ -354,8 +348,7 @@ public actor MicCapture: MicCaptureProtocol { private func emitLevel() { guard let recorder = activeRecorder else { return } - recorder.updateMeters() - let level = Self.linearLevel(fromPowerDB: recorder.averagePower(forChannel: 0)) + let level = Self.linearLevel(fromPowerDB: recorder.meteredPowerDB()) // Only yield transitions. `linearLevel` floors room ambient to exactly 0, so a // silent stretch would otherwise push the same value 20×/s, resuming the // host's `@MainActor` observer each time just to have it discard the @@ -367,33 +360,7 @@ public actor MicCapture: MicCaptureProtocol { } // The dB→0...1 conversion `emitLevel` uses lives in `MicCapture+Meter.swift` - // — pure math the coverage gate counts, unlike this hardware-bound actor. The - // warm-recorder lifecycle (`warmUp`, `takeWarmRecorder`, the re-warm and its - // expiry) lives in `MicCapture+Warm.swift`, split off for the lint - // file-length budget — which is why the prepared-recorder state, `logger`, - // `removeFile` and `makeRecorder` are internal rather than private. - - // MARK: - File helpers - - /// Read a recorded PCM file back as raw S16LE bytes — the dictation API's upload - /// encoding. The on-disk WAV already holds 16-bit int samples, so asking - /// `AVAudioFile` for the int16 common format makes this a straight copy-out: - /// no detour through Float32 (which the default `processingFormat` would - /// impose, and which the transcriber would only convert straight back). Int16 - /// is host-endian; Apple platforms (arm64/x86_64) are little-endian, so the - /// bytes are already the S16LE the dictation API expects. - static func decodePCM(fromFileAt url: URL) throws -> Data { - let file = try AVAudioFile(forReading: url, commonFormat: .pcmFormatInt16, interleaved: true) - let frameCount = AVAudioFrameCount(file.length) - guard frameCount > 0, - let buffer = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: frameCount) - else { return Data() } - try file.read(into: buffer) - guard let channel = buffer.int16ChannelData?[0] else { return Data() } - return Data(bytes: channel, count: Int(buffer.frameLength) * SyncSTTLimits.bytesPerSample) - } - - static func removeFile(at url: URL) { - try? FileManager.default.removeItem(at: url) - } + // — pure math the coverage gate counts, unlike this hardware-bound actor, + // which is why `logger` and `deviceSelection` are internal rather than + // private. } diff --git a/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift b/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift index 1adb65ca..0072e9bc 100644 --- a/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift +++ b/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift @@ -14,8 +14,9 @@ public protocol MicCaptureProtocol: Sendable { func start() async throws /// Stop capture and return the captured audio as raw S16LE PCM bytes — the /// exact encoding the dictation request uploads, so no conversion pass sits on - /// the release hot path. Throws if the captured audio couldn't be read back, - /// so the pipeline can surface an error instead of silently dropping speech. + /// the release hot path. `throws` so a conformer that has to fetch the audio + /// from somewhere can surface a failure instead of silently dropping speech; + /// `MicCapture` accumulates it in memory as it arrives and never does. func stop() async throws -> Data /// Stop capture and discard the audio — the teardown behind a *cancel*, where /// the user asked for nothing to happen. Split from `stop()` because the two @@ -30,11 +31,13 @@ public protocol MicCaptureProtocol: Sendable { /// through the same seam they inject — a stub without a meter satisfies it /// for free instead of every composition threading a side-channel stream. var levels: AsyncStream { get } - /// Optionally pre-open the capture device so the first `start()` doesn't pay - /// hardware route discovery on the hot path. Must not begin capture (no mic - /// indicator) and must not throw — a failure just means `start()` prepares - /// lazily as before. Declared here (not only in the default extension) so it - /// dispatches dynamically through `any MicCaptureProtocol`. + /// Optionally pay whatever set-up cost the first `start()` would otherwise + /// pay on the hot path. Must **not** open the input device — no capture, no + /// mic indicator, nothing held between presses — and must not throw; a failure + /// just means `start()` pays it. `MicCapture` documents what is and isn't + /// pre-payable here, which is less than it looks. Declared here (not only in + /// the default extension) so it dispatches dynamically through + /// `any MicCaptureProtocol`. func warmUp() async } diff --git a/Sources/BlurtEngine/Audio/MicDeviceStore.swift b/Sources/BlurtEngine/Audio/MicDeviceStore.swift new file mode 100644 index 00000000..ea00ce17 --- /dev/null +++ b/Sources/BlurtEngine/Audio/MicDeviceStore.swift @@ -0,0 +1,84 @@ +import Foundation + +/// Which input device dictation records from: the system's default input (the +/// unset default, and Blurt's only behavior before microphone selection +/// existed), or one specific device pinned by its persistent CoreAudio UID. +/// +/// This is the *pure* half of microphone selection — the decode rule and the +/// missing-device fallback — kept apart from the hardware questions (what +/// devices exist, what a UID currently resolves to), which live in +/// `AudioInputDevices` and are excluded from the coverage gate. These rules are +/// what `swift test` pins. +public enum MicDeviceSelection: Hashable, Sendable { + /// Follow the system's default input, re-resolved at every capture. + case systemDefault + /// Record from the device with this UID (`kAudioDevicePropertyDeviceUID`). + /// The UID rather than an `AudioDeviceID` because only the UID survives an + /// unplug/replug and a reboot — and because it is the identity + /// `AVCaptureDevice(uniqueID:)` takes, so the string stored here is the one + /// that opens the device at press time, not a translation of it. + case pinned(uid: String) + + /// Decode the persisted slot: the empty string — which is also what an unset + /// key reads back as — means "same as system". Shared by `MicDeviceStore` and + /// the Settings picker (which observes the raw slot via `@AppStorage`), so + /// the two can't disagree about what an untouched install means; the same + /// rule as `SoundPackCatalog.fromPersisted` and `TriggerKey.fromPersisted`. + public static func fromPersisted(_ raw: String) -> MicDeviceSelection { + raw.isEmpty ? .systemDefault : .pinned(uid: raw) + } + + /// The pinned device's UID, or nil for the system default — what a recorder + /// is built around. Here beside the other encode/decode rules rather than + /// re-derived by callers switching on the case. + var pinnedUID: String? { + switch self { + case .systemDefault: nil + case .pinned(let uid): uid + } + } + + /// The raw value the store writes for this selection — `fromPersisted`'s + /// inverse, owned here so the encode and decode rules sit side by side. + var persistedValue: String { + switch self { + case .systemDefault: "" + case .pinned(let uid): uid + } + } + + /// The selection a capture should actually bind to, given whether the pinned + /// device is currently present. A pinned device that has disappeared falls + /// back to the system default rather than failing the press: unplugging a USB + /// mic should degrade dictation to the built-in one, not break it. The pin + /// itself stays stored, so reconnecting the device pins it again with no trip + /// through Settings. + func effective(pinnedDevicePresent: Bool) -> MicDeviceSelection { + guard case .pinned = self, !pinnedDevicePresent else { return self } + return .systemDefault + } +} + +/// Persists the microphone selection in `UserDefaults` as the pinned device's +/// UID string, empty (or unset) meaning "same as system". Same shape as +/// `TriggerKeyStore`; lives in the engine so its key is a `DefaultsKey` case and +/// the setting joins `PersistedSettings.resetAll`'s sweep by construction. +public struct MicDeviceStore { + /// UserDefaults key holding the pinned device UID. Public so SwiftUI views + /// can observe it directly (e.g. `@AppStorage`) and re-render on change. + public static var defaultsKey: String { DefaultsKey.micDeviceUID.key } + private let defaults: UserDefaults + + public init(defaults: UserDefaults = .standard) { + self.defaults = defaults + } + + public var selection: MicDeviceSelection { + get { + MicDeviceSelection.fromPersisted(defaults.string(forKey: Self.defaultsKey) ?? "") + } + nonmutating set { + defaults.set(newValue.persistedValue, forKey: Self.defaultsKey) + } + } +} diff --git a/Sources/BlurtEngine/Audio/MicLiveness.swift b/Sources/BlurtEngine/Audio/MicLiveness.swift index 6e21a774..bfe39966 100644 --- a/Sources/BlurtEngine/Audio/MicLiveness.swift +++ b/Sources/BlurtEngine/Audio/MicLiveness.swift @@ -1,5 +1,3 @@ -import Foundation - /// The pure decision half of `MicCapture.start()`'s liveness gate: how long to /// wait for the input device to actually deliver frames, and the polling loop /// that detects when it has. Kept out of the hardware-bound capture actor (the @@ -9,7 +7,7 @@ enum MicLiveness { /// The first re-check delay, doubling up to `maxPollInterval`. /// /// Geometric rather than a fixed quantum because the two cases this loop - /// serves want opposite things. `AVAudioRecorder.currentTime` is 0 the instant + /// serves want opposite things. No frames have arrived the instant /// `record()` returns, so *every* press sleeps at least once before /// `.recording`, the start chime and the meter — on a wired mic that quantum /// is the entire wait, and it should be as small as possible. A real Bluetooth @@ -27,11 +25,9 @@ enum MicLiveness { /// Wait cap for Bluetooth inputs: bringing an AirPods mic up means a profile /// switch into the mic-capable mode that takes ~1–2 s, and the link drops back /// to the output-only profile after an idle gap — so a dictation after a pause - /// pays it again, not just the first one. - /// - /// `MicCapture`'s re-warm shortens how *often* this is paid (it keeps the - /// input open between dictations); this cap governs what happens when it is - /// paid anyway. + /// pays it again, not just the first one. Nothing on the capture path can + /// pre-pay it — `MicCapture.warmUp()` holds that measurement — so this cap + /// governs every press that lands on a cold link. static let bluetoothTimeout: Duration = .milliseconds(2500) /// Wait cap for a transport known to be wired or on-board @@ -61,12 +57,20 @@ enum MicLiveness { /// /// This is not a speech threshold and must never be raised into one. All it /// has to separate is "the buffers are literally zeroes" from "the ADC is - /// handing us something": `averagePower(forChannel:)` reports dBFS in roughly - /// `[-160, 0]`, a zero-filled buffer bottoms out at or near -160, and a single - /// least-significant bit of a 16-bit sample is already about -90, so any real - /// analog noise floor — including a live mic in a dead-quiet room, which still - /// reads its own self-noise — sits far above this. -115 is in the empty band - /// between the two. + /// handing us something": the capture meter + /// (`AVCaptureAudioChannel.averagePowerLevel`, dBFS with full scale at 0 — + /// the same scale the retired recorder's meter reported) reads a zero-filled + /// buffer at or near its floor, while a single least-significant bit of a + /// 16-bit sample is already about -90, so any real analog noise floor — + /// including a live mic in a dead-quiet room, which still reads its own + /// self-noise — sits far above this. -115 is in the empty band between the + /// two. Verified on AirPods, which was the case that mattered: before the + /// channel's first update the meter reads `-Float.greatestFiniteMagnitude` + /// (~-3.4e38), not the -160 a settled meter floors at, and a live AirPods mic + /// reads far above -115 within a few hundred ms of the first frame. Both fall + /// on the correct side of this floor, and the `!(power > floor)` spelling in + /// `waitUntilLive` is what makes the sentinel — a wildly out-of-range value, + /// like NaN — count as *not live* rather than sneaking through a comparison. /// /// A speech-level floor (`MicCapture.meterFloorDB` is -50) would turn this /// gate into voice-activity detection and fail every press in a quiet room @@ -113,21 +117,21 @@ enum MicLiveness { } /// Polls the recorder on a backoff (see `initialPollInterval`) until it is - /// genuinely live: the clock has advanced past zero **and** one meter reading - /// is above `silenceFloorDB`. + /// genuinely live: at least one frame has been delivered **and** one meter + /// reading is above `silenceFloorDB`. /// - /// The clock alone is not enough, which is what shipped and what still lost - /// the first words on AirPods. macOS can hand a stale or not-yet-switched - /// device's queue **all-zero buffers** (the failure that retired the + /// Frame arrival alone is not enough, which is what shipped and what still + /// lost the first words on AirPods. macOS can hand a stale or + /// not-yet-switched device **all-zero buffers** (the failure that retired the /// `AVAudioEngine` capture path — see `MicCapture`'s header), and frames of - /// digital silence advance `currentTime` exactly like real audio does. So the - /// clock term was satisfied on the first ~1 ms poll while the link was still - /// renegotiating, the wait returned immediately, and the "Connecting…" pill - /// flashed past instead of holding. + /// digital silence arrive and count exactly like real audio does. So the + /// arrival term was satisfied on the first ~1 ms poll while the link was + /// still renegotiating, the wait returned immediately, and the "Connecting…" + /// pill flashed past instead of holding. /// /// The power term is a *device is delivering real samples* test, not a /// voice-activity one — see `silenceFloorDB`. Short-circuited, so a recorder - /// whose clock hasn't moved is never metered. + /// that hasn't delivered anything yet is never metered. /// /// Returns the elapsed wait once the input is live, or nil when `timeout` (or /// a task cancellation) won the race. Nil is pure information — this function @@ -141,7 +145,7 @@ enum MicLiveness { static func waitUntilLive( timeout: Duration, clock: some Clock, - currentTime: @escaping @Sendable () -> TimeInterval, + deliveredFrames: @escaping @Sendable () -> Int, inputPowerDB: @escaping @Sendable () -> Float ) async -> Duration? { let start = clock.now @@ -149,7 +153,7 @@ enum MicLiveness { var interval = initialPollInterval // Negated `>` rather than `<=` on purpose: a NaN reading fails both // comparisons, and this spelling makes that "not live" instead of "live". - while currentTime() <= 0 || !(inputPowerDB() > silenceFloorDB) { + while deliveredFrames() <= 0 || !(inputPowerDB() > silenceFloorDB) { guard clock.now < deadline, !Task.isCancelled else { return nil } try? await clock.sleep(for: interval) interval = min(interval * 2, maxPollInterval) diff --git a/Sources/BlurtEngine/Config/DefaultsKey.swift b/Sources/BlurtEngine/Config/DefaultsKey.swift index 134583ad..58006c80 100644 --- a/Sources/BlurtEngine/Config/DefaultsKey.swift +++ b/Sources/BlurtEngine/Config/DefaultsKey.swift @@ -42,6 +42,9 @@ enum DefaultsKey: String, CaseIterable { case overlayCustomOriginX = "OverlayCustomOriginX" case overlayCustomOriginY = "OverlayCustomOriginY" case lastUpdateCheck = "LastUpdateCheck" + /// The CoreAudio UID of the input device dictation is pinned to, or empty for + /// "same as system" (`MicDeviceStore`). + case micDeviceUID = "MicDeviceUID" /// The key this case actually reads and writes, under the configured host /// identity. A computed property rather than a stored string because the diff --git a/Sources/BlurtEngine/Pipeline/DictationSession.swift b/Sources/BlurtEngine/Pipeline/DictationSession.swift index 85a0d525..4312b6d3 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession.swift @@ -260,8 +260,8 @@ public actor DictationSession { guard phase == .recording else { return } cancelAutoRelease() // Flip the phase before stopping the mic, not after: the stop chime and - // the pill's "Transcribing…" ride this transition, and mic.stop() reads - // the whole recording back from disk — I/O the user's "it heard me" cue + // the pill's "Transcribing…" ride this transition, and mic.stop() waits out + // the Bluetooth tail linger — up to 220 ms the user's "it heard me" cue // must not wait on. This also closes the double-release window: a second // release arriving during the mic.stop() suspension now fails the // `.recording` guard above instead of running the pipeline twice. @@ -316,9 +316,9 @@ public actor DictationSession { func stopAndCancel() async { cancelAutoRelease() do { - // `cancelCapture`, not `stop`: the audio is being thrown away, so neither - // preserving it (the Bluetooth tail linger) nor reading it back off disk - // is worth delaying the user's cancel for. + // `cancelCapture`, not `stop`: the audio is being thrown away, so + // preserving it (the Bluetooth tail linger) is not worth delaying the + // user's cancel for. try await mic.cancelCapture() } catch { // Stays out of the UI: the user asked for nothing to happen, and a cancel diff --git a/Sources/BlurtEngine/README.md b/Sources/BlurtEngine/README.md index 132eac14..e0d8a613 100644 --- a/Sources/BlurtEngine/README.md +++ b/Sources/BlurtEngine/README.md @@ -74,7 +74,7 @@ Key properties of the design, which your integration can rely on: - **One request per utterance, no streaming.** The dictation API returns the complete transcript — and its LLM-rewritten form — in the response body: no upload step, no job polling, no incremental deltas, no second request for the cleanup. `TranscriberProtocol.transcribe` is a single `async throws -> String`. UIs should show a "transcribing…" state and then the whole result; there is nothing to stream. - **Cleanup happens server-side, and it's optional.** The request's `llm` block asks the service to apply our own cleanup instruction (`CleanupInstruction.text` — delete disfluencies, change nothing else) to the verbatim transcript inside the same call. It is one of three steering fields on the request: `config.conversation_context` primes the _transcription_ with the dialogue that came before — the user's recent dictations, then the text before the cursor (`ConversationContext`, omitted when there is neither) — and `config.word_boost` boosts the user's key terms (`KeytermsBoost`, omitted when there are none). There is no `config.prompt`; the context field replaced it. The block is gated by the **enhanced transcripts** setting (`EnhancedTranscriptsStore`, on by default): turned off, the config omits `llm` and the verbatim transcript is pasted as spoken. The **active style profile**'s instructions (`StyleProfileStore`, none by default) are appended to that instruction via `CleanupInstruction.sendable(appending:)`, trimmed to the headroom the API's 2048 instruction cap leaves (measured in UTF-8 bytes, the conservative bound — the cap's own unit is unmeasured); no profile, or blank instructions, means the base instruction goes out unchanged. Only the **active** profile's text is ever sent — never a join of the user's profiles, which would blow that cap and 400 the whole request rather than degrading. The engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error. There is no client-side LLM pass, no styling stage, and deliberately no hook for one. -- **Latency is pre-paid where possible, and never faked.** `press()` claims `.connecting` before it touches the mic, so a host's pill answers the keypress — but `MicCapture.start()` deliberately holds until the input device is actually delivering frames, and the _start cue_ waits for `.recording`. On a Bluetooth route those are ~1–2 s apart and the OS captures nothing in between, so cueing at the press loses the first words. `MicCapture` re-arms its prepared recorder after every capture so that route activation is usually paid between dictations rather than during one. `press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection while the user speaks, ~170 ms saved cold) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall — and never delays the recording indicator. On the way out, `release()` flips the phase to `.transcribing` _before_ reading the recorded audio back, so a host's stop cue fires at key-up rather than after the disk read. +- **Latency is pre-paid where possible, and never faked.** `press()` claims `.connecting` before it touches the mic, so a host's pill answers the keypress — but `MicCapture.start()` deliberately holds until the input device is actually delivering frames, and the _start cue_ waits for `.recording`. On a Bluetooth route those are ~1–2 s apart and the OS captures nothing in between, so cueing at the press loses the first words. Route activation itself cannot be pre-paid — measured, not assumed: nothing opens the device before `record()`, so the connecting window is where that cost lives and the gate is what makes it honest. `press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection while the user speaks, ~170 ms saved cold) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall — and never delays the recording indicator. On the way out, `release()` flips the phase to `.transcribing` _before_ reading the recorded audio back, so a host's stop cue fires at key-up rather than after the disk read. - **A held trigger auto-releases.** `DictationSession` stops recording after `maxRecordingSeconds` (default `SyncSTTLimits.autoReleaseSeconds`, 115 s) so audio never exceeds what the endpoint accepts, and transcribes what it has. Clips shorter than `SyncSTTLimits.minPCMBytes` (~100 ms of audio — an accidental tap) are dropped as a silent no-op rather than sent to earn a 400. ## DictationSession @@ -161,22 +161,22 @@ func start() async throws func stop() async throws -> Data // raw S16LE mono PCM, 16 kHz, in order func cancelCapture() async throws // stop and discard; default: stop-and-drop var levels: AsyncStream { get } // 0…1 meter; default: empty stream -func warmUp() async // pre-open the device; default: no-op +func warmUp() async // pay set-up early, never open the device; default: no-op ``` -Only `start()`/`stop()` must be implemented — `cancelCapture()`, `levels` and `warmUp()` have defaults, so a stub or headless capture conforms for free while hosts still read the meter and warm the device through the same seam they inject. +Only `start()`/`stop()` must be implemented — `cancelCapture()`, `levels` and `warmUp()` have defaults, so a stub or headless capture conforms for free while hosts still read the meter and warm the capture stack through the same seam they inject. `cancelCapture()` is what the session calls when a dictation is cancelled, and it exists because the two teardowns want opposite things: `stop()` may legitimately spend time preserving the audio, while a cancel has nothing to preserve and must take effect at once. Override it only if stopping cheaply differs from stopping carefully in your capture. -`MicCapture` records with `AVAudioRecorder` straight to a temp 16 kHz / mono / 16-bit PCM WAV — exactly the geometry the dictation API wants — and reads it back as raw S16LE bytes on `stop()` (no float detour; the blob uploads as-is). A **fresh recorder per session** resolves the current default input device at `record()` time, which is why device switches (headset ↔ built-in) just work. Do **not** replace this with a long-lived `AVAudioEngine`/`installTap` graph: that design was tried, bound itself to one device, and failed with `-10868` or all-zero buffers on device switches. +`MicCapture` records through a per-session `AVCaptureSession` (`CaptureSessionRecorder`, used directly — the owner-directed move from `AVAudioRecorder`, 2026-08-25) whose data output converts to 16 kHz / mono / 16-bit LPCM on the fly — exactly the geometry the dictation API wants — accumulating the raw S16LE bytes in memory so `stop()` returns them as-is (no temp file, no float detour). A **fresh recorder per session** is built around the current resolution of the host's microphone selection at press time — the device pinned via `MicDeviceStore`, or the system default — which is why device switches (headset ↔ built-in) just work, with a per-press fallback to the default while a pinned device isn't connected. Do **not** replace this with a long-lived `AVAudioEngine`/`installTap` graph: that design was tried, bound itself to one device, and failed with `-10868` or all-zero buffers on device switches. -`MicCapture`'s `levels` is a ~20 Hz meter of the recorder's dBFS power mapped to `0…1` (floored at −50 dBFS so room ambient reads as silence) — feed it to a voice-bars view; it costs nothing when unobserved. Its `warmUp()` pre-creates and prepares a recorder so the first `start()` skips hardware route discovery (Blurt calls it at launch, once mic permission is granted, so warming never triggers the permission prompt). +`MicCapture`'s `levels` is a ~20 Hz meter of the recorder's dBFS power mapped to `0…1` (floored at −50 dBFS so room ambient reads as silence) — feed it to a voice-bars view; it costs nothing when unobserved. Its `warmUp()` builds one session and drops it, which absorbs the ~75 ms a process pays the first time it touches AVFoundation's capture stack; it holds no recorder and never engages the microphone (no input indicator, verified against `kAudioDevicePropertyDeviceIsRunningSomewhere`). It cannot do more than that: the device only opens at `record()`, so the 180–600 ms route activation is unavoidably inside the connecting window the liveness gate covers. Blurt calls `warmUp()` at launch, once mic permission is granted, so warming never triggers the permission prompt. -**Bluetooth inputs get four accommodations**, because opening the mic on AirPods makes the system renegotiate the link into its mic-capable mode — one to two seconds, during which the OS receives no audio at all — and that link then buffers audio. +**Bluetooth inputs get two accommodations**, because opening the mic on AirPods makes the system renegotiate the link into its mic-capable mode — one to two seconds, during which the OS receives no audio at all — and that link then buffers audio. -The load-bearing one is that **`start()` doesn't return until the input is live.** `record()` returning `true` only means the AudioQueue started, so `start()` polls the recorder until its clock has advanced past 0 **and** its meter reads above `MicLiveness.silenceFloorDB` — the clock alone advances over the all-zero buffers a stale or not-yet-switched device delivers, so it confirmed a route that was still renegotiating. The floor separates _digital_ silence from any real analog input, not speech from quiet. The wait is capped per transport by `MicLiveness` (2.5 s Bluetooth, 300 ms on a recognised local transport, 1 s for an aggregate/virtual/unreadable one) and **fails closed** on timeout: `start()` tears the recorder down and throws `.audioCaptureFailed`, which the session surfaces as the same error pill as any other capture failure — never a recording the mic wasn't delivering for. Speech during the switch is not recovered by this — nothing receives it — the point is to stop inviting it. +The load-bearing one is that **`start()` doesn't return until the input is live.** `record()` returning `true` only means the session started running, so `start()` polls the recorder until it has delivered at least one frame **and** its meter reads above `MicLiveness.silenceFloorDB` — frame arrival alone counts the all-zero buffers a stale or not-yet-switched device delivers, so it confirmed a route that was still renegotiating. The floor separates _digital_ silence from any real analog input, not speech from quiet. The wait is capped per transport by `MicLiveness` (2.5 s Bluetooth, 300 ms on a recognised local transport, 1 s for an aggregate/virtual/unreadable one) and **fails closed** on timeout: `start()` tears the recorder down and throws `.audioCaptureFailed`, which the session surfaces as the same error pill as any other capture failure — never a recording the mic wasn't delivering for. Speech during the switch is not recovered by this — nothing receives it — the point is to stop inviting it. -The other three reduce how often that wait is paid, and fix the tail. `MicCapture` re-arms the warm recorder after _every_ capture rather than only at launch (the cost is per session, paid at `prepareToRecord()`); it records which device the warm recorder was built against and discards it when the default input has changed, since `AVAudioRecorder` resolves its device once and never re-resolves; and it releases an unused warm recorder after 60 s, because holding the input device open is what pins AirPods in the profile where output audio is degraded. On top of that, `stop()` keeps capturing for a further 220 ms when the input is Bluetooth, so speech still travelling over the link lands in the file instead of being truncated — the missing last word. `cancelCapture()` skips both that linger and the file read-back. +The other one fixes the tail: `stop()` keeps capturing for a further 220 ms when the input is Bluetooth, so speech still travelling over the link lands in the recording instead of being truncated — the missing last word. `cancelCapture()` skips that linger. ### `TranscriberProtocol` → `AssemblyAITranscriber` diff --git a/Sources/BlurtEngine/STT/SyncSTTLimits.swift b/Sources/BlurtEngine/STT/SyncSTTLimits.swift index b933c2da..ea856957 100644 --- a/Sources/BlurtEngine/STT/SyncSTTLimits.swift +++ b/Sources/BlurtEngine/STT/SyncSTTLimits.swift @@ -31,6 +31,19 @@ public enum SyncSTTLimits { /// audio with `minPCMBytes`; only the capture/upload code needs the factor. static let bytesPerSample = 2 + /// Channels in that geometry. Mono, and the capture side has to agree: every + /// byte-count-to-duration conversion here assumes one channel, so a stereo + /// recorder would report half the duration it captured. + static let channelCount = 1 + + /// Bit depth of that geometry, derived from `bytesPerSample` rather than + /// restated. `CaptureSessionRecorder.audioSettings` asks for these three + /// values instead of spelling 16 and 1 as literals: capture geometry and the + /// byte math above are one contract, and they used to be two unlinked + /// definitions with nothing failing if they drifted (`MicCaptureFormatTests` + /// pins the link). + static let bitDepth = bytesPerSample * 8 + /// The fewest PCM bytes worth sending: `minSamples` expressed in the raw /// S16LE encoding the pipeline captures and uploads — the floor /// `DictationSession` applies to the blob `MicCaptureProtocol.stop()` returns. diff --git a/Tests/BlurtEngineTests/AudioInputDevicesTests.swift b/Tests/BlurtEngineTests/AudioInputDevicesTests.swift new file mode 100644 index 00000000..73e3dfb6 --- /dev/null +++ b/Tests/BlurtEngineTests/AudioInputDevicesTests.swift @@ -0,0 +1,185 @@ +@preconcurrency import AVFoundation +import CoreAudio +import Foundation +import Testing + +@testable import BlurtEngine + +/// Live-hardware checks for the microphone-selection plumbing: the device +/// enumeration the Settings picker lists, the UID lookups the capture path +/// resolves a pin with, and the session recorder itself. +/// +/// Gated on BLURT_LIVE_AUDIO_TESTS=1 like the other capture suites — every test +/// here talks to real devices (and two of them open one), which a headless CI +/// runner cannot answer for. Documents and locks the behavior for a human +/// running it on a Mac with a microphone; `AudioInputDevices.swift` and +/// `CaptureSessionRecorder.swift` are excluded from the coverage gate for the +/// same reason `MicCapture.swift` is. +@Suite( + "AudioInputDevices & capture session (live)", + ConditionTrait.requiresLiveAudio, + .tags(.liveAudio), + // Serialized: three of these open the default input, and the engagement test + // reads whether *anyone* has it open. Run in parallel they'd flag each other. + .serialized, + .timeLimit(.minutes(1))) +struct AudioInputDevicesTests { + @Test("enumeration lists named, uniquely-identified input devices") + func enumerationListsInputDevices() throws { + let devices = AudioInputDevices.all() + try #require(!devices.isEmpty, "a machine running this suite needs at least one input device") + + // Every entry must be renderable in the picker and pinnable by the store. + // Closures rather than key paths inside the macros — the rethrows + key-path + // combination is the documented `#expect` trap (see AGENTS.md's testing notes). + #expect(devices.allSatisfy { !$0.uid.isEmpty && !$0.name.isEmpty }) + let uids = devices.map { $0.uid } + #expect(Set(uids).count == uids.count, "device UIDs must be unique") + + // The default input has a name for the "Same as system (…)" label. + #expect(AudioInputDevices.systemDefaultInputName() != nil) + } + + @Test("a listed UID classifies; a bogus one reads as absent") + func uidLookupsAgreeWithTheEnumeration() throws { + let first = try #require(AudioInputDevices.all().first) + + // One read answers both questions `MicCapture.resolveInput` asks of a pin — + // is the device there, and what transport is it on. That it holds for every + // enumerated device is the load-bearing part: the picker must not offer a + // device the capture path would then treat as missing (or fail to classify, + // which silently loses the Bluetooth caps). + #expect(AudioInputDevices.transportType(forUID: first.uid) != nil) + + // The missing-device signal `MicDeviceSelection.effective` falls back on. + #expect(AudioInputDevices.transportType(forUID: "blurt-test-no-such-device") == nil) + } + + @Test("building a recorder — and warming up — leaves the microphone closed") + func buildingASessionDoesNotEngageTheDevice() async throws { + await LiveAudioDevice.acquire() + defer { LiveAudioDevice.release() } + + // The measurement the warm-up design rests on, as a regression test: a built + // but unstarted session must not open the device, or `warmUp()` at launch + // would light the input indicator and pin AirPods into their mic-capable + // profile (where *output* audio is degraded) for as long as the app runs. + let device = try #require(AVCaptureDevice.default(for: .audio)) + // Wait for the device to be free rather than demanding it already is: this + // suite is `.serialized`, but the other live suites are separate suites and + // run in parallel with it, and two of them open this same device. Polling + // makes the precondition deterministic instead of dependent on which suite + // got there first; a device that never frees up is a real failure with a + // message that says so. + try #require( + await Self.poll(upTo: .seconds(3)) { Self.isRunningSomewhere(uid: device.uniqueID) == false }, + "the default input stayed busy — another app (or suite) is capturing from it") + + let recorder = try await CaptureSessionRecorder.make(pinnedUID: device.uniqueID) + #expect(Self.isRunningSomewhere(uid: device.uniqueID) == false, "building must not open the device") + + await MicCapture(deviceSelection: { .systemDefault }).warmUp() + #expect(Self.isRunningSomewhere(uid: device.uniqueID) == false, "warm-up must not open the device") + + // And the same recorder still opens it on demand, so "closed" isn't just a + // recorder that was never usable. + // + // Polled rather than asserted outright, because when the open *completes* + // is transport-dependent — the measurement that shaped the liveness gate. + // On a USB interface `startRunning()` blocks until the device is running, so + // this holds the instant `record()` returns; on AirPods it returns in ~80 ms + // and the device comes up ~400 ms later. Asserting immediately passes on + // wired hardware and fails on the transport the gate exists for. + try #require(await recorder.record()) + #expect( + await Self.waitForRunning(uid: device.uniqueID), + "record() is what opens the device, even if the link finishes opening it later") + recorder.stopAndDiscard() + } + + @Test("the session recorder captures S16LE audio from the device it was pinned to") + func pinnedRecorderCapturesAudio() async throws { + await LiveAudioDevice.acquire() + defer { LiveAudioDevice.release() } + + // Pin to the current default input — the one device a machine running this + // suite is known to have working — by the same UID the picker would store. + let uid = try #require(AVCaptureDevice.default(for: .audio)?.uniqueID) + #expect(AudioInputDevices.all().contains { $0.uid == uid }, "the default input should be listed") + + let recorder = try await CaptureSessionRecorder.make(pinnedUID: uid) + try #require(await recorder.record(), "the session recorder should start on a live device") + + // Wait for frames the way the liveness gate does, rather than sleeping a + // fixed 500 ms: on AirPods the first buffer lands ~490 ms after + // `startRunning()` returns, so a fixed sleep sits right on the edge and this + // suite would flake on a cold link. + try #require( + await Self.poll(upTo: MicLiveness.bluetoothTimeout) { recorder.deliveredFrames > 0 }, + "buffers should be arriving") + // The meter must read as a live analog input, not digital silence — this is + // the reading `MicLiveness.silenceFloorDB` gates on, and the assertion that + // proves the session meter's floor semantics on real hardware. + // + // Polled for the same reason the frame wait is, and this is the sharper + // case: the channel's power lags the first frames, reporting + // `-Float.greatestFiniteMagnitude` until its first update (measured on + // AirPods). That is exactly why the gate polls *both* terms instead of + // metering once frames appear. + #expect( + await Self.poll(upTo: MicLiveness.bluetoothTimeout) { + recorder.meteredPowerDB() > MicLiveness.silenceFloorDB + }, "a live mic must out-read the silence floor") + + let pcm = recorder.stopAndReadPCM() + #expect(!pcm.isEmpty, "half a second of capture should deliver samples") + #expect(pcm.count % SyncSTTLimits.bytesPerSample == 0, "the blob must be whole S16LE samples") + } + + /// Polls `condition` on a short tick until it holds or `timeout` elapses, + /// answering whether it ever held. The suite's own miniature of + /// `MicLiveness.waitUntilLive`, for the same reason that exists: nothing about + /// a capture device is synchronous just because the call that started it + /// returned. + private static func poll( + upTo timeout: Duration, _ condition: @Sendable () -> Bool + ) async -> Bool { + let deadline = ContinuousClock().now.advanced(by: timeout) + while ContinuousClock().now < deadline { + if condition() { return true } + try? await Task.sleep(for: .milliseconds(20)) + } + return condition() + } + + /// Whether the device carrying `uid` comes up as capturing within the + /// Bluetooth cap — the widest the product itself ever waits. + private static func waitForRunning(uid: String) async -> Bool { + await poll(upTo: MicLiveness.bluetoothTimeout) { isRunningSomewhere(uid: uid) } + } + + /// CoreAudio's own answer to "is this device capturing right now" — the bit + /// behind the system input indicator. Test-local rather than a production + /// helper: nothing in the app needs to ask, and the point of the assertions + /// above is to check the *device*, not our bookkeeping about it. + private static func isRunningSomewhere(uid: String) -> Bool { + var translate = AudioRoute.globalAddress(kAudioHardwarePropertyTranslateUIDToDevice) + var cfUID = uid as CFString + var deviceID = AudioDeviceID(0) + var idSize = UInt32(MemoryLayout.size) + let translated = withUnsafeMutablePointer(to: &cfUID) { qualifier in + AudioObjectGetPropertyData( + AudioRoute.systemObject, &translate, UInt32(MemoryLayout.size), qualifier, + &idSize, &deviceID) + } + guard translated == noErr, deviceID != 0 else { return false } + + var address = AudioRoute.globalAddress(kAudioDevicePropertyDeviceIsRunningSomewhere) + var running = UInt32(0) + var size = UInt32(MemoryLayout.size) + guard AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, &running) == noErr else { + return false + } + return running != 0 + } +} diff --git a/Tests/BlurtEngineTests/LiveAudioSupport.swift b/Tests/BlurtEngineTests/LiveAudioSupport.swift new file mode 100644 index 00000000..99326658 --- /dev/null +++ b/Tests/BlurtEngineTests/LiveAudioSupport.swift @@ -0,0 +1,58 @@ +import Foundation +import Synchronization +import Testing + +extension Tag { + /// Marks tests that drive a real `AVCaptureSession` and the system mic. They + /// only run when BLURT_LIVE_AUDIO_TESTS=1; the tag lets a run include/exclude + /// them as a group (e.g. `--filter-tag liveAudio`). + @Tag static var liveAudio: Self +} + +extension ConditionTrait { + /// The BLURT_LIVE_AUDIO_TESTS gate, in one place. Three suites need it, and it + /// was three copies of the same condition *and* the same skip message — a + /// rename of the variable was a three-file edit with nothing to catch a miss. + /// + /// `.enabled(if:)` rather than an in-body `guard … else { return }` so a normal + /// run reports these as *skipped* instead of a silent pass: the skip is + /// visible, and no one mistakes "didn't run" for "passed". + static var requiresLiveAudio: Self { + .enabled( + if: ProcessInfo.processInfo.environment["BLURT_LIVE_AUDIO_TESTS"] == "1", + "set BLURT_LIVE_AUDIO_TESTS=1 to run (needs a real microphone)") + } +} + +/// Exclusive use of the system microphone, across suites. +/// +/// `.serialized` orders tests *within* a suite, and the live suites are separate +/// suites — so they run concurrently, and three of them open the input device. +/// Mostly that is harmless (CoreAudio allows several clients on one input), but +/// `AudioInputDevicesTests` asserts on `kAudioDevicePropertyDeviceIsRunningSomewhere`, +/// which answers for the *device* rather than for our client: another suite's +/// capture reads exactly like a failure of "building must not open the device". +/// It failed that way on two runs out of two before this existed. +/// +/// Not an `actor`: the bodies these tests want to protect capture non-`Sendable` +/// AVFoundation objects, so a `withLock`-style async closure would fight strict +/// concurrency for no benefit. Acquire/release around the hardware section +/// instead — the callers are a handful of tests, and `defer` keeps it honest. +enum LiveAudioDevice { + private static let held = Mutex(false) + + /// Waits until no other live test holds the microphone, then claims it. + static func acquire() async { + while !held.withLock({ claimed -> Bool in + guard !claimed else { return false } + claimed = true + return true + }) { + try? await Task.sleep(for: .milliseconds(20)) + } + } + + static func release() { + held.withLock { $0 = false } + } +} diff --git a/Tests/BlurtEngineTests/MicCaptureBringUpTests.swift b/Tests/BlurtEngineTests/MicCaptureBringUpTests.swift new file mode 100644 index 00000000..2852b89a --- /dev/null +++ b/Tests/BlurtEngineTests/MicCaptureBringUpTests.swift @@ -0,0 +1,100 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +/// The bring-up race: a teardown arriving while the input device is still +/// opening must be serviced, not queued behind the open. +/// +/// `record()` opens the device, and how long that takes is the hardware's +/// business — ~100 ms on AirPods, ~180 ms on the built-in mic, ~600 ms on a USB +/// interface. Run inline on the actor it blocked every other call for that whole +/// window (measured: 578 ms on a 635 ms open); off-actor the same teardown takes +/// microseconds. Hence the absolute budget below rather than a ratio against a +/// calibration run: an unblocked `cancelCapture()` is one actor hop and a nil +/// check either way, so the budget holds on every input, and a regression fails +/// it on any device whose open outlasts the budget — which is all of them. +@Suite( + "MicCapture bring-up (live)", + ConditionTrait.requiresLiveAudio, + .tags(.liveAudio), + .serialized, + .timeLimit(.minutes(1))) +struct MicCaptureBringUpTests { + /// Generous next to the ~24 µs an unblocked teardown measures, and far below + /// the tens-to-hundreds of ms a blocked one costs on any real input. + private static let teardownBudget = Duration.milliseconds(10) + + @Test("a teardown during the device open isn't blocked by it") + func teardownDuringOpenIsNotBlocked() async throws { + await LiveAudioDevice.acquire() + defer { LiveAudioDevice.release() } + + // An unqueued teardown mid-bring-up — the shape a host that doesn't + // serialize its own commands produces. `DictationSession` does serialize + // (its release and cancel run only after the press turn completes), but this + // is a public actor and the contract has to hold without that. + let mic = MicCapture(deviceSelection: { .systemDefault }) + + // Warm first, exactly as the app does at launch, so what is measured below + // is the property under test and not this process's first touch of the + // capture stack. That first touch is ~185 ms — most of it the first device + // query, which `start()` makes inline while resolving the selection — and + // absorbing it is `warmUp()`'s entire job. Without this the measurement + // reports a real cost that a launched app has already paid. + await mic.warmUp() + + // An unstructured `Task` so the press stays in flight while this function + // cancels it and then inspects how it ended. + let press = Task { try await mic.start() } + // 30 ms is a bracket, not a guarantee: it has to clear the ~5 ms session + // build (cancelling before the open is reached would pass even if `record()` + // went back to blocking the actor) and still land inside the open, whose + // fastest measurement here is ~80 ms. Both ends matter, so the outcome is + // checked below rather than assumed. + try await Task.sleep(for: .milliseconds(30)) + + let cancelCost = await ContinuousClock().measure { + try? await mic.cancelCapture() + } + + // Where the cancel actually landed, straight from the press: an abandoned + // bring-up throws `CancellationError` from one of the three + // `checkStillWanted` points instead of installing its recorder. + var pressFailure: Error? + do { + try await press.value + } catch { + pressFailure = error + } + let abandonedMidBringUp = pressFailure is CancellationError + + // An unrelated capture failure is its own report, not evidence about the + // bracket — otherwise a mic that never delivers reads as "too fast". + if let pressFailure, !abandonedMidBringUp { + Issue.record("start() failed for an unrelated reason: \(pressFailure)") + } + + // Judged before the budget, because the two are different failures. If the + // bring-up finished inside the wait — a virtual or aggregate input that + // opens *and* delivers non-silent audio in under 30 ms — then + // `cancelCapture()` took the installed path, whose inline `stopRunning()` is + // documented at 19–41 ms, and asserting the teardown budget on that reports + // "the actor was blocked" about an actor that was never blocked. Naming the + // bracket separately is what stops one failure wearing the other's message. + #expect( + abandonedMidBringUp, + """ + the bring-up completed inside the 30 ms wait, so the teardown below measured an installed \ + recorder rather than one mid-open — this input is too fast for a timing bracket, and pinning \ + the property on it wants an injectable recorder rather than a longer sleep + """) + + if abandonedMidBringUp { + #expect( + cancelCost < Self.teardownBudget, + "cancelCapture took \(cancelCost) while the device was still opening — the actor was blocked" + ) + } + } +} diff --git a/Tests/BlurtEngineTests/MicCaptureFormatTests.swift b/Tests/BlurtEngineTests/MicCaptureFormatTests.swift index 17258a51..a7f3acd6 100644 --- a/Tests/BlurtEngineTests/MicCaptureFormatTests.swift +++ b/Tests/BlurtEngineTests/MicCaptureFormatTests.swift @@ -1,14 +1,42 @@ -@preconcurrency import AVFoundation +import AVFoundation import Foundation import Testing @testable import BlurtEngine -/// Pure-logic tests for `MicCapture`'s level-metering math and recorded-file -/// decoding. The recorder capture lifecycle itself needs a real device and is -/// exercised by the env-gated `MicCaptureLevelsTests`. -@Suite("MicCapture decoding & metering") +/// Pure-logic tests for `MicCapture`'s level-metering math and the capture +/// errors' user-facing wording. The recorder capture lifecycle itself needs a +/// real device and is exercised by the env-gated `MicCaptureLevelsTests` and +/// `AudioInputDevicesTests`. +@Suite("MicCapture metering & errors") struct MicCaptureFormatTests { + // MARK: - Capture geometry + + @Test("the geometry the recorder asks for is the one the upload math assumes") + func captureGeometryMatchesUploadMath() throws { + // These used to be two unlinked definitions: six literals in the recorder, + // and `SyncSTTLimits.bytesPerSample`/`durationMs` independently assuming + // mono 16-bit. Nothing failed if they drifted, and the drift is silent — + // a stereo or 8-bit recorder halves or doubles every duration the pipeline + // reports. The recorder now derives all three from `SyncSTTLimits`; this + // pins that it still does, from the covered side of the coverage gate. + let settings = CaptureSessionRecorder.audioSettings() + + #expect(settings[AVFormatIDKey] as? AudioFormatID == kAudioFormatLinearPCM) + #expect(settings[AVSampleRateKey] as? Double == Double(SyncSTTLimits.sampleRate)) + #expect(settings[AVNumberOfChannelsKey] as? Int == 1, "durationMs assumes one channel") + + let bitDepth = try #require(settings[AVLinearPCMBitDepthKey] as? Int) + #expect( + bitDepth / 8 == SyncSTTLimits.bytesPerSample, + "bit depth and bytesPerSample describe the same sample") + + // S16LE specifically: int, not float, and little-endian — the encoding the + // dictation request uploads byte for byte. + #expect(settings[AVLinearPCMIsFloatKey] as? Bool == false) + #expect(settings[AVLinearPCMIsBigEndianKey] as? Bool == false) + } + // MARK: - dBFS → linear level @Test func fullScalePowerMapsToOne() { @@ -34,30 +62,7 @@ struct MicCaptureFormatTests { #expect(MicCapture.linearLevel(fromPowerDB: -11) > MicCapture.linearLevel(fromPowerDB: -22)) } - // MARK: - PCM file decoding - - @Test func decodePCMRoundTripsRecordedAudio() throws { - let known: [Float] = [0, 0.5, -0.5, 1.0, -1.0, 0.25, -0.25, 0] - let url = try Self.writeWAV(samples: known) - defer { try? FileManager.default.removeItem(at: url) } - - let pcm = try MicCapture.decodePCM(fromFileAt: url) - - // Two bytes per sample, no RIFF/WAVE header — the blob uploads as-is. - #expect(pcm.count == known.count * 2) - for (i, want) in known.enumerated() { - let got = Float(Self.readInt16LE(pcm, i * 2)) / 32_767 - // int16 quantization tolerance (full scale is ±32767, ~3e-5 per step; - // the write side's ±32768-vs-±32767 scaling convention also fits inside). - #expect(abs(got - want) < 0.001) - } - } - - @Test func decodePCMReturnsEmptyForEmptyFile() throws { - let url = try Self.writeWAV(samples: []) - defer { try? FileManager.default.removeItem(at: url) } - #expect(try MicCapture.decodePCM(fromFileAt: url).isEmpty) - } + // MARK: - Error wording @Test func noInputDeviceHasHumanReadableMessage() { // This message reaches the overlay via BlurtError.audioCaptureFailed's @@ -72,45 +77,4 @@ struct MicCaptureFormatTests { // also keep MicCaptureError.swift's errorDescription fully covered. #expect(MicCaptureError.inputNeverDelivered.errorDescription == "The microphone didn't start.") } - - // MARK: - Helpers - - /// Little-endian Int16 at `offset` — decodes the raw S16LE blob under test. - private static func readInt16LE(_ data: Data, _ offset: Int) -> Int16 { - Int16(bitPattern: UInt16(data[offset]) | (UInt16(data[offset + 1]) << 8)) - } - - /// Write the given mono samples to a temp 16 kHz / 16-bit PCM WAV — the same - /// on-disk format `MicCapture` records — and return its URL. The file is closed - /// (flushed) before returning so `decodePCM` reads a complete file. - private static func writeWAV(samples: [Float]) throws -> URL { - let url = FileManager.default.temporaryDirectory - .appendingPathComponent("blurt-test-\(UUID().uuidString).wav") - let settings: [String: Any] = [ - AVFormatIDKey: kAudioFormatLinearPCM, - // The engine's rate, not a restated literal: this helper claims to write - // "the same on-disk format MicCapture records", and MicCapture derives its - // targetSampleRate from here too. Pinned independently, a rate change would - // leave decodePCM's round trip exercising a format the recorder never makes, - // with the suite still green. (SyncSTTLimitsTests pins the value itself.) - AVSampleRateKey: Double(SyncSTTLimits.sampleRate), - AVNumberOfChannelsKey: 1, - AVLinearPCMBitDepthKey: 16, - AVLinearPCMIsFloatKey: false, - AVLinearPCMIsBigEndianKey: false, - ] - // Scoped so the AVAudioFile is released (and the file flushed) before we read. - try { - let file = try AVAudioFile(forWriting: url, settings: settings) - guard !samples.isEmpty else { return } - let buffer = try #require( - AVAudioPCMBuffer( - pcmFormat: file.processingFormat, frameCapacity: AVAudioFrameCount(samples.count))) - buffer.frameLength = AVAudioFrameCount(samples.count) - let channel = try #require(buffer.floatChannelData) - for (i, sample) in samples.enumerated() { channel[0][i] = sample } - try file.write(from: buffer) - }() - return url - } } diff --git a/Tests/BlurtEngineTests/MicCaptureLevelsTests.swift b/Tests/BlurtEngineTests/MicCaptureLevelsTests.swift index 07fc5c07..ec85f586 100644 --- a/Tests/BlurtEngineTests/MicCaptureLevelsTests.swift +++ b/Tests/BlurtEngineTests/MicCaptureLevelsTests.swift @@ -3,30 +3,24 @@ import Testing @testable import BlurtEngine -extension Tag { - /// Marks tests that drive the real AVAudioEngine and the system mic. They only - /// run when BLURT_LIVE_AUDIO_TESTS=1; the tag lets a run include/exclude them - /// as a group (e.g. `--filter-tag liveAudio`). - @Tag static var liveAudio: Self -} - -/// Hits the real AVAudioEngine and the system mic, so it's gated on -/// BLURT_LIVE_AUDIO_TESTS=1 (set it in the scheme to enable). Using -/// `.enabled(if:)` rather than an in-body `guard … else { return }` means a -/// normal run reports this as *skipped* instead of a silent pass — the skip is -/// visible, so no one mistakes "didn't run" for "passed". `.timeLimit` fails fast -/// if the capture hangs instead of stalling the whole run. +/// Hits a real `AVCaptureSession` and the system mic, so it rides the +/// `requiresLiveAudio` gate (set BLURT_LIVE_AUDIO_TESTS=1 in the scheme to +/// enable). `.timeLimit` fails fast if the capture hangs instead of stalling the +/// whole run. @Suite("MicCapture.levels (live)") struct MicCaptureLevelsTests { @Test( "levels yield during capture", - .enabled( - if: ProcessInfo.processInfo.environment["BLURT_LIVE_AUDIO_TESTS"] == "1", - "set BLURT_LIVE_AUDIO_TESTS=1 to run (needs a real microphone)"), + ConditionTrait.requiresLiveAudio, .tags(.liveAudio), .timeLimit(.minutes(1))) func levelsYieldDuringCapture() async throws { - let mic = MicCapture() + // See `LiveAudioDevice`: the suites that open the mic take turns, so the one + // asserting on the device's own "is anyone capturing" bit isn't reading ours. + await LiveAudioDevice.acquire() + defer { LiveAudioDevice.release() } + + let mic = MicCapture(deviceSelection: { .systemDefault }) let collector = Task { () -> [Float] in var collected: [Float] = [] diff --git a/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift b/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift index e9884469..3c182ff1 100644 --- a/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift +++ b/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift @@ -46,8 +46,8 @@ struct MicCaptureProtocolDefaultsTests { // A capture with no cancel-specific teardown must still *end* on a cancel — // the default is stop-and-discard, so a conformance that never heard of // `cancelCapture` keeps the behavior it had when the session called `stop()` - // directly. (`MicCapture` overrides it to skip the tail linger and the - // read-back; that path needs real hardware, so it isn't covered here.) + // directly. (`MicCapture` overrides it to skip the tail linger; that path + // needs real hardware, so it isn't covered here.) let mic = BareMic() try await mic.cancelCapture() let stops = mic.stops.withLock { $0 } diff --git a/Tests/BlurtEngineTests/MicCaptureWarmTests.swift b/Tests/BlurtEngineTests/MicCaptureWarmTests.swift deleted file mode 100644 index f63dbfea..00000000 --- a/Tests/BlurtEngineTests/MicCaptureWarmTests.swift +++ /dev/null @@ -1,193 +0,0 @@ -import Foundation -import Testing - -@testable import BlurtEngine - -/// The warm-recorder lifecycle (`MicCapture+Warm`): prepare ahead of the press, -/// refuse to double-prepare, validate against the live input before reuse, and -/// tear down on a stale expiry ticket. -/// -/// Gated on BLURT_LIVE_AUDIO_TESTS=1, like `MicCaptureLevelsTests`, because every -/// test here goes through `MicCapture.makeRecorder()` — and that calls -/// `prepareToRecord()`, which is *the* route-activation call this whole change is -/// about. On a runner with no input device it is not merely unreliable, it is -/// hostile: it blocks the calling thread rather than suspending, so several of -/// these running concurrently occupy the cooperative pool and wedge the entire -/// `swift test` run, not just this suite. That is not a hypothetical — an -/// ungated first attempt failed three expectations here and then hung 171 -/// unrelated tests until the job's 30-minute timeout killed it. -/// -/// So this suite documents and locks the warm lifecycle for a human running it -/// on a real Mac with a real microphone; `MicCapture+Warm.swift` is excluded -/// from the coverage gate for the same reason `MicCapture.swift` is. -@Suite( - "MicCapture warm recorder (live)", - .enabled( - if: ProcessInfo.processInfo.environment["BLURT_LIVE_AUDIO_TESTS"] == "1", - "set BLURT_LIVE_AUDIO_TESTS=1 to run (needs a real microphone)"), - .tags(.liveAudio), - .timeLimit(.minutes(1))) -struct MicCaptureWarmTests { - private let builtIn = AudioRoute.InputSnapshot(deviceID: 7, transportType: nil) - private let airPods = AudioRoute.InputSnapshot(deviceID: 8, transportType: nil) - - @Test("warmUp prepares a recorder once; a second call has been overtaken and no-ops") - func warmUpPreparesOnce() async throws { - let mic = MicCapture() - #expect(await mic.canPrepareWarmRecorder) - - await mic.warmUp() - #expect(await mic.hasWarmRecorder) - // The slot is taken, so it is no longer safe to open the input for another. - #expect(await mic.canPrepareWarmRecorder == false) - - // A second warm-up — e.g. a scheduled re-warm that lost its race with a - // launch-time warmUp — must not stack a second open recorder onto the input. - let generation = await mic.preparedGeneration - await mic.warmUp() - #expect(await mic.preparedGeneration == generation) - - await mic.discardWarmRecorder() - } - - @Test("warmUp is refused across the bring-up window, when both recorder slots are nil") - func warmUpRefusedDuringBringUp() async throws { - // The regression `bringingUpCapture` exists for: across `start()`'s liveness - // wait both `activeRecorder` and `warm` are nil, so a guard reading only - // those would call a live capture "idle" and prepare a second recorder onto - // the already-open input. - let mic = MicCapture() - await mic.setBringingUpCapture(true) - await mic.warmUp() - #expect(await mic.hasWarmRecorder == false) - - await mic.setBringingUpCapture(false) - await mic.warmUp() - #expect(await mic.hasWarmRecorder) - - await mic.discardWarmRecorder() - } - - @Test("a warm recorder is reused only while still bound to the live default input") - func warmRecorderReusedWhenDeviceUnchanged() async throws { - let mic = MicCapture() - try await mic.installWarmRecorder(boundTo: builtIn) - - #expect(await mic.takeWarm(matching: builtIn)) - // Consumed either way — the take empties the slot, so the next press can't - // double-dip and the re-warm guard sees it as free again. - #expect(await mic.hasWarmRecorder == false) - #expect(await mic.canPrepareWarmRecorder) - - // Nothing held: the take answers nil rather than conjuring a recorder. - #expect(await mic.takeWarm(matching: builtIn) == false) - } - - @Test("a device change — or an unreadable route on either side — discards the warm recorder") - func warmRecorderDiscardedOnDeviceChangeOrUnknown() async throws { - // Reuse requires *positively* confirming the device is unchanged. A recorder - // bound to the wrong device doesn't fail loudly — it records the wrong mic, - // or silence — so unknown means discard: paying route activation again is - // the cheaper mistake. - let mic = MicCapture() - - // The user connected their AirPods after the warm-up. - try await mic.installWarmRecorder(boundTo: builtIn) - #expect(await mic.takeWarm(matching: airPods) == false) - #expect(await mic.hasWarmRecorder == false) - - // The route was unreadable when the recorder was warmed. - try await mic.installWarmRecorder(boundTo: nil) - #expect(await mic.takeWarm(matching: builtIn) == false) - - // The route is unreadable now, at press time. - try await mic.installWarmRecorder(boundTo: builtIn) - #expect(await mic.takeWarm(matching: nil) == false) - } - - @Test("an expiry with a stale generation ticket leaves a later warm recorder alone") - func staleExpiryTicketDoesNothing() async throws { - // Cancellation alone doesn't cover this: an expiry already past its - // cancellation check still gets its actor turn, and without the ticket it - // would tear down a recorder prepared a moment ago. - let mic = MicCapture() - try await mic.installWarmRecorder(boundTo: builtIn) - let generation = await mic.preparedGeneration - - await mic.releasePreparedRecorder(generation: generation - 1) - #expect(await mic.hasWarmRecorder) - - // The live ticket is what tears the idle recorder down, freeing the input. - await mic.releasePreparedRecorder(generation: generation) - #expect(await mic.hasWarmRecorder == false) - } - - @Test("a scheduled re-warm eventually prepares a recorder on its own turn") - func scheduledRewarmPreparesARecorder() async throws { - // `stop()`/`cancelCapture()` schedule rather than prepare inline because - // preparing re-opens the input — the slow part — and both sit on paths the - // user is waiting behind. All this can pin deterministically is the other - // half of that contract: the scheduled task does land, and prepares. - // Polled on a deadline with a real sleep between reads, NOT a bare - // `Task.yield()` spin. `scheduleRewarm` hands the work to a child task that - // needs a cooperative thread to run on, and `warmUp()` then blocks that - // thread inside CoreAudio — so a hot spin here competes with the very task - // it is waiting for, and on a machine where the recorder never materialises - // it never terminates at all. Sleeping yields the thread outright, and the - // deadline turns "never landed" into a failed expectation rather than a hang - // the suite's time limit has to clean up. - let mic = MicCapture() - let before = await mic.preparedGeneration - await mic.scheduleRewarm() - let deadline = ContinuousClock().now.advanced(by: .seconds(5)) - while await mic.preparedGeneration == before, ContinuousClock().now < deadline { - try await Task.sleep(for: .milliseconds(10)) - } - #expect(await mic.hasWarmRecorder) - - await mic.discardWarmRecorder() - } -} - -/// Actor-isolated test seams over `MicCapture`'s internal warm-recorder state. -/// Extensions because `AVAudioRecorder` is not `Sendable`, so neither the `warm` -/// slot nor `takeWarmRecorder`'s return can cross the actor boundary into a -/// test — each helper reduces it to a `Sendable` answer on the actor instead. -extension MicCapture { - /// Whether a warm recorder is currently held. - var hasWarmRecorder: Bool { warm != nil } - - /// Opens or closes the bring-up window `canPrepareWarmRecorder` guards on, - /// standing in for a `start()` suspended in its liveness wait (which needs - /// real hardware to reach). - func setBringingUpCapture(_ value: Bool) { - bringingUpCapture = value - } - - /// Installs a warm recorder bound to a *known* input — `prepareWarmRecorder` - /// with the `AudioRoute.currentInput()` read replaced by `input`, so the - /// device-identity tests don't depend on what the test machine's routing - /// happens to answer. - func installWarmRecorder(boundTo input: AudioRoute.InputSnapshot?) throws { - preparedGeneration += 1 - warm = WarmRecorder( - recorder: try Self.makeRecorder(), input: input, generation: preparedGeneration) - } - - /// Consumes the warm slot through the production validation, reporting whether - /// the recorder was reusable. A reused recorder's temp file is cleaned up here, - /// the job `start()` would otherwise inherit; the discard path already does so. - func takeWarm(matching input: AudioRoute.InputSnapshot?) -> Bool { - guard let recorder = takeWarmRecorder(matching: input) else { return false } - Self.removeFile(at: recorder.url) - return true - } - - /// Test teardown: drops the warm recorder (and its expiry countdown) so a - /// finished test doesn't leave a 60 s expiry task holding the suite's actor. - /// Through `takeWarmRecorder` — which cancels the expiry — rather than a bare - /// `warm = nil`, so teardown can't diverge from how production empties the slot. - func discardWarmRecorder() { - _ = takeWarmRecorder(matching: nil) - } -} diff --git a/Tests/BlurtEngineTests/MicDeviceStoreTests.swift b/Tests/BlurtEngineTests/MicDeviceStoreTests.swift new file mode 100644 index 00000000..dc5b01d5 --- /dev/null +++ b/Tests/BlurtEngineTests/MicDeviceStoreTests.swift @@ -0,0 +1,62 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +/// The pure half of microphone selection: the persisted-slot decode shared by +/// the store and the Settings picker, the store's round trip, and the +/// missing-device fallback the capture path applies per press. +@Suite("MicDeviceStore") +struct MicDeviceStoreTests { + @Test("unset reads as same-as-system") + func unsetReadsAsSystemDefault() { + let store = MicDeviceStore(defaults: freshDefaults()) + #expect(store.selection == .systemDefault) + } + + @Test("the empty string is same-as-system, anything else is a pin") + func fromPersistedDecodesEmptyAsSystemDefault() { + // The empty string is also what an unset key reads back as, so the decode + // rule gives the untouched install exactly one meaning. + #expect(MicDeviceSelection.fromPersisted("") == .systemDefault) + #expect(MicDeviceSelection.fromPersisted("uid:built-in") == .pinned(uid: "uid:built-in")) + } + + @Test("a pinned selection round-trips through the store") + func pinRoundTrips() { + let defaults = freshDefaults() + let store = MicDeviceStore(defaults: defaults) + + store.selection = .pinned(uid: "AppleUSBAudioEngine:test") + #expect(store.selection == .pinned(uid: "AppleUSBAudioEngine:test")) + // The on-disk shape is the bare UID string — the contract the Settings + // picker's `@AppStorage` observation reads. + #expect(defaults.string(forKey: MicDeviceStore.defaultsKey) == "AppleUSBAudioEngine:test") + } + + @Test("selecting same-as-system overwrites a stored pin") + func systemDefaultOverwritesPin() { + let store = MicDeviceStore(defaults: freshDefaults()) + store.selection = .pinned(uid: "uid:external") + + store.selection = .systemDefault + #expect(store.selection == .systemDefault) + } + + @Test("a pinned device that disappeared falls back to the system default") + func missingPinFallsBack() { + // The graceful degradation: unplugging the pinned mic must degrade the next + // press to the system default, never fail it — while the pin itself stays + // stored (`effective` is per-capture; nothing rewrites the slot). + let pinned = MicDeviceSelection.pinned(uid: "uid:gone") + #expect(pinned.effective(pinnedDevicePresent: false) == .systemDefault) + #expect(pinned.effective(pinnedDevicePresent: true) == pinned) + } + + @Test("same-as-system is unaffected by device presence") + func systemDefaultIsStableUnderEffective() { + #expect(MicDeviceSelection.systemDefault.effective(pinnedDevicePresent: true) == .systemDefault) + #expect( + MicDeviceSelection.systemDefault.effective(pinnedDevicePresent: false) == .systemDefault) + } +} diff --git a/Tests/BlurtEngineTests/MicLivenessTests.swift b/Tests/BlurtEngineTests/MicLivenessTests.swift index 02343d45..26730b15 100644 --- a/Tests/BlurtEngineTests/MicLivenessTests.swift +++ b/Tests/BlurtEngineTests/MicLivenessTests.swift @@ -64,13 +64,13 @@ struct MicLivenessTests { #expect(MicLiveness.unknownTransportTimeout < MicLiveness.bluetoothTimeout) } - @Test("already-advancing recorder clock confirms immediately, without sleeping") + @Test("frames already arriving confirm immediately, without sleeping") func immediateLiveness() async { let clock = TestClock() // Never advanced: a sleep would park forever, so returning at all proves // the fast path never sleeps (the suite's time limit backs that up). let gap = await MicLiveness.waitUntilLive( - timeout: .seconds(1), clock: clock, currentTime: { 0.1 }, + timeout: .seconds(1), clock: clock, deliveredFrames: { 1024 }, inputPowerDB: { quietRoomPowerDB }) #expect(gap == .zero) } @@ -81,12 +81,12 @@ struct MicLivenessTests { let polls = Mutex(0) async let gap = MicLiveness.waitUntilLive( timeout: MicLiveness.bluetoothTimeout, clock: clock, - currentTime: { - // Stuck at 0 for the first three checks — the route still switching — - // then the recorder clock starts moving. + deliveredFrames: { + // Nothing delivered for the first three checks — the route still + // switching — then buffers start arriving. polls.withLock { polls in polls += 1 - return polls < 4 ? 0 : 0.05 + return polls < 4 ? 0 : 1024 } }, inputPowerDB: { quietRoomPowerDB }) // Each wait is twice the last: 1 ms, 2 ms, 4 ms. Driving the clock by the @@ -112,10 +112,10 @@ struct MicLivenessTests { let polls = Mutex(0) async let gap = MicLiveness.waitUntilLive( timeout: .seconds(30), clock: clock, - currentTime: { + deliveredFrames: { polls.withLock { polls in polls += 1 - return polls < 12 ? 0 : 0.05 + return polls < 12 ? 0 : 1024 } }, inputPowerDB: { quietRoomPowerDB }) var expected = MicLiveness.initialPollInterval @@ -128,12 +128,12 @@ struct MicLivenessTests { #expect(await gap != nil) } - @Test("a clock advancing over digital silence is not live — the shipped bug") + @Test("frames arriving as digital silence are not live — the shipped bug") func zeroFilledBuffersAreNotLive() async { - // The gate's original signal was `currentTime > 0` alone, and macOS can hand + // The gate's original signal was frame arrival alone, and macOS can hand // a stale or not-yet-switched device's queue all-zero buffers (the same // failure that retired the `AVAudioEngine` capture path). Frames of digital - // silence advance the recorder's clock exactly like real audio, so the wait + // silence arrive and count exactly like real audio, so the wait // was satisfied on the first ~1 ms poll while the AirPods link was still // renegotiating: the "Connecting…" pill flashed past and the start chime // fired over a dead mic. With the power term the wait holds, and a device @@ -152,7 +152,7 @@ struct MicLivenessTests { #expect(await waitToCap(powerDB: { -Float.infinity }) == nil) } - @Test("an advancing clock plus real input power confirms at once") + @Test("delivered frames plus real input power confirm at once") func realInputPowerConfirmsPromptly() async { // The other half of the floor's job: it must not become voice-activity // detection. A live mic in a silent room reads its own self-noise, well above @@ -162,7 +162,7 @@ struct MicLivenessTests { // would fail this rather than let it pass slowly. let clock = TestClock() let gap = await MicLiveness.waitUntilLive( - timeout: MicLiveness.bluetoothTimeout, clock: clock, currentTime: { 0.05 }, + timeout: MicLiveness.bluetoothTimeout, clock: clock, deliveredFrames: { 1024 }, inputPowerDB: { quietRoomPowerDB }) #expect(gap == .zero) } @@ -189,7 +189,7 @@ struct MicLivenessTests { #expect(failedOpen.contains("powerDB=-160.0")) } - @Test("a clock that never advances times out with nil — the not-live verdict") + @Test("a recorder that never delivers times out with nil — the not-live verdict") func timeoutReturnsNil() async { // nil is pure information — the *caller* decides what it means, and // `MicCapture.start()` fails the press closed on it (tears the recorder @@ -198,7 +198,7 @@ struct MicLivenessTests { let clock = TestClock() let timeout = MicLiveness.initialPollInterval * 3 async let gap = MicLiveness.waitUntilLive( - timeout: timeout, clock: clock, currentTime: { 0 }, + timeout: timeout, clock: clock, deliveredFrames: { 0 }, inputPowerDB: { digitalSilencePowerDB }) var expected = MicLiveness.initialPollInterval for _ in 1...2 { @@ -210,14 +210,14 @@ struct MicLivenessTests { } /// Drives the wait to its cap against a clock this test owns, for a power probe - /// that never reports live while the recorder's clock is already advancing. + /// that never reports live while frames are already arriving. /// Shared so each "not live" reading is one assertion rather than a copy of the /// backoff-driving loop. private func waitToCap(powerDB: @escaping @Sendable () -> Float) async -> Duration? { let clock = TestClock() let timeout = MicLiveness.initialPollInterval * 3 async let gap = MicLiveness.waitUntilLive( - timeout: timeout, clock: clock, currentTime: { 0.05 }, inputPowerDB: powerDB) + timeout: timeout, clock: clock, deliveredFrames: { 1024 }, inputPowerDB: powerDB) var expected = MicLiveness.initialPollInterval for _ in 1...2 { await clock.waitUntilSleeping(for: expected) diff --git a/Tests/BlurtEngineTests/PersistedSettingsTests.swift b/Tests/BlurtEngineTests/PersistedSettingsTests.swift index a20378c6..7504d456 100644 --- a/Tests/BlurtEngineTests/PersistedSettingsTests.swift +++ b/Tests/BlurtEngineTests/PersistedSettingsTests.swift @@ -34,14 +34,17 @@ struct PersistedSettingsTests { // The launch update check's throttle: left out of the sweep, a UI-test run // (or a "clean install") would inherit yesterday's stamp and skip the check. #expect(PersistedSettings.allDefaultsKeys.contains(LastUpdateCheckStore.defaultsKey)) + // The pinned microphone: left out, a reset "clean install" would keep + // recording from a previously pinned device. + #expect(PersistedSettings.allDefaultsKeys.contains(MicDeviceStore.defaultsKey)) } @Test("the roster carries no stale or duplicate keys") func rosterHasNoStrays() { - // Exactly the eight known stores' keys (OverlayOriginStore contributes two, + // Exactly the nine known stores' keys (OverlayOriginStore contributes two, // StyleProfileStore three): a removed store must leave the roster in the same // change, and a key listed twice would hint at a copy-paste slip. - #expect(PersistedSettings.allDefaultsKeys.count == 11) + #expect(PersistedSettings.allDefaultsKeys.count == 12) #expect(Set(PersistedSettings.allDefaultsKeys).count == PersistedSettings.allDefaultsKeys.count) } @@ -65,11 +68,12 @@ struct PersistedSettingsTests { OverlayOriginStore.xDefaultsKey, OverlayOriginStore.yDefaultsKey, LastUpdateCheckStore.defaultsKey, + MicDeviceStore.defaultsKey, ] #expect(storeKeys == Set(DefaultsKey.allCases.map(\.key))) // No two stores sharing a slot — the Set above would have quietly absorbed a // collision, and two stores on one key means each overwrites the other. - #expect(storeKeys.count == 11) + #expect(storeKeys.count == 12) } @Test("resetAll clears every roster key and leaves unrelated ones alone") diff --git a/scripts/check-invariants.sh b/scripts/check-invariants.sh index 88d5580a..da1ab044 100755 --- a/scripts/check-invariants.sh +++ b/scripts/check-invariants.sh @@ -110,7 +110,7 @@ SCOPES=( "$ENGINE $APP" ) ADVICE=( - "MicCapture uses a fresh AVAudioRecorder per session — a long-lived engine goes stale on a device switch" + "MicCapture builds a fresh AVCaptureSession recorder per capture — a long-lived engine goes stale on a device switch" "the dictation API returns the full text in one response; there is no streaming path" "cleanup is the API's server-side rewrite via the llm block on the same /transcribe call" "transcription is a remote AssemblyAI call — no on-device ASR/LLM, no model cache" diff --git a/scripts/check.sh b/scripts/check.sh index c91fa655..f95a9802 100755 --- a/scripts/check.sh +++ b/scripts/check.sh @@ -600,28 +600,13 @@ else command -v python3 >/dev/null 2>&1 || die_check "python3 not found — needed to read the coverage summary" # Exclusions (so the figure reflects deterministically-testable engine code): # - Tests/ : test files themselves, not shipping code. - # - MicCapture.swift : the AVAudioRecorder capture actor. It needs a real - # audio device, so it can't run in CI (its integration - # test, MicCaptureLevelsTests, is env-gated for the same + # - MicCapture.swift : the capture actor. It needs a real audio device, so + # it can't run in CI (its integration test, + # MicCaptureLevelsTests, is env-gated for the same # reason). Its pure meter math lives in # MicCapture+Meter.swift, which IS covered. Keep this # list tight — exclude only code that genuinely cannot # be exercised without hardware. - # - MicCapture+Warm.swift : the same actor's warm-recorder lifecycle, split - # out of MicCapture.swift only for the lint file-length - # budget. Every path through it runs makeRecorder(), - # i.e. prepareToRecord() — the route-activation call, the - # one thing here that genuinely needs a device. Unlike - # +Meter (pure math, covered), splitting this out moved - # hardware-bound code onto the counted side by accident: - # the pattern above pins a literal filename. Trying to - # cover it in CI didn't merely fail, it deadlocked the - # whole test run — prepareToRecord blocks its thread - # instead of suspending, so concurrent attempts drained - # the cooperative pool. Its suite is env-gated alongside - # MicCaptureLevelsTests. The transport and liveness - # *policy* it consults stays covered, in AudioTransport - # and MicLiveness. # - AudioRoute(Monitor).swift : the CoreAudio routing reads (AudioRoute) and the # property listeners (AudioRouteMonitor). Both answer # questions only real hardware can answer — which device @@ -629,8 +614,24 @@ else # when the user switches output — and the listener half # can only fire on an actual route change. Same # justification as MicCapture.swift above. + # - AudioInputDevices.swift : the input-device enumeration and UID translation + # behind the Settings microphone picker — HAL reads with + # the same justification as AudioRoute.swift. The pure + # selection/fallback rules it serves (MicDeviceSelection, + # MicDeviceStore) stay covered. + # - CaptureSessionRecorder.swift : the AVCaptureSession recorder — building a + # session around a real device input and running it is + # hardware through and through, and it needs mic + # authorization on top. Trying to cover its predecessor + # in CI didn't merely fail, it deadlocked the whole test + # run: the route-activation call blocked its thread + # instead of suspending, so concurrent attempts drained + # the cooperative pool. Its live suites ride the env gate + # (MicCaptureLevelsTests, AudioInputDevicesTests). The + # transport and liveness *policy* it serves stays + # covered, in AudioTransport and MicLiveness. COVERAGE="$(xcrun llvm-cov export -summary-only -instr-profile "$PROFDATA" "$XCTEST_BIN" \ - -ignore-filename-regex='Tests/|Audio/MicCapture(\+Warm)?\.swift|Audio/AudioRoute(Monitor)?\.swift' \ + -ignore-filename-regex='Tests/|Audio/MicCapture\.swift|Audio/AudioRoute(Monitor)?\.swift|Audio/AudioInputDevices\.swift|Audio/CaptureSessionRecorder\.swift' \ | python3 -c 'import sys,json; print(round(json.load(sys.stdin)["data"][0]["totals"]["lines"]["percent"],2))')" echo "engine line coverage: ${COVERAGE}%" if ! awk -v c="$COVERAGE" -v min="$MIN_COVERAGE" 'BEGIN{ exit (c+0 < min+0) }'; then