Skip to content

Add microphone selection to Settings - #163

Open
claude[bot] wants to merge 4 commits into
mainfrom
feat/mic-device-selection
Open

Add microphone selection to Settings#163
claude[bot] wants to merge 4 commits into
mainfrom
feat/mic-device-selection

Conversation

@claude

@claude claude Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Requested by Mez · Slack thread

Before: Blurt always records from the system default input, via a per-session AVAudioRecorder writing a temp WAV. After: a Microphone dropdown in Settings › General — "Same as system (current default's name)" plus each input device — and capture runs on a single per-session AVCaptureSession recorder that binds to the pinned device (or the default), converts to the dictation API's 16 kHz mono S16LE in its data output, and accumulates the upload blob in memory. The choice persists as the device UID, and dictation falls back to the system default whenever the pinned device isn't connected (without unpinning).

What & why

Lets the user pin dictation to a specific microphone instead of always following the system default — e.g. keep AirPods for output while dictating into a USB mic.

How (single path, at Alex's direction): this PR originally shipped two backends — the untouched AVAudioRecorder WAV path for "same as system" plus a device-pinned AudioQueue. Per Alex Kroman's direction it now has one: CaptureSessionRecorder, an AVCaptureSession built fresh per capture behind MicCapture's CaptureRecorder seam (the fresh-recorder-per-session invariant is preserved; the AVAudioEngine/installTap ban is untouched and its check-invariants.sh anchors still pass --self-test; the settled-decisions prose in AGENTS.md / the guardrails skill / the script was updated consistently). Device selection is AVCaptureDevice(uniqueID:) — on macOS the uniqueID is the CoreAudio UID MicDeviceStore persists — with the pure, unit-tested missing-device fallback (MicDeviceSelection.effective) unchanged. The liveness gate's inputs are re-derived with MicLiveness itself unchanged: the clock is frames actually delivered (summed per sample buffer), power is the connection's AVCaptureAudioChannel.averagePowerLevel; fail-closed timeout, transport-keyed caps, and the Bluetooth tail linger still key off the resolved device's CoreAudio snapshot. Warm-up pre-builds the session without startRunning() — the device stays closed and no input indicator shows while idle — so route activation lands inside the press's connecting window (the liveness budget starts only after record() returns). A built-but-idle session holds nothing open, so the 60 s warm expiry (which existed to un-pin AirPods from their degraded output profile) is deleted.

Verify on real hardware (please read)

  • MicLiveness.silenceFloorDB (-115 dBFS) — the load-bearing one. Kept at -115: the dBFS math is meter-independent (full scale = 0, one int16 LSB ≈ -90.3, digital silence at/near the meter floor, -115 in the empty band between). But it was calibrated on AVAudioRecorder's meter; where AVCaptureAudioChannel.averagePowerLevel actually bottoms out on zero-filled buffers — and what it reads before its first update — must be confirmed, or the gate could fail open. AirPods mid A2DP→HFP switch is the case that matters: the gate must keep holding on all-zero buffers until real samples flow. AudioInputDevicesTests (env-gated, BLURT_LIVE_AUDIO_TESTS=1) now asserts a live mic out-reads the floor.
  • Warm/start latency: warm-up no longer pre-opens the route; startRunning() cost lands at every press, inside the connecting pill. Measure it wired and on AirPods against the old feel (the liveness frame-arrival caps — 300 ms local / 1 s unknown / 2.5 s BT — start after startRunning() returns, so they shouldn't need retuning, but the perceived press→chime time may grow).
  • Tail linger on AirPods: confirm the 220 ms Bluetooth linger still recovers the last word with the session backend.
  • Uniqueness assumption: AVCaptureDevice(uniqueID:) accepting the CoreAudio UID string for every device the picker lists (aggregates/virtual devices included).

Notes for review:

  • App UI lives in the existing SoundStepView.swift (+ UITestIdentifiers, already in both targets), so project.yml/project.pbxproj are untouched — written on Linux where xcodegen can't run.
  • Engine imports gained CoreMedia (block-buffer copy in the delegate; ships under AVFoundation's umbrella). The AudioQueue backend and its AudioToolbox import are gone. No SPM dependencies.
  • The temp-WAV write/read-back and decodePCM are gone with the old backend — the release path hands the accumulated S16LE straight to upload.

How it was tested

On Linux — can't build/test Swift locally; CI on macOS is the authority. scripts/check.sh --portable passes (exit 0) including check-invariants.sh and its --self-test (the reworded settled-decision prose stays pinned in all three places). Suites: MicDeviceStoreTests (decode/round-trip/fallback), PersistedSettingsTests (roster 12 keys), MicCaptureFormatTests (meter math + error wording), the warm suite rewritten for the expiry-free lifecycle (identity probe, device/pin validation), live-gated AudioInputDevicesTests (enumeration, UID round-trip, session-recorder capture + silence-floor assertion — engine tests never touch real CoreAudio ungated), and the SettingsUITests picker case ("Same as system" default only).

  • scripts/check.sh passes (or CI will, if I'm not on a Mac) — portable subset passes; full check runs on CI
  • I read AGENTS.md and this doesn't reintroduce anything deliberately removed (the capture-backend prose was updated at the owner's direction; the bans stand)
  • Docs updated if behavior changed (AGENTS.md table + MicCapture section + repo map, engine README, guardrails skill)

claude added 4 commits August 25, 2026 23:00
A Microphone picker in Settings > General pins dictation to a specific
input device, persisted as the device's CoreAudio UID (MicDeviceStore /
BlurtMicDeviceUID). Un-pinned capture — the default — keeps the shipped
AVAudioRecorder WAV path untouched; a pinned capture records through a
fresh-per-session AudioQueue bound to the device via
kAudioQueueProperty_CurrentDevice, behind a new CaptureRecorder seam
inside MicCapture.

The transport-keyed policies (liveness timeout, Bluetooth tail linger)
and the warm-recorder identity check key off the pinned device's
snapshot; a pinned device that isn't connected falls back to the system
default per press (MicDeviceSelection.effective, pure and unit-tested)
without unpinning. Device enumeration and UID translation live in
AudioInputDevices (hardware-bound, coverage-excluded like AudioRoute).

Engine tests stay off real CoreAudio: the new live suites ride the
BLURT_LIVE_AUDIO_TESTS gate, and the pure decode/fallback/store rules
are covered by MicDeviceStoreTests and the roster tests (now 12 keys).
- AudioQueueNewInput takes a capture-free closure literal forwarding to
  the static callback: a C function pointer cannot be formed from a
  static-method reference, only a top-level func or a literal closure.
- The static callback drops the unused packet timing/description
  parameters (raw LPCM never needs them), which also satisfies
  swiftlint's five-parameter limit.
- MicCapture.start(): break the warm-take/make-backend assignment the
  way swift-format asks (AddLines at the try).
Swift refuses the implicit inout-to-UnsafeRawPointer conversion for a
variable whose type carries an object reference, so the
kAudioQueueProperty_CurrentDevice value (the CFString reference itself)
goes through withUnsafeMutablePointer — the same pattern
AudioInputDevices already uses for the UID-translation qualifier, which
this CI run compiled cleanly.

Also swap the one key-path-inside-#expect in AudioInputDevicesTests for
an explicit closure, per AGENTS.md's rethrows/key-path macro trap.
Owner-directed (Alex Kroman, 2026-08-25): replace both capture backends
— the AVAudioRecorder/WAV path and the device-pinned AudioQueue — with
one AVCaptureSession recorder (CaptureSessionRecorder) behind the
existing CaptureRecorder seam. Still fresh per session: the session is
built around the press-time resolution of the selection (pinned device
via AVCaptureDevice(uniqueID:), else the default input, with the same
pure missing-device fallback), its data output converts to 16 kHz mono
16-bit LPCM, and the delegate accumulates upload-ready S16LE in memory
— no temp file, no decode pass.

Liveness gate inputs re-derived on the new API with MicLiveness itself
unchanged: the clock is the frames actually delivered (summed off each
sample buffer), power is the connection's AVCaptureAudioChannel
averagePowerLevel; fail-closed timeout, transport-keyed caps, and the
Bluetooth tail linger all still key off the resolved device's CoreAudio
snapshot. silenceFloorDB stays -115 dBFS — the dBFS math (0 = full
scale, one int16 LSB ~ -90) is meter-independent — but was calibrated
on the retired meter and must be re-verified on hardware.

Warm-up now pre-builds the session without starting it — the device
stays closed and no input indicator shows while idle — so route
activation lands inside the connecting window at record()'s
startRunning(). A built-but-idle session holds nothing open, so the
60 s warm expiry (which existed to un-pin AirPods from their degraded
output profile) is deleted along with its generation tickets.

Settled-decision prose updated consistently in AGENTS.md's table, the
project-guardrails skill, and check-invariants.sh's advice string; the
AVAudioEngine/installTap ban is unchanged and its anchors still pass
--self-test. Tests updated to the new backend (warm suite identity
probe replaces the generation counter; live suites stay env-gated).
@claude
claude Bot marked this pull request as ready for review August 25, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant