Exclusive-mode output: feedpak/sloppak song audio is silent (stems bypass the native backing funnel)

[ChrisBeWithYou]

Handoff → @OmikronApex. Follow-up to the WASAPI distortion hunt, picking up the item you scoped out of #86. The docs/native-song-playback-plan.md you referenced never landed on any branch, so this reconstructs the plan from the code — root cause is pinned to a specific gate, with both a cheap-interim and a proper fix. Findings gathered by @ChrisBeWithYou.

Context / status

This tracks the one remaining known issue from the WASAPI distortion hunt (tester: MajorMokoto, RTC + Komplete Audio; reproduced by @OmikronApex). The distortion / digital-static cluster is already fixed and awaiting merge:

• fix(audio): input-callback double registration, stale-format race, effects schema alignment #86 — engine fixes (block-size chunking already merged in fix(audio): never feed chain processors blocks larger than prepared size #85; input-callback double-registration, stale-format race, effects-schema alias)
• fix(audio-effects): accept pre-rebrand chain plan schema as alias feedBack#816 — accept pre-rebrand chain-plan schema id as alias
• fix(effects): rebranded plan schema id + stop poll/stop-handler rebuild ping-pong feedBack-plugin-rig-builder#52 — plugin sends new schema id + stops the poll/stop-handler rebuild ping-pong

#86 explicitly scopes this out: "song audio silent with exclusive output — song playback runs through the renderer; plan exists separately."

Symptom

Exclusive output mode: guitar monitoring is clean, but feedpak song audio is silent (playback pauses / won't advance). Console shows:

The AudioContext encountered an error from the audio device or the WebAudio renderer.

Shared-mode WASAPI works. Tester repro'd on a feedpak (Accept - Metal Heart).

Root cause (confirmed in code)

Device-ownership conflict. The native JUCE engine (guitar in/out + VST/NAM) opens the device in exclusive mode, so it holds the output device exclusively. Chromium's WebAudio / HTML5 <audio> — the renderer path song playback normally uses — then cannot open that same output device → the AudioContext error above → silence.

Why loose-folder songs survive but feedpak songs don't. The core already has a native-backing funnel that migrates song audio onto the engine's own backing transport when the engine is running:

• static/app.js → _installJuceEngineRoutingWatcher() / window.jucePlayer → loadBackingTrack / startBacking / seekBacking (desktop src/main/audio-bridge.ts → AudioEngine::loadBackingTrack, a JUCE AudioTransportSource mixed into the engine output — so it plays fine under exclusive mode).

But feedpak/sloppak songs are gated out of that funnel:

• static/highway.js:3414 — const isAudioUrl = msg.audio_url.startsWith('/audio/'); … window._currentSongAudio = { url: msg.audio_url, juceEligible: isAudioUrl };
• static/app.js:5122-5123 — // Only /audio/ songs are JUCE-routable; sloppak stems stay on HTML5. if (!songAudio || !songAudio.juceEligible) return;

Loose-folder songs are served at /audio/… (single mixed file) → juceEligible = true → routed through JUCE → survive exclusive mode. Sloppak/feedpak songs are served at /api/sloppak/… → juceEligible = false → stay on HTML5/WebAudio → die in exclusive output. feedpak is the primary format, so most real songs hit this.

Why stems were excluded (two reasons):

1. /api/sloppak/… stem URLs aren't resolvable to a single local file via /api/audio-local-path (what loadBackingTrack needs).
2. More fundamentally, stems need per-stem mixing — the stems plugin mutes the core <audio> and drives its own WebAudio graph with per-stem faders (static/audio-mixer.js:142 "Multi-stem sloppak: the stems plugin mutes the core element … drive that master instead"). A single-file loadBackingTrack transport can't reproduce a multi-stem mix.

Fix options

Option A — native single-mix fallback (small, interim). feedpak always ships stems/full.ogg (per feedpak-spec). Expose a resolvable local path for it and mark the full-mix juceEligible, so in exclusive mode the funnel loads that single mixed file through JUCE. Cost: per-stem mixing is unavailable in exclusive mode only (acceptable degradation — the user gets song audio back). Smallest change; unblocks the common case.

Option B — mix the WebAudio master into the engine output bus (proper, general). Feed the renderer's song/stem master into the engine's output bus so it mixes with guitar and rides out the exclusively-held device. Reuse the existing stream-bus / backing-mix plumbing (setStreamBus(includeBacking, includeGuitar, gain) at src/audio/NodeAddon.cpp:1371, backing mix bus + getBackingLevel). Added latency lands only on song audio (~10–20ms per @OmikronApex's estimate) — the guitar-monitoring path is untouched, and backing latency isn't perceptually critical. Works for all songs including full stem mixing. This is the real fix; shares infrastructure with #48 (streamer mix outputs).

Rejected:

• Exclusive input + shared output — tester confirmed unplayable latency (split-mode output ring floor).
• WASAPI loopback / OS capture — Windows 10+ only, heavier; keep as last resort.

Key pointers

• static/highway.js:3414 — juceEligible = audio_url.startsWith('/audio/') (the gate)
• static/app.js:5119-5148 — _reevaluateJuceRouting() (the reroute watcher; the juceEligible early-return)
• static/app.js:4880-5010 — _switchHtml5ToJuce() (HTML5→JUCE migration; the reject/fallback path)
• static/audio-mixer.js:142-147 — stems plugin owns the master when a multi-stem sloppak is loaded
• src/audio/AudioEngine.cpp:1248 loadBackingTrack + src/main/audio-bridge.ts:1192 bridge — the native transport the funnel already targets
• src/audio/NodeAddon.cpp:1371 setStreamBus — the mix-bus Option B would extend

Related / secondary observations from the session (triage separately)

• "Windows Audio (Low Latency Mode)" output blows out the input gain when a rig_builder tone loads (persists on the fix-7 build). Latency is otherwise decent in that mode. Possibly a distinct gain-staging bug — see also Audio: consolidate input gain staging — one master input gain #72.
• WASAPI buffer-size slider is a no-op (driver forces the size). Consider hiding/locking it in WASAPI modes (UX).
• Overlaps: Streamer mix outputs — built-in multi-bus audio routing (OBS + Discord, no external router) #48 (streamer mix — same stream-bus infra Option B uses), Audio: consolidate input gain staging — one master input gain #72 (input gain staging), Monitor mute can't silence a loaded amp-sim → idle distorted buzz; make amp-sim opt-in + add a real monitor kill #46 (monitor mute / idle distortion), Audio input setup: driver-type labels (shipped) + wrong-mic-after-wizard & calibration (open) — tracking #45 (audio input setup).

[ChrisBeWithYou]

Implementation shape (scoped from the code) — for @OmikronApex

Dug into the mechanics to de-risk this before you pick it up. Two things worth knowing up front, then the concrete pieces and one product decision.

Correction: OGG is not a blocker

I initially worried the native transport couldn't read feedpak .ogg. It can — src/audio/CMakeLists.txt:113-119 sets JUCE_USE_OGGVORBIS=1 (+ FLAC, and MP3 explicitly enabled for backing tracks), so registerBasicFormats() at AudioEngine.cpp:21 registers the Ogg reader and loadBackingTrack accepts .ogg. The existing funnel is already capable of playing feedpak audio — the work is routing + gating, not codec support.

The fix is ~3 pieces across 3 repos

#	Piece	Repo	Status
1	/api/audio-local-path resolves /api/sloppak/{fn}/file/{rel} → on-disk cached file (today it only handles /audio/…)	core	clean, self-contained
2	juceEligible + the reroute watcher cover the feedpak full-mix URL (highway.js:3414, app.js:5119-5148)	core	clean, but must be gated — see below
3	A "output is exclusive right now" signal exposed to the renderer	feedback-desktop	doesn't exist today — the renderer has no idea it's in exclusive mode (audio-bridge.ts / preload.ts expose no share-mode getter). The engine knows its own WASAPI share mode internally; just needs surfacing.
4	Suspend/resume the stems WebAudio graph across the handoff	feedback-plugin-stems	new hook — see multi-stem note

Why the gate (#3) is load-bearing

The reroute switches to JUCE whenever isAudioRunning() is true. If we make feedpak songs juceEligible unconditionally, then any time an amp/tone is active the song routes through JUCE's single full-mix — silently disabling the stem mixer even in shared mode, where nothing is wrong. So the feedpak branch of the funnel must only engage for exclusive output. That's the reason #3 is required, not optional.

The stems-graph coordination detail (#4)

For single-mix feedpaks (no split stems: audio_url == original_audio_url, server.py:13210-13214), audio plays through the core <audio> element, so the existing reroute (_switchHtml5ToJuce, which does audio.pause() + audio.src='') works as-is once #1/#2 land. These are the easy first slice — no stems coordination needed.

For multi-stem feedpaks, the stems plugin mutes the core <audio> and drives its own AudioContext graph (createBufferSource / optional worklet — feedback-plugin-stems/screen.js). The reroute's audio.pause() does nothing to that graph, so a naïve switch would play JUCE full-mix and the WebAudio stems on top of each other. The plugin has the internal machinery (stopSources(), ctx.suspend, setMasterVolume) but no public "suspend for native backing / resume" hook for core to call around the switch.

Suggested order

1. Slice 1 — single-mix feedpaks: pieces Adding pre-commit example #1 + Sample Issue Templates #2 + feat(updater): remove duplicate Updates UI; now lives in slopsmith #3. Restores song audio for single-mix feedpaks and loose folders under exclusive output. No stems plugin change.
2. Slice 2 — multi-stem: add the stems suspend/resume contract (Fix native slowdown timing and SoundTouch feed behavior #4) and have core drive it around the reroute.

⁉️ Open question (for @OmikronApex + topkoa as stems author): multi-stem behavior in exclusive mode

What should a multi-stem song do under exclusive output?

• (a) Full-mix fallback — play stems/full.ogg through the engine; the per-stem mixer goes inert in exclusive mode only. Simplest; restores audio for everyone. Document the limitation.
• (b) Rethink exclusive as input-only — keep output shared so WebAudio/stems always work, use exclusive only for the latency-critical input. Conceptually cleanest and avoids Fix native slowdown timing and SoundTouch feed behavior #4 entirely. Caveat from the tester session: exclusive-input + shared-output was already tried and felt unplayable (Major: "not playable with the latency") — attributed to the split-mode output-ring floor, so this path needs engine work on that ring first.

(b) is the more holistic fix if the split-mode output-ring latency can be brought down; (a) is the pragmatic unblock. Since the stem mixer is topkoa's, their read on how gracefully it can suspend/hand off should factor in. Flagging rather than deciding.

File anchors: highway.js:3414 (the gate), app.js:5119-5148 (_reevaluateJuceRouting), app.js:4880-5010 (_switchHtml5ToJuce), server.py:13153-13245 (feedpak audio_url / original_audio_url), server.py:13886 (/api/audio-local-path), feedback-plugin-stems/screen.js (own AudioContext graph).