feat: snapshot thumbnails on the dropdown input rows (#242) - #265
Merged
Conversation
Each input in the dropdown now shows a still of what it is sending, taken when the dropdown opens. Snapshots, not live previews, which is the decision that made this small. The app holds exactly two streams; four live previews would have meant rebuilding that, and for a picker open for a few seconds it buys nothing. Iterating means **at most one extra stream open at any moment** -- so this needed no change to how feeds are held, and #261 (the N-feed refactor) closed unbuilt. Where each still comes from is the point. An input already on screen is read straight off the live <video> -- no stream, no negotiation, instant. Only inputs not currently displayed need a temporary getUserMedia, released the moment a frame has been drawn. A cold capture device takes a visible fraction of a second to negotiate, so the sweep is sequential and rows fill in as stills land. The tile renders before any still exists and is its own placeholder, so a row never changes height. A device that never presents a frame times out and stays a placeholder -- which is also what an exclusive-access device does, and what an input with nothing plugged in looks like. A failed capture is a normal outcome here, not an error. Stills are cached as data URLs keyed by deviceId, and rows are found by [data-device-id] rather than by held references: renderDropdownInputLists() rebuilds the row DOM on any device or selection change, which can happen mid-sweep. Re-opening shows the previous still immediately while a fresh sweep replaces it, rather than flashing back to placeholders. Mock inputs get their own branch in openTemporaryStream, because getUserMedia would reject a mock-input-N id. Without it the feature would be untestable in exactly the mode built for testing it (#248). **The first layout attempt was wrong and measuring caught it.** Full-row-width tiles (width:100% + aspect-ratio) gave 292x164 tiles and 218px rows in the single list at a 1280x800 window -- 1166px of content in a 640px panel, so a four-input picker scrolled. A fixed 128x72 is enough to recognise an input and changes what fits: the name sits beside the tile in the single list (~322px rows) and below it in a dual column (~149px). Two rules, each matching its width. Measured after: at the wall's 1200px height neither mode scrolls -- 881px and 678px against a 960px cap. Verified in a browser in BOTH modes this time, which is the thing I got wrong on #258 and which produced the bug reported against it. 23 new tests, 559 -> 582. The invariant the design rests on -- never two temporary streams open at once -- is checked directly, along with release ordering, carrying on past a device that refuses to open, releasing even when drawing throws, ignoring a second sweep while one runs, and reset superseding an in-flight one. Verified by breaking each: parallelising the sweep fails 3, dropping the re-entry guard 1, stretching instead of cropping 1, reverting the tile to full width 1, and leaking the stream 4. One test bug fixed on the way: two tests replaced the shared canvas stub and never restored it, so every later test in the file ran against a context that threw. The cache test failed for that reason rather than for anything in the code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #242.
Each input in the dropdown now shows a still of what it is sending, taken when the dropdown
opens.
Snapshots, not live previews — which is what made this small
The app holds exactly two streams. Four live previews would have meant rebuilding that; for
a picker open a few seconds it buys nothing. Iterating means at most one extra stream open
at any moment, so this needed no change to how feeds are held — and #261, the N-feed
refactor, closed unbuilt.
Where each still comes from is the point:
<video>— no stream, no negotiationgetUserMedia, released the moment a frame is drawnA cold capture device takes a visible fraction of a second to negotiate, so the sweep is
sequential and rows fill in as stills land. The tile renders before any still exists and is
its own placeholder, so a row never changes height. A device that never presents a frame
times out and stays a placeholder — which is also what an exclusive-access device does, and
what an input with nothing plugged in looks like. A failed capture is a normal outcome
here, not an error.
The first layout was wrong, and measuring caught it
Full-row-width tiles (
width: 100%+aspect-ratio) measured at a 1280×800 window:A four-input picker that scrolls is absurd for something you hover over to choose an input.
A fixed 128×72 is enough to recognise an input, and it changes what fits: the name sits
beside the tile in the single list (~322px rows) and below it in a dual column (~149px).
Two rules, each matching its width.
After, at the wall's 1200px height, neither mode scrolls — 881px and 678px against a 960px
cap.
Verified in both modes this time. Checking only Single is exactly what I got wrong on
#258 and what produced the bug you reported against it.
Other things worth knowing
Mock inputs need their own branch.
getUserMediarejects amock-input-Nid, soopenTemporaryStreamcallscreateMockStreaminstead. Without it the feature would beuntestable in exactly the mode built for testing it (#248).
Rows are found by
[data-device-id], not held references.renderDropdownInputLists()rebuilds the row DOM on any device or selection change, whichcan happen mid-sweep. Stills cache as data URLs (~4–6KB each), so re-opening shows the
previous still immediately while a fresh sweep replaces it, rather than flashing back to
placeholders.
Both open paths trigger it —
mouseenteron the trigger (a listener that alreadyexisted for the cursor) and
toggleDropdownfor touch. The panel's visibility is pure CSS,so that listener is the only JS signal a hover-open happened.
Tests
23 new, 559 → 582. Verified by breaking each invariant:
stop)One test bug fixed on the way. Two tests replaced the shared canvas stub and never
restored it, so every later test in the file ran against a context that threw. The cache
test was failing for that reason, not for anything in the code — worth saying because a
suite that poisons itself is worse than one test short.
Not verified
Real capture hardware. Every still here came from a mock input or a live
<video>; how longan actual capture card takes to negotiate, and whether any are exclusive-access on the
wall's host, is what
FRAME_TIMEOUT_MS(2500ms) is guessing at. That number is the one toadjust if tiles come up empty on real inputs.