Skip to content
BranchesTags

Latest commit

 

History

37 Commits

Folders and files

NameName
Last commit message
Last commit date
src
src
 
 
.gitignore
.gitignore
 
 
.yarnrc.yml
.yarnrc.yml
 
 
README.md
README.md
 
 
package.json
package.json
 
 
tsconfig.json
tsconfig.json
 
 
tsup.config.ts
tsup.config.ts
 
 
vitest.config.ts
vitest.config.ts
 
 
yarn.lock
yarn.lock
 
 

Repository files navigation

Rhombus React SDK — @rhombussystems/react

React + TypeScript components for embedding Rhombus video and audio in your own app. The SDK supports MPEG-DASH video, low-latency H.264, live and historical A100/DR40 audio, synchronized wall-clock controls, and browser-microphone talkback.

Your Rhombus API key never ships to the browser. Everything is built around short-lived federated session tokens minted by your backend (see Authentication).

Version: this guide tracks @rhombussystems/react 2.2.0. React 18+.

Contents

Install

npm install @rhombussystems/react
# or: yarn add @rhombussystems/react  /  pnpm add @rhombussystems/react
  • react and react-dom (>= 18) are peer dependencies — install them in your app.
  • dashjs and the worker-backed Opus decoder are bundled — you do not install them separately.
  • The realtime/canvas path uses the browser WebCodecs VideoDecoder (Chrome, Edge, Safari 16.4+; Firefox H.264 is still limited) — no extra dependency.
  • This is a browser media package. In an SSR framework, load the SDK from a client-only module; Dash.js evaluates browser globals when the package is imported.

Quick start

A complete live/VOD player with controls, a timeline, zoom, snapshot, and clip export — from a single cameraUuid:

import { RhombusPlayer } from "@rhombussystems/react";

export function CameraView() {
  return (
    <RhombusPlayer
      cameraUuid="YOUR_CAMERA_UUID"
      apiOverrideBaseUrl="https://your-api.example.com" // proxy mode (recommended)
      style={{ height: 480 }}
    />
  );
}

⚠️ Server setup is required. The SDK calls your origin for a token. Your server must expose POST /api/federated-token (the default path) or set paths.federatedToken to your route. Built-in Save Clip, proxy-mode audio, and talkback capability policy additionally need application-owned proxy routes. See the Backend contract.

Prefer to compose your own layout? Drop down to the individual building blocks — each has a deep-dive section further down the page:

  • RhombusBufferedPlayer — MPEG-DASH live & VOD on a real <video> element; native pause/seek, widest browser support.
  • RhombusRealtimePlayer — sub-second live H.264 over WebSocket, decoded with WebCodecs onto a <canvas> (live only).
  • RhombusAudioPlayer — A100/DR40 live Opus and historical audio, standalone or synchronized with video.
  • RhombusTalkback — send the browser microphone to an A100 or DR40, with Console-aligned RBAC/license/config policy.

Audio full-stack quick start

This is the recommended starting point when a page needs the complete audio feature set:

  • live and historical A100 or DR40 listening;
  • video and audio driven by one epoch-ms timeline;
  • browser-microphone talkback;
  • automatic A100/DR40 click-to-talk versus hold-to-talk behavior;
  • optional talkback blocking while the operator is viewing history;
  • matching-source echo suppression while the operator is speaking; and
  • automatic DR40 ownership handoff so embedded video audio is not played twice.
import {
  RhombusMediaPlayer,
  type RhombusAudioSource,
} from "@rhombussystems/react";

type AudioStationProps = {
  audioSource: RhombusAudioSource;
  /** Omit for an audio-only page. */
  cameraUuid?: string;
};

export function AudioStation({ audioSource, cameraUuid }: AudioStationProps) {
  return (
    <RhombusMediaPlayer
      cameraUuid={cameraUuid}
      audioSource={audioSource}
      apiOverrideBaseUrl="/"
    />
  );
}

Use it with an A100 audio gateway:

<AudioStation
  cameraUuid="CAMERA_UUID"
  audioSource={{ type: "audio-gateway", uuid: "A100_AUDIO_GATEWAY_UUID" }}
/>

Use it with a DR40:

<AudioStation
  cameraUuid="DR40_DEVICE_UUID"
  audioSource={{ type: "dr40", uuid: "DR40_DEVICE_UUID" }}
/>

For a DR40, the video cameraUuid and audioSource.uuid must be the same device UUID for automatic audio ownership handoff — RhombusMediaPlayer then also infers deviceType: "doorbell" for the video participant, so its media resolves through /doorbellcamera/getMediaUris. (When composing RhombusPlayer yourself, pass deviceType="doorbell" explicitly.) An A100 uses its audio gateway UUID, which is normally different from the camera UUID. RhombusMediaPlayer creates and shares the controller automatically: the video timeline seeks both streams, the talkback control knows whether the page is live or historical, and matching incoming far-audio is suppressed while speaking. Use the lower-level composition recipe when your layout needs independent placement of those participants.

Your application backend must expose these routes (defaults shown):

Browser route Purpose Rhombus upstream
POST /api/federated-token Mint a short-lived browser token. /org/generateFederatedSessionToken
POST /api/audio-media-uris Resolve A100 or DR40 live/VOD media URIs. /audiogateway/getMediaUris or /doorbellcamera/getMediaUris
POST /api/audio-talkback-capabilities Normalize device-scope authorization, Enterprise licensing, speaker configuration, connectivity, and interaction mode. Accessible-device inventory, getConfig, and /license/getDeviceLicenses

The backend keeps the API key secret. The realtime audio WebSocket separately authenticates the federated token and must authorize its permission group for the requested device; the capability route is a user-interface policy check, not the WebSocket security boundary. Copyable route contracts are in Backend contract.

Before testing, confirm:

  1. The server-side API key can see the selected device through its assigned permission group.
  2. The A100/DR40 has an Enterprise device license, its speaker is enabled, and it is online.
  3. The federated token is minted for the browser's deployed domain.
  4. The page is served over HTTPS (or localhost), microphone access is allowed by the browser, operating system, and iframe policy, and the media/worker hosts are allowed by CSP.
  5. The user explicitly unmutes listening audio and starts talkback from a click, tap, or keyboard gesture so browser media activation succeeds.

Talkback is allowed while viewing historical footage by default. Set disableTalkbackInVod when the operator must return to live before speaking. Talkback always reaches the physical device now; it is never scheduled at the historical playhead.

Choosing a component

Component Transport Live latency Live Past (VOD) Controls
RhombusMediaPlayer unified video + A100/DR40 audio + talkback sub-second live ✅ ✅ ✅ complete experience
RhombusPlayer both — realtime canvas for live, DASH for VOD, switched automatically sub-second live ✅ ✅ ✅ full bar + ref API
RhombusAudioPlayer live Opus + historical DASH/decoded Opus sub-second live ✅ ✅ ✅ full bar + ref API
RhombusTalkback browser microphone → PCM16/WebSocket → A100/DR40 speaker sub-second ✅ n/a ✅ mic control + ref API
RhombusBufferedPlayer MPEG-DASH (Dash.js) on a <video> ~few seconds ✅ ✅ native <video>
RhombusRealtimePlayer H.264 / WebSocket → WebCodecs → <canvas> sub-second ✅ ❌ none (always live)
Timeline none — a canvas scrubber you pair with any media — — — seek UI only

Rule of thumb: start with RhombusMediaPlayer for the complete experience. Reach for RhombusPlayer, RhombusAudioPlayer, and RhombusTalkback separately when the application needs custom participant placement or lifecycle. Give separate participants the same shared controller when playback time, VOD talk policy, and two-way-audio echo handling must coordinate.

RhombusMediaPlayer — complete video, audio, and talkback

RhombusMediaPlayer is the high-level facade for the most common integration. It composes the existing video, audio, talkback, and controller implementations; it does not introduce a second transport stack. The required audioSource name is deliberately explicit because cameraUuid identifies a different, optional video participant.

Identity props: why audioSource is required

The two identity props name two different participants:

Prop Identifies When required
audioSource The A100 audio gateway or DR40 used for incoming audio and talkback. Always
cameraUuid The optional camera that supplies video and the shared timeline. Only for video + audio

Therefore omitting cameraUuid means audio-only; it does not make audioSource generic. For a DR40 A/V station, use the same DR40 UUID for both props. For an A100 station, audioSource.uuid is the A100 gateway UUID and cameraUuid is the separately chosen camera.

Copy-paste defaults

The minimum audio-only experience is:

import { RhombusMediaPlayer } from "@rhombussystems/react";

<RhombusMediaPlayer
  audioSource={{ type: "audio-gateway", uuid: "A100_AUDIO_GATEWAY_UUID" }}
  apiOverrideBaseUrl="/"
/>;

Add cameraUuid for synchronized video:

<RhombusMediaPlayer
  cameraUuid="CAMERA_UUID"
  audioSource={{ type: "audio-gateway", uuid: "A100_AUDIO_GATEWAY_UUID" }}
  apiOverrideBaseUrl="/"
/>;

Those examples intentionally pass no control configuration, participant overrides, custom styles, callbacks, or external controller. Out of the box, the facade supplies:

  • the complete audio toolbar and audio timeline on an audio-only page;
  • the video toolbar and its single shared timeline when cameraUuid is present;
  • only volume control on the separate audio row when video owns the timeline;
  • listening audio, initially muted for browser autoplay compatibility;
  • talkback with the device-resolved click-to-talk or hold-to-talk interaction;
  • live and historical playback with synchronized timeline seeks;
  • automatic matching-DR40 audio ownership handoff; and
  • a private shared playback controller that requires no application wiring.

The sibling rhombus-react-example project exposes /media-player as a dedicated out-of-the-box test page. Its selectors are only test-fixture UI; the rendered RhombusMediaPlayer receives exactly the props shown above. Use its /audio page for the advanced customization and diagnostics lab.

Defaults:

  • talkback is rendered and follows the server-resolved hold/toggle policy;
  • talkback remains available in VOD and always targets the live physical device;
  • one private RhombusPlaybackController coordinates every participant;
  • without video, audio renders its complete controls and wall-clock timeline;
  • with video, the video owns the timeline and audio renders volume controls only;
  • matching DR40 video/audio automatically hands buffered/VOD audio ownership to video;
  • listening starts muted, volume 1, playing, live, and at rate 1; and
  • the facade uses the same token, endpoint, network, recovery, and error props for every participant.

Set talkback={false} for playback only. Set disableTalkbackInVod when operators must return to the live edge before speaking.

RhombusMediaPlayer props

Prop Type Default Purpose
audioSource RhombusAudioSource required A100 audio gateway or DR40 used for listening and talkback.
cameraUuid string — Optional synchronized video participant. Omit for audio-only.
deviceType `"camera" "doorbell"` inferred
apiOverrideBaseUrl and shared media props RhombusMediaBaseProps SDK defaults Applied consistently to video, audio, and talkback.
playbackController RhombusPlaybackController private controller Join an existing playback group instead of creating one.
playbackOptions RhombusPlaybackControllerOptions controller defaults Seed the private controller's mode, time, play state, rate, volume, mute, and timeline behavior.
talkback boolean true Render or omit microphone talkback.
disableTalkbackInVod boolean false Block and immediately stop TX while viewing history.
videoProps RhombusMediaPlayerVideoProps — Video-specific controls, quality, fit, callbacks, and styling.
audioProps RhombusMediaPlayerAudioProps contextual controls Audio-specific controls, VOD window, callbacks, and styling.
talkbackProps RhombusMediaPlayerTalkbackProps — Talkback interaction, microphone, capability, callback, and styling overrides.
className / style React root styling — Customize the facade root.
classNames / styles facade slot maps — Customize root, video, audio, and talkback slots. Inline values override defaults.
onError / onRecoveryAttempt shared callbacks — Receive failures/recovery from every participant.

The nested prop types intentionally omit participant identity, shared authentication, controller-owned playback state, and the shared controller itself. This prevents a nested override from silently splitting the group. Use playbackOptions to seed internal state:

<RhombusMediaPlayer
  cameraUuid={cameraUuid}
  audioSource={audioSource}
  apiOverrideBaseUrl="/"
  disableTalkbackInVod
  playbackOptions={{
    initialMuted: true,
    defaultRewindSec: 30,
  }}
  videoProps={{
    videoFit: "contain",
    timeline: { fetchSeekPoints: true },
  }}
  audioProps={{
    vodWindowSec: 30 * 60,
  }}
  talkbackProps={{
    microphoneGain: 5,
  }}
/>;

When cameraUuid is present, audioProps.controls defaults to ["volume"]. Pass an explicit list to change it. Without a camera, an omitted control list renders the full audio toolbar and timeline.

Styling and imperative access

The facade uses zero-specificity defaults on:

  • .rhombus-media-player
  • .rhombus-media-player-video
  • .rhombus-media-player-audio
  • .rhombus-media-player-talkback

Normal CSS overrides those classes. classNames adds design-system classes, while styles sets inline styles on the corresponding slots. The root also exposes data-rhombus-media-has-video, data-rhombus-media-audio-source, and data-rhombus-media-talkback for state-aware selectors:

<RhombusMediaPlayer
  audioSource={audioSource}
  className="security-station"
  classNames={{ talkback: "security-station-microphone" }}
  styles={{
    root: { gap: 16 },
    talkback: { borderColor: "var(--brand-border)" },
  }}
/>;

The RhombusMediaPlayerHandle exposes the shared playbackController plus getVideoPlayer(), getAudioPlayer(), and getTalkback(). The getters return each existing participant handle or null when that participant is disabled:

import { useRef } from "react";
import {
  RhombusMediaPlayer,
  type RhombusMediaPlayerHandle,
} from "@rhombussystems/react";

const media = useRef<RhombusMediaPlayerHandle>(null);

<RhombusMediaPlayer ref={media} audioSource={audioSource} />;

<button onClick={() => media.current?.playbackController.goLive()}>
  Go live
</button>

Use the lower-level components when participants must render in unrelated parts of the DOM, when separate controllers are intentional, or when application-specific layout exceeds the facade's video → audio → talkback ordering.

RhombusPlayer — the unified player

RhombusPlayer composes RhombusRealtimePlayer and RhombusBufferedPlayer behind one interface and adds player-level controls: play/pause, go-live, rewind, playback speed, digital zoom + pan, snapshot, an event-aware timeline, and save clip. It automatically switches between Live and VOD as the user interacts with the timeline and Go-Live button.

import { RhombusPlayer } from "@rhombussystems/react";

<RhombusPlayer
  cameraUuid="YOUR_CAMERA_UUID"
  apiOverrideBaseUrl="https://your-api.example.com"
  showLiveTypeSwitcher            // optional Console-style Realtime/Buffered + quality menu
  saveClip={{ defaultTitle: "Door cam" }}
  timeline={{ fetchSeekPoints: true }}   // 24h day window by default, ±12h chevrons
  onModeChange={(mode, atMs) => console.log(mode, new Date(atMs))}
/>

How Live ⇄ VOD switching works

Switching is a pure function of time vs. now:

  • Live uses the realtime transport by default (RhombusRealtimePlayer, WebCodecs canvas, sub-second). It auto-falls back to buffered DASH when WebCodecs is unavailable.
  • Pause, rewind, change speed, or seek into the past drops the player into VOD (RhombusBufferedPlayer anchored on a manifest window containing the target time).
  • Go Live (or seeking within liveEdgeToleranceSec of now) returns to the live edge.

Only one transport is mounted at a time, so a switch costs one brief reconnect (no double bandwidth). Seeking within the loaded VOD window is instant (native <video> seek); seeking outside it loads a fresh manifest window. A seek preserves the play/pause state: if playback was paused, it stays paused at the new time; if playing, it keeps playing (seeking to the live edge always resumes, since realtime live cannot be paused).

RhombusPlayer props

Every prop RhombusPlayer accepts. Only cameraUuid is required; everything else is optional. (The auth / endpoint / resilience props are the shared base props common to all players.)

Prop Type Required Default Notes
cameraUuid string ✅ — Camera UUID from Rhombus. Safe in the browser. For a DR40 this is the doorbell's device UUID.
deviceType `"camera" "doorbell"` — "camera"
connectionMode `"wan" "lan"` — "wan"
apiOverrideBaseUrl string — — Base for the token and media requests (proxy mode). Required for built-in Save Clip. When omitted, media is fetched directly from Rhombus.
rhombusApiBaseUrl string — https://api2.rhombussystems.com/api Rhombus REST base when apiOverrideBaseUrl is omitted.
paths RhombusPlayerPaths — see backend Override video, audio, token, seekpoint, and availability routes.
federatedSessionToken string — — Supply & rotate your own token; the SDK skips its token endpoint.
tokenDurationSec number — 86400 Requested token TTL (SDK-managed mode).
headers HeadersInit — — Static headers for the token request (+ media when apiOverrideBaseUrl set).
getRequestHeaders `() => HeadersInit Promise<…>` — —
maxRetryIntervalMs number — 30000 Auto-recovery backoff ceiling. 0 disables.
stallTimeoutMs number — 12000 Stall watchdog. 0 disables.
playbackController RhombusPlaybackController — private controller Join one video and one audio participant. Controller playhead/rate/mute/volume state takes precedence over equivalent player props.
liveTransport `"realtime" "buffered"` — "realtime"
videoFit `"contain" "cover" "fill" "auto"`
onVideoFitChange (fit) => void — — Fired when the video-display fit changes.
playing boolean — — Controlled play/pause. Omit = uncontrolled (starts playing). Pair with onPlayingChange. See Controlled vs. imperative.
playbackRate number — — Controlled VOD speed (no-op while live). Pair with onPlaybackRateChange.
zoom number (1–4) — — Controlled digital zoom. Pair with onZoomChange.
positionMs number (epoch ms) — — Controlled playhead — seeks when its value changes; mode is derived (near now ⇒ live). Mirror onProgress/onSeek for two-way binding.
showLiveTypeSwitcher boolean — false Render the Console-style Realtime/Buffered + quality menu in the bar.
realtimeStreamQuality `"HD" "SD"` — "HD"
bufferedStreamQuality `"HIGH" "MEDIUM" "LOW"` —
applyBufferedStreamQuality boolean — true Set false to omit the _ds downscale.
initialMode `"live" "vod"` — "live"
initialStartTimeMs number (epoch ms) — — Anchor used when initialMode="vod".
vodWindowSec number — 7200 Length of the VOD manifest window the SDK requests.
defaultRewindSec number — 15 Step used by the Rewind button / rewind().
liveEdgeToleranceSec number — 5 A seek within this many seconds of now counts as live.
autoGoLiveAtEdge boolean — false Auto-return to live when VOD playback catches up to the edge.
controls RhombusPlayerControl[] — undefined Which built-in controls to render. Leaving it undefined renders every control; [] = headless. There is no "all" value. See below.
classNames RhombusPlayerClassNames — — Per-slot class names for the bar. See Styling.
renderControls (api, state) => ReactNode — — Replace the bar entirely (timeline still renders).
saveClip RhombusSaveClipConfig — — Built-in clip export config. See Save Clip.
timeline RhombusPlayerTimelineConfig — — Timeline/scrubber config. See Timeline.
className / style string / CSSProperties — — Applied to the player's root element.
onReady () => void — — First underlying transport became ready.
onError (error: Error) => void — — Token / media / setup failure.
onRecoveryAttempt (attempt, error) => void — — Fires on each auto-recovery retry.
onModeChange (mode, atWallClockMs) => void — — Fired on every Live ⇄ VOD transition.
onTransportChange (transport) => void — — Resolved live transport changed (incl. WebCodecs fallback).
onSeek (wallClockMs, mode) => void — — A seek happened.
onProgress (wallClockMs, mode) => void — — Throttled playback progress (~4Hz VOD / ~1Hz live). Use to mirror a controlled positionMs.
onPlayingChange (playing) => void — — Play/pause state changed.
onPlaybackRateChange (rate) => void — — Playback speed changed.
onSnapshot (RhombusSnapshotResult) => void — — A snapshot was captured.
onZoomChange (zoom, panX, panY) => void — — Zoom/pan changed.
onClipRangeSelect (RhombusClipRange) => void — — User selected a clip range (fires regardless of built-in export).
onClipExport (RhombusClipExportStatus) => void — — Built-in clip export progress/result.

Controlled, uncontrolled & imperative

You don't have to choose one approach. Props, the ref, and the built-in control bar all read and write the same internal state, so they work together and stay in sync — drive some aspects declaratively and others imperatively, or let users click the built-in bar; every path fires the matching on*Change callback so your state can follow. (The only caveat is the standard React one: see "Notes" below.)

  1. Controlled value props — drive a steady-state value declaratively. Each is optional: omit it and the player owns it internally (uncontrolled, seeded by initial*/defaults); provide it (and update it from the matching on*Change) and it becomes the source of truth. The built-in controls and the ref still work — in controlled mode they fire on*Change so your state updates.
Prop Callback
playing onPlayingChange
playbackRate onPlaybackRateChange
zoom onZoomChange
liveTransport onTransportChange
videoFit onVideoFitChange
  1. Controlled playhead — positionMs (epoch ms). It seeks when its value changes (the player derives live vs. VOD: within liveEdgeToleranceSec of now ⇒ live in the current transport, else VOD — there is no mode prop). The player still advances on its own; for a two-way binding, mirror onProgress (throttled) and/or onSeek back into positionMs:
  2. Imperative actions — one-shot commands on the ref. Some are just sugar over a declarative prop (use whichever you prefer); two are strictly imperative because they return a value / run an async side-effect and have no meaningful "state" to bind:
ref method Declarative equivalent
play() / pause() playing
setPlaybackRate(r) playbackRate
zoomIn() / zoomOut() / setZoom() / resetZoom() zoom
setLiveTransport(t) liveTransport
seekTo(ms) / rewind(s) / goLive() positionMs (set to the time / now − s / now)
**snapshot()** none — strictly imperative (returns the captured frame).
**startClipExport(range?, opts?)** none — strictly imperative (clip capture; runs the async render, returns status). Clip range selection is the built-in UI / onClipRangeSelect, but the export itself is a command.

So: everything that has a steady-state value is available as a controlled prop; the only things that are ref-only are **snapshot()** and **startClipExport()** (and you'd typically also reach for getState() imperatively).

Notes (controlled semantics): when you provide a controlled prop, you own it — if you ignore its on*Change, the prop and the player can diverge until the next prop change (standard React controlled behavior; e.g. the built-in Pause button fires onPlayingChange(false), and if you don't update your playing state the player re-asserts your prop). getState() always returns the effective values regardless of how you drive it, and the ref works in controlled or uncontrolled mode.

Imperative handle (ref)

Pass a ref to drive the player programmatically. The built-in control bar uses this exact API internally, so anything the buttons do, you can do too.

import { useRef } from "react";
import { RhombusPlayer, type RhombusPlayerHandle } from "@rhombussystems/react";

function Controlled() {
  const player = useRef<RhombusPlayerHandle>(null);
  return (
    <>
      <RhombusPlayer ref={player} cameraUuid="…" apiOverrideBaseUrl="https://api.example.com" />
      <button onClick={() => player.current?.pause()}>Pause</button>
      <button onClick={() => player.current?.goLive()}>Go live</button>
      <button onClick={() => player.current?.rewind(30)}>« 30s</button>
      <button onClick={() => player.current?.seekTo(Date.now() - 3_600_000)}>1h ago</button>
      <button onClick={async () => {
        const shot = await player.current?.snapshot();
        if (shot) downloadDataUrl(shot.dataUrl, "frame.png");
      }}>Snapshot</button>
    </>
  );
}
Method Description
play() / pause() Play / pause. Pausing live drops into a frozen VOD frame.
goLive() Return to the live edge (restores the live transport).
seekTo(wallClockMs) Seek to an absolute time (epoch ms); auto-switches Live ⇄ VOD.
rewind(seconds?) Jump back seconds (default defaultRewindSec).
setPlaybackRate(rate) VOD only; ignored while live.
zoomIn(step?) / zoomOut(step?) Digital zoom (1×–4×).
setZoom(zoom, panX?, panY?) / resetZoom() Set zoom + pan directly / reset to 1×.
snapshot() Promise<RhombusSnapshotResult> — capture the current frame.
`setLiveTransport("realtime" "buffered")`
startClipExport(range?, options?) Promise<RhombusClipExportStatus> — export a clip (proxy mode).
getState() Current RhombusPlayerState snapshot.

Observable state

renderControls(api, state) receives — and getState() returns — a RhombusPlayerState:

type RhombusPlayerState = {
  cameraUuid: string;
  mode: "live" | "vod";
  liveTransport: "realtime" | "buffered";  // resolved (may have fallen back)
  playing: boolean;
  playbackRate: number;
  currentWallClockMs: number | null;        // ≈ Date.now() while live
  zoom: number;
  isAtLiveEdge: boolean;
  canSaveClip: boolean;                      // built-in export available (proxy mode)
  clipSelection: { startMs: number; endMs: number } | null; // current clip selection, or null
  clipExport?: RhombusClipExportStatus;      // in-progress / finished export
};

Choosing which controls render

controls is a list of RhombusPlayerControl. It's exported both as a string union and as a runtime constant (RhombusPlayerControl.Play, etc.), so use plain strings or named members — whichever you prefer:

"play" | "goLive" | "rewind" | "speed" | "zoom" | "snapshot" | "saveClip" | "timeline" | "liveType" | "videoFit" | "goToDate"
import { RhombusPlayer, RhombusPlayerControl } from "@rhombussystems/react";

<>
  {/* All controls — omit the prop entirely: */}
  <RhombusPlayer cameraUuid="…" />

  {/* A subset — plain strings: */}
  <RhombusPlayer cameraUuid="…" controls={["play", "timeline"]} />

  {/* …or the named constant (autocompletes, refactor-safe): */}
  <RhombusPlayer
    cameraUuid="…"
    controls={[RhombusPlayerControl.Play, RhombusPlayerControl.Timeline]}
  />

  {/* Headless — no built-in UI at all; drive everything through the ref: */}
  <RhombusPlayer ref={player} cameraUuid="…" controls={[]} />
</>

Go to date ("goToDate" control / RhombusDateTimePicker)

The toolbar includes a date/time jump picker (the "goToDate" control, on by default): a calendar + time-of-day popover that seeks the player to any moment — the SDK counterpart of the Rhombus Console's toolbar date picker. When footage availability is enabled, days with no recorded footage are struck through and disabled (one getPresenceWindows fetch per viewed month, cached; failures just leave days enabled).

The component is also exported standalone for custom layouts — it interops with any player via seekTo:

import { RhombusDateTimePicker } from "@rhombussystems/react";

<RhombusDateTimePicker
  value={positionMs}                              // epoch ms (or null)
  onChange={(ms) => playerRef.current?.seekTo(ms)}
  cameraUuid="…"                                  // optional: enables no-footage day disabling
  apiOverrideBaseUrl="https://your-api.example.com"
  minTimeMs={retentionFloorMs}                    // optional: disable pre-retention days
  direction="down"                                // "up" for bottom toolbars (player default)
/>

All calendar math is in the viewer's local time zone (consistent with the Timeline). Style it via the rhombus-datepicker-* classes (zero-specificity defaults, like the control bar) or the classNames={{ anchor, popover }} prop.

Inside RhombusPlayer, a picker jump that lands outside the visible timeline window also re-centers the timeline on the target (in-window seeks — i.e. timeline clicks — never move the window). In custom layouts composing the standalone picker with a standalone Timeline, you own the window: update your rangeStartMs/rangeEndMs in the same onChange that calls seekTo.

Video display / fit

Cameras are usually 16:9; when the player box isn't, you get letter/pillar-boxing. The videoFit prop controls how the footage fills its area, mirroring the Rhombus Console video-wall "Video Display" options:

videoFit Console label Behavior
"auto" (default) Auto-Size The player box takes the video's aspect ratio — no bars, no cropping.
"contain" Default Aspect Ratio Full frame, letter/pillar-boxed (object-fit: contain).
"cover" Full View Cropped Fills the box, crops overflow (object-fit: cover).
"fill" Stretch to Fit Distorts to fill, no cropping (object-fit: fill).

There's a built-in video-display control in the bar (the "videoFit" control) so users can switch between these live; it fires onVideoFitChange. You can also drive it as a controlled prop:

<RhombusPlayer cameraUuid="…" videoFit="cover" onVideoFitChange={setFit} />

**"auto" sizes by width:** the player measures the video's intrinsic aspect ratio and sets the stage's aspect-ratio (height is derived), so give the player a width and don't impose a fixed height in that mode. The other three modes fill whatever box you give it.

For the low-level RhombusBufferedPlayer / RhombusRealtimePlayer, set object-fit yourself via videoProps.style / canvasProps.style.

Styling the controls

Three options, least → most custom:

1. Plain CSS overrides. The bar uses stable class names, and the SDK ships its defaults as a zero-specificity :where() stylesheet injected once at runtime. Because every default selector sits inside :where() (specificity 0,0,0), your CSS always wins — no !important, no import, regardless of load order:

Element class
the bar rhombus-player-controls
every button rhombus-player-btn (active: [data-active="true"]; disabled: :disabled)
speed <select> rhombus-player-speed
quality <select> rhombus-player-quality
live-type group rhombus-player-livetype
clip group rhombus-player-clip
clip status text rhombus-player-clip-status
timeline wrapper rhombus-player-timeline 
.rhombus-player-controls { background: #fff; color: #111; gap: 12px; }
.rhombus-player-btn { background: #0a7; border-color: #0a7; border-radius: 999px; }
.rhombus-player-btn[data-active="true"] { outline: 2px solid #0a7; }

2. classNames prop — attach your own class per slot (Tailwind, CSS-modules, design systems). Appended to the SDK's class on that element:

<RhombusPlayer
  cameraUuid="…"
  classNames={{ controls: "flex gap-3 p-2 bg-white", button: "btn btn-sm", clip: "ml-auto" }}
/>

3. renderControls — replace the bar entirely (the timeline still renders above it) and build your own buttons against the imperative api:

<RhombusPlayer
  cameraUuid="…"
  renderControls={(api, s) => (
    <div className="my-bar">
      <button onClick={() => (s.playing ? api.pause() : api.play())}>
        {s.playing ? "Pause" : "Play"}
      </button>
      {s.mode === "vod" && <button onClick={() => api.goLive()}>Go live</button>}
      <button onClick={() => api.rewind()}>« 15s</button>
      <button disabled={s.mode === "live"} onClick={() => api.setPlaybackRate(2)}>2×</button>
      <button onClick={() => api.zoomIn()}>＋</button>
      <button onClick={() => void api.snapshot()}>Snapshot</button>
    </div>
  )}
/>

renderControls is fully optional — omit it to keep the built-in bar. For total control, combine controls={[]} (no bar) with the ref handle and your own layout.

Snapshots

The Snapshot tool captures the current frame and hands the image data back to you — it does not auto-download and there is no target container/ref to render into. It works in both modes (the realtime canvas and the MSE-fed DASH <video> are both untainted, so toDataURL / toBlob succeed) and returns a RhombusSnapshotResult:

type RhombusSnapshotResult = {
  dataUrl: string;   // PNG data: URL
  blob: Blob | null; // PNG blob (null only if toBlob is unavailable)
  wallClockMs: number;
  mode: "live" | "vod";
  width: number;
  height: number;
};

You receive it two ways — both deliver the same result, including for the built-in Snapshot button:

// 1) Callback — fires for the built-in button AND for api.snapshot()
<RhombusPlayer cameraUuid="…" onSnapshot={(shot) => setPreview(shot.dataUrl)} />

// 2) Imperative — capture on demand and use the returned result
const shot = await playerRef.current!.snapshot();

The SDK never downloads or displays the image itself — render it (<img src={shot.dataUrl} />), upload shot.blob, or trigger a download yourself:

const shot = await playerRef.current!.snapshot();
const a = document.createElement("a");
a.href = shot.dataUrl;                          // or URL.createObjectURL(shot.blob!)
a.download = `snapshot-${shot.wallClockMs}.png`;
a.click();

A common pattern is to store onSnapshot's dataUrl in state and render a thumbnail (<img src={dataUrl} />). For lower-level use, snapshotCanvasElement / snapshotVideoElement are exported too.

Save Clip

The clip flow is drag-to-select on the timeline, then export:

  1. **✂ Create clip** in the bar enters clip mode — it seeds a selection at the playhead and zooms the timeline so it's easy to adjust.
  2. On the timeline you get a shaded region with draggable start/end handles, a draggable body (move the whole range), and a live duration label. The selection clamps to a minimum (default 5s), a maximum (default 60 min — the server cap), and never includes the future.
  3. **Save clip** opens a small title / description / visibility form (skippable — set saveClip.showOptionsForm: false), then runs the export: /video/spliceV3 → progress polling → a download URL.

onClipRangeSelect({ startMs, endMs, cameraUuid }) fires as the selection changes, and onClipExport(status) reports progress/result.

Proxy mode required for export. The clip endpoints are API-key / session authed, not federated-token compatible, so the request must go through your backend (which attaches the API key) — exactly like the media-URI proxy. Built-in export is only available when apiOverrideBaseUrl is set; selection + onClipRangeSelect work regardless. See the Backend contract.

<RhombusPlayer
  cameraUuid="…"
  apiOverrideBaseUrl="https://your-api.example.com"
  saveClip={{ defaultDurationSec: 30, defaultVisibility: "PRIVATE", showOptionsForm: true }}
  onClipExport={(s) => {
    if (s.phase === "rendering") setProgress(s.percentComplete);
    if (s.phase === "complete") window.location.assign(s.downloadUrl!);
  }}
/>
type RhombusSaveClipConfig = {
  enabled?: boolean;           // default true when apiOverrideBaseUrl is set
  paths?: { splice?: string; progress?: string; download?: string };
  defaultTitle?: string;
  defaultDurationSec?: number; // seeded selection width. Default 60
  minDurationSec?: number;     // drag clamp. Default 5
  maxDurationSec?: number;     // drag clamp. Default 3600 (server caps at 60 min)
  progressTimeoutMs?: number;  // give up polling a stuck render. Default 300000 (5 min); 0 = never
  defaultVisibility?: RhombusClipVisibility; // "ORG_WIDE" (default) | "PRIVATE" | "ROLE_RESTRICTED"
  showOptionsForm?: boolean;   // show the title/description/visibility form. Default true
  requireFootage?: "any" | "full" | "off"; // footage pre-check policy. Default "any" (see below)
};

type RhombusClipExportOptions = {
  title?: string;
  description?: string;
  visibility?: RhombusClipVisibility;
  saveToConsole?: boolean;     // default true
  audioIncluded?: boolean;     // also splices the camera's .a0 audio facet
};

type RhombusClipExportStatus = {
  phase: "selecting" | "submitting" | "rendering" | "complete" | "error" | "canceled";
  clipUuid?: string;
  percentComplete?: number;  // 0–100 while rendering
  currentOperation?: string;
  downloadUrl?: string;      // set when complete
  error?: string;
  errorCode?: "no-footage" | "partial-footage"; // set when the footage pre-check blocked the export
  coverage?: RhombusRangeCoverage;              // footage coverage of the range, when the check ran
};

Footage pre-check (requireFootage)

Rhombus renders time ranges with no recorded footage (camera offline during the window, or footage past retention) as "VIDEO NOT AVAILABLE" placeholder frames — and /video/spliceV3 happily renders a clip over such a range, returning a "successful" clip with no real video in it. Before submitting an export, the player therefore fetches /camera/getPresenceWindows for the selected range and applies requireFootage:

  • "any" (default) — block only when the range has zero recorded footage.
  • "full" — block when the range has any confirmed gap.
  • "off" — no pre-check (legacy behavior).

A blocked export emits phase: "error" with errorCode ("no-footage" / "partial-footage") and coverage — key both your custom UI messages off errorCode, not the human-readable error string. Exports that proceed carry coverage on every subsequent status so you can warn about partial footage. The check fails open: if availability can't be fetched (missing proxy route, timeout), the export proceeds ungated and coverage is absent.

Build your own clip UI instead of the built-in form: read the live selection from onClipRangeSelect (or getState().clipSelection) and call the imperative handle with your own options:

await player.current!.startClipExport(
  { startMs, endMs, cameraUuid },
  { title: "Front door", visibility: "PRIVATE", audioIncluded: true }
);

Timeline configuration

RhombusPlayer renders a Timeline when controls includes "timeline" (the default). Configure it with the timeline prop:

type RhombusPlayerTimelineConfig = {
  windowSec?: number;        // span of the scrubber, seconds. Default 86400 (a full day)
  fetchSeekPoints?: boolean; // fetch event markers from /camera/getFootageSeekpointsV2. Default true
  includeAnyMotion?: boolean;
  fetchAvailability?: boolean; // fetch footage coverage from /camera/getPresenceWindows and draw
                               // no-footage gaps on the availability bar. Default: true in proxy
                               // mode (apiOverrideBaseUrl set), false in direct mode.
  onAvailabilityLoaded?: (availability: RhombusFootageAvailability) => void;
  marks?: TimelineMark[];    // extra static event bands / gaps
  colors?: TimelineColors;   // recolor seekpoints, bars, playhead, buttons (see below)
  height?: number;           // px, default 56
  onSeekPointsLoaded?: (points: RhombusFootageSeekPoint[]) => void; // diagnostics
};

Footage availability

With fetchAvailability on, the availability bar stops pretending all past time is recorded: ranges with confirmed no footage render in colors.availabilityGap (default a muted red) — the same ranges the Rhombus stream would play as "VIDEO NOT AVAILABLE" placeholder frames. Gaps are only drawn where the answer is actually known (inside the fetched range, in the past, and older than a ~2-minute live-edge grace window for presence-ingest lag); everything else keeps the legacy look. The in-clip-mode toolbar also shows a warning (and disables Save, per requireFootage) when the selection overlaps a gap.

The raw client + coverage math are exported for custom UIs: fetchPresenceWindows, mergeFootageWindows, computeFootageGaps, computeRangeCoverage, and the RhombusFootageWindow / RhombusFootageAvailability / RhombusRangeCoverage types. Windows carry source: "cloud" | "local" — local windows live on the camera's SD card and are only retrievable while the camera is online; cloud windows are always retrievable.

By default the window is a 24h span aligned to local midnight (Console-style). RhombusPlayer renders ‹/› chevrons that pan by half a span (±12h at the day view), and −/+ zoom buttons + mouse-wheel zoom that step through 24h → 8h → 3h → 1h → 20m → 5m (centered on the cursor or playhead, with an animated transition) so you can pinpoint a moment when seekpoints bunch up. It auto-follows the current day/playhead until you navigate; Go Live resets to the day view.

The player keeps the window stable while you scrub and only scrolls it once playback leaves the visible range, so the playhead always lands exactly where you click.

RhombusPlayer recipes

Open straight into an event (past footage):

<RhombusPlayer
  cameraUuid="…"
  apiOverrideBaseUrl="https://api.example.com"
  initialMode="vod"
  initialStartTimeMs={new Date("2025-04-15T09:30:00Z").getTime()}
/>

Auto-return to live when caught up, wider rewind step:

<RhombusPlayer cameraUuid="…" autoGoLiveAtEdge defaultRewindSec={30} />

Force broadest browser support (buffered live everywhere):

<RhombusPlayer cameraUuid="…" liveTransport="buffered" />

Headless — your UI, our engine:

function MyPlayer() {
  const ref = useRef<RhombusPlayerHandle>(null);
  const [state, setState] = useState<RhombusPlayerState>();
  return (
    <>
      <RhombusPlayer ref={ref} cameraUuid="…" controls={[]} onModeChange={() => setState(ref.current?.getState())} />
      {/* render your own toolbar from `state` and `ref.current` */}
    </>
  );
}

RhombusAudioPlayer — A100 and DR40 audio

RhombusAudioPlayer is the drop-in audio equivalent of RhombusPlayer. It plays an A100 audio gateway or DR40 by absolute wall-clock time, switches between live and historical audio, and can either own its controls or participate in the same controller and timeline as video.

Use it for:

  • standalone live A100 or DR40 listening;
  • standalone historical audio with rewind, pause, speed, and epoch-ms seeking;
  • synchronized camera video plus independent A100 audio;
  • synchronized DR40 video/audio without downloading or playing the DR40 audio twice;
  • a headless audio engine controlled by application UI;
  • one external timeline controlling both a video and audio participant.

Use RhombusTalkback alongside this component for full two-way audio. DSP/equalizer controls and multiple simultaneous audible tracks on one shared controller remain outside the public API.

Audio sources and device UUIDs

Sources are explicit so the SDK can select the correct Rhombus endpoint and request body:

import type { RhombusAudioSource } from "@rhombussystems/react";

const a100: RhombusAudioSource = {
  type: "audio-gateway",
  uuid: "A100_AUDIO_GATEWAY_UUID",
};

const dr40: RhombusAudioSource = {
  type: "dr40",
  uuid: "DR40_DEVICE_UUID",
};
  • "audio-gateway" calls /audiogateway/getMediaUris with { gatewayUuid: source.uuid }.
  • "dr40" calls /doorbellcamera/getMediaUris with { deviceUuid: source.uuid }.
  • An A100 UUID is the audio gateway UUID, not an associated camera UUID.
  • A source must be visible to the organization represented by the API key or federated token. An unknown or cross-organization UUID produces no usable media URIs.

The SDK intentionally does not include a device-list UI. Most applications select devices from their own inventory. A backend-powered picker can use Rhombus /audiogateway/getMinimalAudioGatewayStateList for A100s and /doorbellcamera/getMinimalStateList for DR40s; keep the API key server-side.

Standalone live audio

The smallest complete player is:

import { RhombusAudioPlayer } from "@rhombussystems/react";

export function LobbyAudio() {
  return (
    <RhombusAudioPlayer
      source={{ type: "audio-gateway", uuid: "A100_UUID" }}
      apiOverrideBaseUrl="https://your-api.example.com"
    />
  );
}

Omit apiOverrideBaseUrl only when your application uses direct federated-token mode.

The uncontrolled defaults are:

  • mode "live";
  • playing true;
  • muted true (for reliable browser autoplay);
  • volume 1 on a linear 0–1 scale;
  • playback rate 1;
  • 15-second rewind;
  • five-second live-edge tolerance;
  • no automatic return to live when historical playback reaches the edge.

The user must select Unmute to hear audio. Play and unmute actions resume Web Audio synchronously inside the user gesture so Chrome, Edge, Safari, and iOS can authorize output. Do not default a production page to audible autoplay: browsers may block it and users generally expect embedded surveillance media to start silent.

Use a DR40 the same way:

<RhombusAudioPlayer
  source={{ type: "dr40", uuid: dr40Uuid }}
  apiOverrideBaseUrl="/"
/>

apiOverrideBaseUrl="/" uses the default proxy routes on the current origin. The shorter recipes below omit repeated auth props where they are not the subject; use the same proxy configuration in production unless the recipe explicitly demonstrates direct mode.

Live and historical audio transports

The component selects an internal transport; consumers normally observe it but do not choose it directly:

state.transport When it is used Output path
"opus-live" Live A100/DR40 audio Opus WebSocket → worker-backed WASM decoder → Web Audio
"dash-vod" Historical audio when Opus/WebM MSE is supported Dash.js → hidden <audio> element
"decoded-vod" Safari/iOS capability fallback or recoverable DASH failure Rhombus two-second Opus segments → WASM → Web Audio
"embedded-dr40" A matching DR40 video owns buffered/VOD audio The RhombusPlayer video element

Live A100 and DR40 streams carry Opus frames at mono 48 kHz. The player parses Rhombus TLV records, tracks their epoch-ms timestamps, maintains a bounded jitter buffer, and discards isolated non-TLV A100 startup packets. Repeated malformed packets are treated as stream corruption and trigger recovery.

Historical mode prefers MSE/Dash.js. If MediaSource.isTypeSupported('audio/webm; codecs="opus"') is false—or DASH initialization fails—the decoded transport fetches the Rhombus segment format, keeps roughly ten seconds buffered, and schedules PCM through the same Web Audio graph as live playback.

Start directly in history:

const fiveMinutesAgo = Date.now() - 5 * 60_000;

<RhombusAudioPlayer
  source={{ type: "audio-gateway", uuid: a100Uuid }}
  apiOverrideBaseUrl="https://your-api.example.com"
  initialMode="vod"
  initialStartTimeMs={fiveMinutesAgo}
  vodWindowSec={30 * 60}
/>

All public positions are Unix epoch milliseconds. MPD template parameters are converted to seconds internally. If initialMode="vod" is supplied without initialStartTimeMs, the initial target is approximately one minute before now.

RhombusAudioPlayer props

source is the only audio-specific required prop. Authentication and resilience properties come from RhombusMediaBaseProps.

Prop Type Default Purpose
source RhombusAudioSource required A100 or DR40 identity.
connectionMode "wan" | "lan" "wan" Select WAN or the first usable LAN media URI.
apiOverrideBaseUrl string direct mode Base URL for token and audio-media proxy routes. Proxy mode is recommended.
rhombusApiBaseUrl string Rhombus production API Direct-mode REST base.
paths RhombusPlayerPaths SDK defaults Override token, proxy, A100-direct, or DR40-direct paths.
federatedSessionToken string SDK-managed Supply and rotate a token yourself.
tokenDurationSec number 86400 Requested SDK-managed token TTL.
headers HeadersInit — Static headers for your token/proxy endpoints.
getRequestHeaders () => HeadersInit | Promise<HeadersInit> — Resolve fresh application headers per request.
maxRetryIntervalMs number 30000 Live reconnect backoff ceiling; 0 disables reconnects.
stallTimeoutMs number 12000 Recover a live socket with no messages; 0 disables.
playbackController RhombusPlaybackController private controller Join video/audio/timeline playback state.
playing boolean true Controlled play/pause without an external controller.
positionMs number current/initial time Controlled epoch-ms playhead without an external controller.
playbackRate number 1 Controlled historical playback rate.
muted boolean true Controlled mute state.
volume number 1 Controlled linear gain, clamped to 0–1.
initialMode "live" | "vod" "live" Initial uncontrolled mode.
initialStartTimeMs number about 60s ago in VOD Initial historical wall-clock target.
vodWindowSec number 7200 Requested historical manifest/segment window.
defaultRewindSec number 15 Default rewind() and control-bar step.
liveEdgeToleranceSec number 5 A seek this close to now becomes live.
autoGoLiveAtEdge boolean false Return to live when VOD catches the edge.
controls RhombusAudioPlayerControl[] all Select built-ins; [] is fully headless.
renderControls (api, state) => ReactNode built-in bar Replace the audio control bar.
classNames RhombusAudioPlayerClassNames — Add classes to control-bar slots.
timeline RhombusPlayerTimelineConfig 24h window Configure audio timeline span, marks, colors, and height.
className / style React root props — Style the component root.
onReady () => void — Owned live socket or Dash.js VOD is ready; may repeat after reconnect/reinit. Use status callbacks for decoded/embedded audio.
onError (error: Error) => void — Token, URI, decoder, segment, or transport failure.
onRecoveryAttempt (attempt, error) => void — Live socket retry notification.

If playbackController is supplied, it takes precedence over playing, positionMs, playbackRate, muted, and volume. Passing both is allowed for migration, but the SDK emits a development warning because the per-player values cannot win.

Built-in, selective, and custom audio controls

With controls omitted, the player renders rewind, play/pause, Go Live, speed, mute/volume, and a wall-clock timeline. Select only the controls your layout needs:

import {
  RhombusAudioPlayer,
  RhombusAudioPlayerControl,
} from "@rhombussystems/react";

<RhombusAudioPlayer
  source={source}
  controls={[
    RhombusAudioPlayerControl.Play,
    RhombusAudioPlayerControl.Volume,
    RhombusAudioPlayerControl.Timeline,
  ]}
/>;

// Plain strings are equally valid:
<RhombusAudioPlayer source={source} controls={["play", "volume"]} />;

Available identifiers are "play", "goLive", "rewind", "speed", "volume", and "timeline". The speed picker is disabled at the live edge; Go Live is disabled while live. Omitting "timeline" removes the timeline. controls={[]} renders no SDK UI.

Replace the bar but keep player-managed state:

<RhombusAudioPlayer
  source={source}
  renderControls={(api, state) => (
    <div className="my-audio-controls">
      <button onClick={() => (state.playing ? api.pause() : api.play())}>
        {state.playing ? "Pause" : "Play"}
      </button>
      <button onClick={() => api.setMuted(!state.muted)}>
        {state.muted ? "Unmute" : "Mute"}
      </button>
      <button onClick={() => api.rewind(30)}>Back 30s</button>
      <span>{state.status}</span>
    </div>
  )}
/>;

renderControls replaces the toolbar, but the built-in timeline still renders while "timeline" is selected (including the default controls={undefined}). Pass a control list without "timeline" if the custom layout renders its own timeline.

RhombusAudioPlayerControls and RhombusAudioPlayerControlsProps are also exported for layouts that store onStateChange state and render the stock bar elsewhere:

import { useState } from "react";
import {
  RhombusAudioPlayer,
  RhombusAudioPlayerControls,
  type RhombusAudioPlayerHandle,
  type RhombusAudioPlayerState,
} from "@rhombussystems/react";

function AudioWithDetachedControls() {
  const [api, setApi] = useState<RhombusAudioPlayerHandle | null>(null);
  const [state, setState] = useState<RhombusAudioPlayerState | null>(null);

  return (
    <>
      <RhombusAudioPlayer
        ref={setApi}
        source={{ type: "audio-gateway", uuid: "A100_UUID" }}
        apiOverrideBaseUrl="/"
        controls={[]}
        onStateChange={setState}
      />
      {api && state ? (
        <RhombusAudioPlayerControls
          api={api}
          state={state}
          controls={["play", "volume"]}
        />
      ) : null}
    </>
  );
}

The built-in audio timeline uses wall-clock time and does not fetch camera event seekpoints. Its supported inline configuration is windowSec, marks, colors, and height. For camera seekpoints, availability, window shifting, or zoom controls, render a standalone Timeline with the same playbackController.

Imperative and controlled audio playback

Use the ref API when commands originate from buttons, keyboard shortcuts, or application events:

import { useRef } from "react";
import {
  RhombusAudioPlayer,
  type RhombusAudioPlayerHandle,
  type RhombusAudioSource,
} from "@rhombussystems/react";

function ImperativeAudio({ source }: { source: RhombusAudioSource }) {
  const audio = useRef<RhombusAudioPlayerHandle>(null);

  return (
    <>
      <RhombusAudioPlayer ref={audio} source={source} controls={[]} />
      <button onClick={() => audio.current?.play()}>Play</button>
      <button onClick={() => audio.current?.pause()}>Pause</button>
      <button onClick={() => audio.current?.setMuted(false)}>Unmute</button>
      <button onClick={() => audio.current?.seekTo(Date.now() - 60_000)}>
        One minute ago
      </button>
      <button onClick={() => audio.current?.goLive()}>Go Live</button>
    </>
  );
}

The complete RhombusAudioPlayerHandle is:

Method Effect
play() / pause() Resume or suspend playback. User-triggered play() unlocks browser audio.
goLive() Switch to live, reset rate to 1, and resume.
seekTo(epochMs) Seek to absolute wall-clock time; near-now targets become live.
rewind(seconds?) Enter VOD and move back by the argument or defaultRewindSec.
setPlaybackRate(rate) Set historical rate.
setMuted(boolean) Mute/unmute; user-triggered unmute unlocks Web Audio.
setVolume(0to1) Set clamped linear gain.
getState() Read the most recent RhombusAudioPlayerState.

For React-controlled state, omit playbackController and pair each controlled prop with its change callback:

import { useState } from "react";
import {
  RhombusAudioPlayer,
  type RhombusAudioSource,
} from "@rhombussystems/react";

function ControlledAudio({ source }: { source: RhombusAudioSource }) {
  const [playing, setPlaying] = useState(true);
  const [positionMs, setPositionMs] = useState(Date.now());
  const [muted, setMuted] = useState(true);
  const [volume, setVolume] = useState(1);

  return (
    <RhombusAudioPlayer
      source={source}
      playing={playing}
      positionMs={positionMs}
      muted={muted}
      volume={volume}
      onPlayingChange={setPlaying}
      onProgress={atMs => setPositionMs(atMs)}
      onMutedChange={setMuted}
      onVolumeChange={setVolume}
    />
  );
}

Use a shared controller instead of duplicating controlled props when more than one media participant must move together.

Shared video/audio playback

useRhombusPlaybackController() owns the shared mode, epoch-ms position, play intent, playback rate, mute, volume, status, and explicit seek sequence. The current controller supports one video plus one audio playback participant. Use separate controllers for independently controlled players or multiple audible tracks.

Video is the historical clock authority when present; standalone audio is the authority otherwise. Explicit seeks, rewinds, and Go Live commands reach every participant. In VOD, a required participant reporting buffering temporarily pauses the group while preserving play intent. Audio corrects small drift with a bounded rate adjustment and resets for large drift.

Pair independent A100 audio with a camera:

import {
  RhombusAudioPlayer,
  RhombusPlayer,
  useRhombusPlaybackController,
} from "@rhombussystems/react";

function PairedPlayer() {
  const playback = useRhombusPlaybackController();
  return (
    <>
      <RhombusPlayer
        cameraUuid="CAMERA_UUID"
        apiOverrideBaseUrl="https://your-api.example.com"
        playbackController={playback}
      />
      <RhombusAudioPlayer
        source={{ type: "audio-gateway", uuid: "A100_UUID" }}
        apiOverrideBaseUrl="https://your-api.example.com"
        playbackController={playback}
        controls={["volume"]}
      />
    </>
  );
}

The timeline rendered by RhombusPlayer controls both participants. Rendering only controls={["volume"]} on the audio player avoids a duplicate timeline while retaining the audio output control.

An audio-only layout can use an external timeline:

import {
  RhombusAudioPlayer,
  Timeline,
  useRhombusPlaybackController,
  type RhombusAudioSource,
} from "@rhombussystems/react";

function AudioWithExternalTimeline({ source }: { source: RhombusAudioSource }) {
  const playback = useRhombusPlaybackController();
  const now = Date.now();

  return (
    <>
      <RhombusAudioPlayer
        source={source}
        playbackController={playback}
        controls={["volume"]}
      />
      <Timeline
        playbackController={playback}
        rangeStartMs={now - 60 * 60_000}
        rangeEndMs={now}
        fetchSeekPoints={false}
      />
    </>
  );
}

Timeline.cameraUuid is unnecessary when both fetchSeekPoints and fetchAvailability are false. If you enable fetched camera data, pass the relevant camera UUID and configure its proxy route.

Controller options seed the group's initial state and timeline behavior:

Option Default Purpose
initialMode "live" Initial live/VOD mode.
initialPositionMs now Initial epoch-ms playhead.
initialPlaying true Initial play intent.
initialPlaybackRate 1 Initial historical rate.
initialMuted true Initial group mute.
initialVolume 1 Initial linear gain, clamped to 0–1.
defaultRewindSec 15 Default shared rewind step.
liveEdgeToleranceSec 5 Distance from now that counts as live.
autoGoLiveAtEdge false Switch from VOD to live when playback catches up.

The returned controller exposes reactive state plus play(), pause(), goLive(), seekTo(epochMs), rewind(seconds?), setPlaybackRate(rate), setMuted(boolean), and setVolume(0to1). Drive the group from application UI:

import {
  RhombusAudioPlayer,
  RhombusPlayer,
  useRhombusPlaybackController,
} from "@rhombussystems/react";

function GroupControls() {
  const playback = useRhombusPlaybackController({
    initialMuted: true,
    defaultRewindSec: 30,
  });

  return (
    <>
      <RhombusPlayer cameraUuid="CAMERA_UUID" playbackController={playback} />
      <RhombusAudioPlayer
        source={{ type: "audio-gateway", uuid: "A100_UUID" }}
        playbackController={playback}
        controls={[]}
      />
      <button onClick={playback.play}>Play group</button>
      <button onClick={playback.pause}>Pause group</button>
      <button onClick={() => playback.setMuted(false)}>Unmute group</button>
      <button onClick={() => playback.seekTo(Date.now() - 5 * 60_000)}>
        Five minutes ago
      </button>
    </>
  );
}

Call play, goLive, or setMuted(false) directly inside the user's click/tap handler so browser audio activation remains associated with that gesture.

DR40 audio ownership

For a DR40 video/audio pair, give both participants the same DR40 UUID:

const playback = useRhombusPlaybackController();

<>
  <RhombusPlayer
    cameraUuid={dr40Uuid}
    playbackController={playback}
  />
  <RhombusAudioPlayer
    source={{ type: "dr40", uuid: dr40Uuid }}
    playbackController={playback}
    controls={["volume"]}
  />
</>

Ownership changes automatically:

  • realtime live video: RhombusAudioPlayer owns the live Opus audio;
  • buffered live video: the video element's embedded DASH audio owns output;
  • DR40 VOD: the video element's embedded DASH audio owns output;
  • while video owns output, the separate audio transport reports "embedded-dr40" and makes no duplicate audio request;
  • shared mute and volume are applied to whichever participant currently owns output.

This handoff only occurs for source.type === "dr40" and exactly matching UUIDs. A100 audio always remains separate, even when it is associated with the displayed camera.

Audio callbacks, state, and recovery

Use callbacks for telemetry and application UI:

Callback When it fires
onReady() An owned live WebSocket opens or the Dash.js VOD element can play; can fire again after reconnect/reinit. For decoded VOD and "embedded-dr40", observe status === "ready".
onModeChange(mode, atMs) Live/VOD mode changes.
onTransportChange(transport) Internal transport or DR40 ownership changes.
onSeek(atMs, mode) An explicit seek, rewind, or Go Live command is applied.
onProgress(atMs, mode) Best-effort wall-clock progress, approximately every 250 ms.
onPlayingChange(playing) Play intent changes.
onPlaybackRateChange(rate) Rate changes.
onMutedChange(muted) Mute changes.
onVolumeChange(volume) Gain changes.
onStatusChange(status) Status changes among idle/connecting/buffering/ready/reconnecting/error.
onStateChange(state) Any observable RhombusAudioPlayerState field changes.
onRecoveryAttempt(attempt, error) Live socket schedules an exponential-backoff reconnect.
onError(error) A non-recovered token, URI, decoder, segment, or transport error occurs.

RhombusAudioPlayerState contains source, mode, transport, playing, playbackRate, muted, volume, currentWallClockMs, isAtLiveEdge, and status. With a shared controller, status is the group's aggregate status: errors, reconnecting, buffering, and connecting participants take precedence over ready participants.

Live sockets reconnect with a 2s → 4s → 8s exponential delay capped by maxRetryIntervalMs. Server "reconnect" messages force media-URI re-resolution. Token rotation also re-resolves media and reconnects live audio. Historical requests use the latest token and are aborted when source, token, seek, or manifest window changes.

<RhombusAudioPlayer
  source={source}
  onReady={() => setMessage("Audio ready")}
  onRecoveryAttempt={attempt => setMessage(`Reconnecting (${attempt})…`)}
  onError={error => setMessage(`Audio failed: ${error.message}`)}
  onStateChange={state => analytics.track("audio-state", state)}
/>

Audio authentication and network modes

The same federated-token rules apply to audio, with additional audio endpoints and requests:

  • live WebSocket URL: token in x-auth-scheme / x-auth-ft query parameters;
  • historical MPD and Dash.js requests: latest token in query parameters;
  • decoded historical segment requests: latest token in query parameters;
  • token rotation: reconnect live; subsequent historical requests use the new token.

Proxy mode (recommended):

<RhombusAudioPlayer
  source={source}
  apiOverrideBaseUrl="https://app-api.example.com"
  paths={{
    federatedToken: "/media/federated-token",
    audioMediaUris: "/media/audio-uris",
  }}
/>

The browser sends { source } to paths.audioMediaUris; your backend chooses the A100 or DR40 upstream contract. Your API key never leaves the server.

Direct Rhombus media-URI mode:

<RhombusAudioPlayer
  source={source}
  paths={{
    federatedToken: "/api/federated-token",
    audioGatewayMediaUris: "/audiogateway/getMediaUris",
    dr40MediaUris: "/doorbellcamera/getMediaUris",
  }}
  rhombusApiBaseUrl="https://api2.rhombussystems.com/api"
/>

Here the token still comes from your same-origin backend, but the browser calls Rhombus getMediaUris directly using federated headers. The token must be minted for the browser origin/domain and your deployment must allow the cross-origin request.

Consumer-managed token rotation:

<RhombusAudioPlayer
  source={source}
  apiOverrideBaseUrl="/"
  federatedSessionToken={currentFederatedToken}
/>

Changing currentFederatedToken reconnects live audio and invalidates outstanding decoded historical work. Never pass an API key as this prop.

connectionMode="wan" selects wanLiveOpusUri and wanVodMpdUriTemplate. connectionMode="lan" selects the first non-empty lanLiveOpusUris and lanVodMpdUrisTemplates entry. A100 live URLs are normalized with /ws exactly once; DR40 socket paths are used unchanged. The SDK does not silently fall back between WAN and LAN: choose the mode that is reachable from the browser.

Audio styling and browser behavior

classNames adds classes to the built-in slots:

<RhombusAudioPlayer
  source={source}
  className="audio-player"
  classNames={{
    controls: "audio-toolbar",
    button: "audio-button",
    speed: "audio-speed",
    volume: "audio-volume",
    status: "audio-status",
    timeline: "audio-timeline",
  }}
/>

The SDK classes are .rhombus-audio-controls, .rhombus-audio-btn, .rhombus-audio-speed, .rhombus-audio-volume, and .rhombus-audio-status. Defaults use zero-specificity :where(...), so normal application CSS overrides them. The root exposes data-rhombus-audio-transport for transport-specific styling.

The component contains a non-visual <audio> element for DASH VOD. Do not use that element as the live-output contract: live and decoded VOD audio are scheduled through Web Audio and therefore have no meaningful audio.currentTime or audio.src for consumers. Use onProgress, onStateChange, or getState() instead.

Browser requirements:

  • live and decoded VOD: Web Audio, WebAssembly, and Web Workers;
  • preferred historical path: MSE with Opus/WebM;
  • Safari/iOS: normally uses the decoded historical fallback;
  • output begins muted; unmute must be user initiated;
  • background-tab throttling and OS audio routing still apply.

For two independent standalone audio players, omit playbackController on both (each creates its own controller), or give each a different controller. A single controller is intentionally limited to one video and one audio playback participant. Talkback coordinates through the controller but is not itself a playback clock participant.

RhombusTalkback — A100 and DR40 two-way audio

RhombusTalkback captures the user's browser microphone and sends it to the live speaker of an A100 audio gateway or DR40. It is deliberately separate from RhombusAudioPlayer: applications may offer speaking without listening, place the microphone control next to video, or compose full two-way audio without forcing microphone code or permission prompts into listen-only pages.

Talkback always targets the physical device now. If the page is showing historical footage, starting talkback still speaks through the device's live speaker; it never schedules voice at the historical playhead.

The upstream wire path is:

browser microphone
  → Web Audio capture/resampling
  → mono 48 kHz PCM16, exact 20 ms frames
  → authenticated realtime audio WebSocket
  → Rhombus server Opus encoding
  → A100 or DR40 speaker

The SDK requests the microphone only after startTalking() is invoked from a user gesture. It stops the microphone tracks after talking ends, so an idle component does not leave the browser's microphone privacy indicator active.

Talkback quick starts

Talk to an A100:

import { RhombusTalkback } from "@rhombussystems/react";

export function LobbyTalkback() {
  return (
    <RhombusTalkback
      source={{ type: "audio-gateway", uuid: "A100_AUDIO_GATEWAY_UUID" }}
      apiOverrideBaseUrl="https://your-api.example.com"
    />
  );
}

Talk to a DR40:

<RhombusTalkback
  source={{ type: "dr40", uuid: dr40Uuid }}
  apiOverrideBaseUrl="/"
/>

Use the A100 audio gateway UUID for "audio-gateway" and the DR40 device UUID for "dr40". The same source rules used by RhombusAudioPlayer apply.

Full two-way A100 audio:

import {
  RhombusAudioPlayer,
  RhombusTalkback,
  useRhombusPlaybackController,
} from "@rhombussystems/react";

function TwoWayAudio({ gatewayUuid }: { gatewayUuid: string }) {
  const playback = useRhombusPlaybackController();
  const source = {
    type: "audio-gateway" as const,
    uuid: gatewayUuid,
  };

  return (
    <>
      <RhombusAudioPlayer
        source={source}
        apiOverrideBaseUrl="/"
        playbackController={playback}
        controls={["volume"]}
      />
      <RhombusTalkback
        source={source}
        apiOverrideBaseUrl="/"
        playbackController={playback}
      />
    </>
  );
}

The shared controller lets the incoming player suppress Rhombus far-audio echo frames while that matching talkback participant is transmitting.

Video, listening, one timeline, and talkback:

function DoorStation({
  cameraUuid,
  audioSource,
}: {
  cameraUuid: string;
  audioSource: RhombusAudioSource;
}) {
  const playback = useRhombusPlaybackController();

  return (
    <>
      <RhombusPlayer
        cameraUuid={cameraUuid}
        apiOverrideBaseUrl="/"
        playbackController={playback}
      />
      <RhombusAudioPlayer
        source={audioSource}
        apiOverrideBaseUrl="/"
        playbackController={playback}
        controls={["volume"]}
      />
      <RhombusTalkback
        source={audioSource}
        apiOverrideBaseUrl="/"
        playbackController={playback}
        disableTalkbackInVod
      />
    </>
  );
}

For a DR40 whose video and audio share the same device UUID, pass that UUID to all matching DR40 participants. The existing playback ownership rules prevent duplicate incoming audio; talkback still uses the realtime audio socket to reach the speaker.

Automatic click/hold behavior

interactionMode="auto" is the default and follows the same effective AEC calculation as Rhombus Console:

const effectiveAec =
  !audio_use_external_mic &&
  !audio_use_external_speaker &&
  audio_internal_mic_aec_enabled;
  • Effective AEC enabled → "toggle": click/tap once to start; click/tap again to stop.
  • Effective AEC disabled → "hold": hold pointer, touch, Space, or Enter; releasing stops.

The capability proxy derives this mode from /audiogateway/getConfig or /doorbellcamera/getConfig. It also accounts for device_speaker_enabled, Enterprise licensing, API-key role/device access, and device availability.

Override only when a product intentionally wants different interaction:

<RhombusTalkback source={source} interactionMode="hold" apiOverrideBaseUrl="/" />

state.interactionMode is always the resolved "toggle" or "hold" value; it never reports "auto".

Talkback while viewing VOD

By default, talkback remains available while watching history:

<RhombusTalkback
  source={source}
  playbackController={playback}
  disableTalkbackInVod={false}
/>

This is useful for operators who understand that their voice goes to the live device even while the screen shows an earlier time.

For workflows where that could be confusing or unsafe:

<RhombusTalkback
  source={source}
  playbackController={playback}
  disableTalkbackInVod
/>

When the shared timeline enters VOD, the control becomes disabled. If a user was already talking, transmission and browser microphone capture stop immediately. Go Live re-enables the control.

Custom video players can provide the same information explicitly:

<RhombusTalkback
  source={source}
  viewingMode={showingHistory ? "vod" : "live"}
  disableTalkbackInVod
/>

playbackController takes precedence over viewingMode. Enabling VOD blocking without either produces a development warning because the component cannot infer what unrelated media UI is displaying.

Talkback props

source and proxy configuration are the normal required inputs:

Prop Type Default Purpose
source RhombusAudioSource required A100 or DR40 target.
apiOverrideBaseUrl string required for automatic capability Application backend hosting token, media, and capability routes.
connectionMode "wan" | "lan" "wan" Select the reachable realtime audio URI.
playbackController RhombusPlaybackController — VOD awareness and matching incoming-audio echo suppression.
viewingMode "live" | "vod" "live" Explicit VOD state when no controller is used.
disableTalkbackInVod boolean false Block/stop talking while viewing history.
interactionMode "auto" | "toggle" | "hold" "auto" Follow or override effective device AEC.
disabled boolean false Application-level disable.
microphoneGain number ≈6.62 Console-matched linear browser-microphone gain before PCM16 clipping.
microphoneConstraints MediaTrackConstraints mono/48 kHz/AEC Override browser capture device and processing constraints.
capability RhombusTalkbackCapability fetched from proxy Advanced server-resolved capability injection.
className / style root styling — Styles the component wrapper.
classNames RhombusTalkbackClassNames — Adds classes to default-control slots.
styles RhombusTalkbackStyles — Inline slot styling, merged last.
renderControl (api, state) => ReactNode default microphone Fully replace the UI while keeping the engine.
onReady () => void — Capability, token, and realtime URI are ready.
onTalkingChange (boolean) => void — Actual socket transmission starts/stops.
onPermissionChange (permission) => void — Browser microphone permission becomes known.
onStateChange (state) => void — Any observable talkback state changes.
onRecoveryAttempt (attempt, error) => void — An interrupted talk socket schedules a retry.
onError (error) => void — Capability, permission, media, or transport failure.

microphoneConstraints are merged after the defaults, so applications can select a specific deviceId or turn browser processing off. The SDK always converts the resulting Web Audio stream to the Rhombus-required mono 48 kHz PCM16 format.

Talkback custom controls and styling

The default control uses stable, zero-specificity CSS classes:

  • .rhombus-microphone-control
  • .rhombus-microphone-button
  • .rhombus-microphone-icon
  • .rhombus-microphone-content
  • .rhombus-microphone-label
  • .rhombus-microphone-status

Normal application CSS wins without !important:

.rhombus-microphone-button {
  width: 64px;
  height: 64px;
  background: var(--brand-action);
}

.rhombus-microphone-button[data-state="talking"] {
  background: var(--brand-danger);
}

Use classNames for CSS modules, Tailwind, or a design system. Use styles for inline prop-based styling; those values are placed directly on each slot and override stylesheet rules:

<RhombusTalkback
  source={source}
  apiOverrideBaseUrl="/"
  classNames={{ root: styles.root, label: styles.label }}
  styles={{
    button: { width: 56, height: 56, borderRadius: 12 },
    status: { color: "#7dd3fc" },
  }}
/>

Replace the UI entirely:

<RhombusTalkback
  ref={talkbackRef}
  source={source}
  apiOverrideBaseUrl="/"
  renderControl={(api, state) => (
    <button
      disabled={!state.canTalk}
      onClick={() => void api.toggleTalking()}
    >
      {state.talking ? "Stop speaking" : "Speak"}
    </button>
  )}
/>

For a detached reusable control, export state from onStateChange and render RhombusMicrophoneControl with the imperative API and state. Its RhombusMicrophoneControlProps type is exported:

import { useState } from "react";
import {
  RhombusMicrophoneControl,
  RhombusTalkback,
  type RhombusAudioSource,
  type RhombusTalkbackHandle,
  type RhombusTalkbackState,
} from "@rhombussystems/react";

function DetachedTalkControl({ source }: { source: RhombusAudioSource }) {
  const [api, setApi] = useState<RhombusTalkbackHandle | null>(null);
  const [state, setState] = useState<RhombusTalkbackState | null>(null);

  return (
    <>
      <RhombusTalkback
        ref={setApi}
        source={source}
        apiOverrideBaseUrl="/"
        renderControl={() => null}
        onStateChange={setState}
      />

      {api && state ? (
        <RhombusMicrophoneControl
          api={api}
          state={state}
          classNames={{ root: "toolbar-microphone" }}
          styles={{ button: { width: 56, height: 56 } }}
          buttonProps={{ "aria-describedby": "talkback-help" }}
        />
      ) : null}
      <span id="talkback-help">Voice is sent to the live device.</span>
    </>
  );
}

buttonProps accepts ordinary native button attributes except the interaction, disabled, class, and style fields owned by the control. Use classNames and styles for those visual fields; inline styles are merged last.

The imperative handle contains:

Method Behavior
startTalking() Request/open microphone and begin transmission. Call from a user gesture.
stopTalking() Immediately close TX, stop capture tracks, and clear buffered frames.
toggleTalking() Start or stop; convenient for custom click-to-talk controls.
requestMicrophonePermission() Prompt without starting transmission, then release the track.
getState() Return the latest RhombusTalkbackState.

Talkback state, callbacks, permissions, and safety

RhombusTalkbackState includes:

type RhombusTalkbackState = {
  source: RhombusAudioSource;
  status:
    | "loading-capability"
    | "ready"
    | "requesting-permission"
    | "connecting"
    | "talking"
    | "reconnecting"
    | "blocked"
    | "error";
  talking: boolean;
  interactionMode: "toggle" | "hold";
  canTalk: boolean;
  blockedReason:
    | "disabled"
    | "vod"
    | "not-authorized"
    | "license-required"
    | "speaker-disabled"
    | "device-unavailable"
    | "capability-unavailable"
    | null;
  microphonePermission: "unknown" | "prompt" | "granted" | "denied";
  viewingMode: "live" | "vod";
  capability: RhombusTalkbackCapability | null;
};

For privacy and stuck-microphone prevention, active talking stops on pointer/keyboard release, pointer cancellation, pointer leave, control blur, window blur, a hidden tab, unmount, source change, application disable, capability loss, and a blocked VOD transition. An asynchronous permission or socket operation is generation-checked, so releasing before connection finishes cannot start transmission later.

Unexpected socket closes retry with the normal exponential backoff. The 16-byte Rhombus ctrl/play-ASAP prefix is resent before the first exact 1,920-byte PCM frame on every new socket. Pending capture is bounded to roughly 200 ms so reconnects cannot replay a long, stale microphone backlog.

Talkback authentication and capability policy

Talkback uses the same federated token and realtime audio URI contract as live listening, but automatic capability resolution intentionally requires an application-owned proxy:

<RhombusTalkback
  source={source}
  apiOverrideBaseUrl="https://your-api.example.com"
  paths={{
    federatedToken: "/media/federated-token",
    audioMediaUris: "/media/audio-uris",
    audioTalkbackCapabilities: "/media/audio-talkback-capabilities",
  }}
/>

The capability response must already reflect:

  • whether the API key's assigned role can access the requested device;
  • an Enterprise license assigned to that A100/DR40;
  • device_speaker_enabled;
  • device connectivity/availability;
  • effective AEC and therefore toggle versus hold behavior.

The SDK treats that response as the product-policy input and never attempts to infer role or license inventory in the browser. Client-side disabling is user experience, not a substitute for server authorization: the realtime service must enforce the federated principal's device scope for the WebSocket itself.

Supplying capability is useful when a parent already fetched the same normalized server answer. Do not create an optimistic { canTalk: true } object in browser code; doing so bypasses the intended product UI gate and can drift from Console behavior.

RhombusBufferedPlayer — DASH live & VOD

Renders live or historical footage with Dash.js into a <video> element. This is the right choice when you want native <video> semantics, the widest browser support, or you're composing your own layout.

<RhombusBufferedPlayer
  cameraUuid="YOUR_CAMERA_UUID"
  connectionMode="wan"          // "wan" (default) | "lan"
  bufferedStreamQuality="HIGH"  // "HIGH" | "MEDIUM" | "LOW"
  videoProps={{ controls: true, style: { width: "100%" } }}
  onReady={() => console.log("playing")}
  onError={(e) => console.error(e)}
/>

Shared base props (all players)

These come from RhombusMediaBaseProps and are accepted by video players, the audio player, talkback, and Timeline. RhombusPlayerBaseProps extends this type with the cameraUuid required by video players plus the optional deviceType ("camera" | "doorbell", default "camera" — set "doorbell" for DR40 video intercoms). Audio/talkback use source; Timeline needs cameraUuid (and deviceType for a DR40) only when it fetches camera-specific data.

Prop Type Default Notes
connectionMode `"wan" "lan"` "wan"¹
apiOverrideBaseUrl string — Base for the token and media requests. Set for proxy mode. When omitted, media is fetched directly from Rhombus (needs a domain-scoped token).
rhombusApiBaseUrl string https://api2.rhombussystems.com/api Rhombus REST base when apiOverrideBaseUrl is omitted.
paths RhombusPlayerPaths see backend Override video/audio media, token, seekpoint, and availability routes.
federatedSessionToken string — Supply & rotate your own token; the SDK skips its token endpoint.
tokenDurationSec number 86400 Requested token TTL (SDK-managed mode).
headers HeadersInit — Static headers for the token request (+ media when apiOverrideBaseUrl set).
getRequestHeaders `() => HeadersInit Promise<…>` —
maxRetryIntervalMs number 30000 Auto-recovery backoff ceiling. 0 disables.
stallTimeoutMs number 12000 Stall watchdog. 0 disables.
onRecoveryAttempt (attempt, error) => void — Fires on each retry.
className / style string / CSSProperties — Applied to the player element.
onError (error: Error) => void — Token / media / setup failure.

¹ connectionMode is required (no default) on RhombusRealtimePlayer.

RhombusBufferedPlayer-specific props

Prop Type Default Notes
startTimeSec number (Unix seconds) — Set to play the past (VOD). Omit for live. Changing it re-attaches a new manifest.
vodDurationSec number 7200 VOD window length; how far you can seek before a new manifest is needed.
seekOffsetSec number 0 Where in the window playback begins.
bufferedStreamQuality `"HIGH" "MEDIUM" "LOW"`
applyBufferedStreamQuality boolean true false omits _ds (full resolution).
videoProps VideoHTMLAttributes — Spread onto the <video> (controls, muted, onClick, style, …).
onReady () => void — Dash.js initialized and manifest loaded.

Exposes a ref handle: { getVideoElement(), getDashPlayer() }.

Live vs. past — the single switch is startTimeSec:

function CameraPlayer({ cameraUuid, mode }: { cameraUuid: string; mode: "live" | "past" }) {
  const startTimeSec =
    mode === "past" ? Math.floor(new Date("2025-04-15T00:00:00Z").getTime() / 1000) : undefined;
  return (
    <RhombusBufferedPlayer
      cameraUuid={cameraUuid}
      startTimeSec={startTimeSec}  // undefined => live, number => VOD
      vodDurationSec={3600}
      videoProps={{ controls: true }}
    />
  );
}

Scrub beyond the window by updating startTimeSec from your own timeline:

const [startTimeSec, setStartTimeSec] = useState(() => Math.floor(Date.now() / 1000) - 3600);
<>
  <input type="datetime-local" onChange={(e) => {
    const ms = new Date(e.target.value).getTime();
    if (!Number.isNaN(ms)) setStartTimeSec(Math.floor(ms / 1000)); // loads a fresh window
  }} />
  <RhombusBufferedPlayer
    cameraUuid={cameraUuid}
    startTimeSec={startTimeSec}
    videoProps={{ controls: true }}
  />
</>

Pausing live DASH lets it fall behind the live edge; Dash.js catches up on resume. For frame-accurate pause use VOD mode (startTimeSec) — or just use RhombusPlayer, which handles this for you.

formatVodMpdUri(template, startTimeSec, durationSec) and getDefaultRhombusVodDashSettings() are exported if you need to build VOD URLs or tune Dash.js yourself.

RhombusRealtimePlayer — low-latency live

Live H.264 over WebSocket, decoded with WebCodecs onto a <canvas>. Live only — no pause, seek, or VOD. Sub-second latency; ideal for a video wall or PTZ control.

<RhombusRealtimePlayer
  cameraUuid="YOUR_CAMERA_UUID"
  connectionMode="wan"          // REQUIRED: "wan" | "lan"
  realtimeStreamQuality="HD"    // "HD" (/ws) | "SD" (/wsl)
  style={{ width: "100%", background: "#111" }}
  onReady={() => console.log("connected")}
  onError={(e) => console.error(e)}
/>

Accepts all shared base props, plus:

Prop Type Default Notes
connectionMode `"wan" "lan"` — (required)
realtimeStreamQuality `"HD" "SD"` "HD"
canvasProps CanvasHTMLAttributes — Spread onto the <canvas>.
onReady () => void — Fires on every WebSocket OPEN (first connect and each reconnect).

Exposes a ref handle: { getCanvasElement() }.

**onReady and token rotation differ from buffered:** realtime onReady fires on every (re)connect, and because auth is on the socket URL, each token refresh closes/reopens the socket (short blip). The buffered player rotates tokens without a teardown.

Optional low-level exports for custom wiring: resolveLiveH264WebSocketUrl(options), startRhombusRealtimeSession(options).

Timeline — standalone scrubber

A vendor-neutral canvas scrubber. It does not embed a player — pair it with any video source (or let RhombusPlayer drive it for you). It draws an availability bar, event seekpoints (optionally fetched from /camera/getFootageSeekpointsV2), static marks, a playhead, and a hover line, and emits onSeek(wallClockMs) on click/drag.

import { Timeline } from "@rhombussystems/react";

<Timeline
  cameraUuid="YOUR_CAMERA_UUID"
  apiOverrideBaseUrl="https://your-api.example.com"
  rangeStartMs={Date.now() - 3_600_000}
  rangeEndMs={Date.now()}
  currentTimeMs={playheadMs}
  fetchSeekPoints
  marks={[{ startMs: t0, endMs: t1, kind: "event", color: "#f80", label: "Motion" }]}
  onSeek={(ms) => setPlayheadMs(ms)}
  onHoverTimeChange={(ms) => setHoverMs(ms)}
/>

Accepts the shared base props (used when fetching seekpoints/availability) plus:

Prop Type Default Notes
rangeStartMs / rangeEndMs number (epoch ms) — (required) Visible time window.
cameraUuid string — Required at runtime only when fetchSeekPoints or fetchAvailability is enabled.
playbackController RhombusPlaybackController — Supplies the playhead and seek action; takes precedence over currentTimeMs and onSeek.
currentTimeMs `number null` —
onSeek (wallClockMs) => void — Click/drag to seek. Required only when no playbackController is supplied.
onHoverTimeChange `(wallClockMs null) => void` —
selection `{ startMs, endMs } null` —
onSelectionChange ({ startMs, endMs }) => void — Fired as the user drags the selection.
selectionMinDurationMs / selectionMaxDurationMs number 5000 / 3600000 Drag clamps for the selection.
onShiftWindow `(direction: -1 1) => void` —
canShiftBack / canShiftForward boolean true Enable/disable the respective chevron at a limit.
onZoom (zoomIn: boolean, centerWallClockMs: number) => void — When provided, enables −/+ zoom buttons and mouse-wheel zoom (centered on the cursor). Range changes animate.
canZoomIn / canZoomOut boolean true Enable/disable the respective zoom button at a limit.
fetchSeekPoints boolean false Fetch event markers for the range. Rendered as clustered colored dashes grouped by activity type.
includeAnyMotion boolean true Include generic motion in the fetch.
fetchAvailability boolean false Fetch recorded-footage coverage and show confirmed gaps. Requires cameraUuid.
onAvailabilityLoaded (RhombusFootageAvailability) => void — Receives normalized cloud/local footage windows after a successful fetch.
marks TimelineMark[] — Static event bands (kind:"event") / gaps (kind:"gap").
onSeekPointsLoaded (RhombusFootageSeekPoint[]) => void — Normalized seekpoints after each fetch (handy for diagnostics).
colors TimelineColors — Override the canvas-drawn colors (see Theming the timeline).
height number 56 Canvas height in px.

Timeline also draws a time axis with auto-spaced tick labels (interval chosen for ~6 divisions, h a / h:mm a format), an availability bar, a playhead, and a hover line.

Exposes a ref handle: { refresh() } to force a seekpoint refetch.

Theming the timeline

The timeline is drawn on a <canvas>, so its colors can't be set with CSS. Pass a colors object instead (every field optional, merged over the defaults). On RhombusPlayer use timeline={{ colors: … }}; on the standalone Timeline use the colors prop:

<RhombusPlayer
  cameraUuid="…"
  timeline={{
    colors: {
      background: "#0b1220",          // canvas fill (default transparent)
      availabilityActive: "#22c55e",  // recorded-footage bar
      availabilityInactive: "#334155",// empty/future bar
      playhead: "#f59e0b",
      hover: "rgba(255,255,255,0.6)",
      tick: "#475569",
      tickLabel: "#94a3b8",
      seekpointDefault: "#60a5fa",     // activities not in eventColors
      seekpointAlert: "#ef4444",       // alerted events
      eventColors: {                   // merged over the built-in per-activity palette
        MOTION_HUMAN: "#facc15",
        MOTION_CAR: "#38bdf8",
        FACE: "#34d399",
      },
      buttonBackground: "#1e293b",     // ‹/›/−/+ buttons
      buttonBorder: "#475569",
      buttonText: "#e2e8f0",
      selection: "rgba(59,130,246,0.22)", // clip-selection region
      selectionHandle: "#3b82f6",         // clip-selection drag handles
    },
  }}
/>

eventColors keys are activity strings from getFootageSeekpointsV2 (e.g. MOTION, MOTION_HUMAN, MOTION_CAR, MOTION_ANIMAL, FACE, SOUND_LOUD, …). The timeline's wrapper (and RhombusPlayer's root) can still be styled via className/style / classNames.timeline — colors.background paints the canvas itself.

Pairing it with a video source

Timeline is just a seek UI — it has no idea what's playing. You wire it to a video by (a) feeding it the current playhead as currentTimeMs, and (b) handling onSeek to move that video. Here it is paired with a RhombusBufferedPlayer in VOD mode, using the player's ref handle (getVideoElement()) to read and drive the underlying <video>. Wall-clock maps to the video as windowStart + video.currentTime:

import { useEffect, useRef, useState } from "react";
import {
  RhombusBufferedPlayer,
  Timeline,
  type RhombusBufferedPlayerHandle,
} from "@rhombussystems/react";

function ScrubbableVod({ cameraUuid }: { cameraUuid: string }) {
  const player = useRef<RhombusBufferedPlayerHandle>(null);
  const windowSec = 3600;
  // Epoch seconds of the VOD manifest window the player currently has loaded.
  const [windowStartSec, setWindowStartSec] = useState(() => Math.floor(Date.now() / 1000) - windowSec);
  const [currentMs, setCurrentMs] = useState(windowStartSec * 1000);

  // Drive the playhead from the <video>'s position.
  useEffect(() => {
    const id = setInterval(() => {
      const v = player.current?.getVideoElement();
      if (v) setCurrentMs(windowStartSec * 1000 + v.currentTime * 1000);
    }, 250);
    return () => clearInterval(id);
  }, [windowStartSec]);

  function handleSeek(ms: number) {
    const v = player.current?.getVideoElement();
    const offsetSec = (ms - windowStartSec * 1000) / 1000;
    if (v && offsetSec >= 0 && offsetSec <= windowSec) {
      v.currentTime = offsetSec;                  // inside the loaded window — instant
    } else {
      setWindowStartSec(Math.floor(ms / 1000));   // outside — load a fresh window at that time
    }
    setCurrentMs(ms);
  }

  return (
    <>
      <RhombusBufferedPlayer
        ref={player}
        cameraUuid={cameraUuid}
        apiOverrideBaseUrl="https://your-api.example.com"
        startTimeSec={windowStartSec}
        vodDurationSec={windowSec}
        videoProps={{ controls: false }}
      />
      <Timeline
        cameraUuid={cameraUuid}
        apiOverrideBaseUrl="https://your-api.example.com"
        rangeStartMs={windowStartSec * 1000}
        rangeEndMs={windowStartSec * 1000 + windowSec * 1000}
        currentTimeMs={currentMs}
        fetchSeekPoints
        onSeek={handleSeek}
      />
    </>
  );
}

The same two wires work for any video: a plain <video> (read/set video.currentTime), an HLS/DASH player, or a multi-camera wall sharing one playhead. (RhombusPlayer does exactly this internally — reach for it if you don't want to own the wiring yourself.)

Authentication & tokens

The SDK is built around short-lived federated session tokens minted by your backend; your Rhombus API key must never reach the browser.

SDK-managed (recommended)

Omit federatedSessionToken. The SDK POSTs to your token route (default /api/federated-token) with { "durationSec": <tokenDurationSec> } and auto-refreshes before expiry (~97% of the effective TTL). Effective TTL = min of your tokenDurationSec and any server hint in the response (expiresInSec, expiresAtMs, or expiresAt).

  • Video DASH / buffered: keeps playing across refreshes (requests read the latest token).
  • Realtime video: reconnects the socket on each refresh (short blip).
  • Live audio: re-resolves media URIs and reconnects with the new token.
  • Talkback: an active microphone session reconnects and resends its control prefix.
  • Historical audio: Dash.js and decoded segment requests read the latest token; outstanding decoded work is invalidated during rotation.

You-managed

Pass federatedSessionToken. The SDK never calls your token endpoint. Rotate by passing a new string. Video DASH reads it without a teardown; realtime video, live audio, and active talkback reconnect; historical audio uses it for subsequent requests.

Two transport topologies

apiOverrideBaseUrl omitted apiOverrideBaseUrl set (proxy mode)
Token request window.location.origin + paths.federatedToken apiOverrideBaseUrl + paths.federatedToken
Video media URIs Direct to Rhombus api2.rhombussystems.com + paths.mediaUris apiOverrideBaseUrl + paths.mediaUris
Audio media URIs Direct to Rhombus + paths.audioGatewayMediaUris or paths.dr40MediaUris apiOverrideBaseUrl + paths.audioMediaUris
Talkback capability Supply a trusted, server-resolved capability only apiOverrideBaseUrl + paths.audioTalkbackCapabilities
Requirement Token minted with a Rhombus domain allowing this origin, or the browser call is blocked (CORS / 401) Your backend proxies getMediaUris; browser never talks to Rhombus directly

Proxy mode is also required for built-in Save Clip (see Save Clip).

WAN vs LAN

connectionMode selects which getMediaUris URI to use:

  • wan (default except low-level realtime, where it is required explicitly) — cloud path; works anywhere with internet.
  • lan — direct-to-device path (lanLive* fields, first non-empty entry). The browser must reach the camera/NVR host (routing, firewall, and HTTPS-vs-HTTP mixed-content rules apply). Federated auth rides as x-auth-scheme=federated-token & x-auth-ft query params.

v1.0 breaking change: LAN no longer uses document.cookie or applyLanAuthCookie, and setRhombusLanAuthCookie was removed. LAN now passes federated-token query params on the URL (works from localhost). Your Rhombus deployment must accept those params on LAN.

For LAN DASH, applyBufferedStreamQuality={false} disables the _ds downscale for full-resolution LAN.

Stream quality

Buffered / DASH — bufferedStreamQuality: "HIGH" (default) | "MEDIUM" | "LOW". Each step asks Rhombus to downscale server-side via a _ds query on segment/manifest URLs. Changing it updates URLs without re-fetching the manifest or token. applyBufferedStreamQuality={false} omits _ds entirely.

Realtime — realtimeStreamQuality: "HD" (default) | "SD". SD rewrites the socket path /ws → /wsl. Changing it reconnects the socket (brief blip).

const [q, setQ] = useState<RhombusBufferedStreamQuality>("HIGH");
<RhombusBufferedPlayer cameraUuid="…" bufferedStreamQuality={q} />

On RhombusPlayer, these are bufferedStreamQuality / realtimeStreamQuality, and the optional showLiveTypeSwitcher surfaces them in the bar.

Auto-recovery / reconnect

Both transports retry indefinitely with exponential backoff (2s → 4s → 8s → 16s → … capped at maxRetryIntervalMs, default 30s). Backoff resets to 2s after ~30s of healthy playback. Set maxRetryIntervalMs={0} to disable; pass onRecoveryAttempt to drive "reconnecting…" UI.

Buffered (DASH) rebuilds Dash.js when a recoverable error fires, the initial buffer never loads within stallTimeoutMs, or currentTime stops advancing for stallTimeoutMs (not paused/seeking/ended). Time spent in a hidden/background tab never counts toward the stall watchdog — browsers throttle timers and pause or suspend muted video there, so frozen playback is expected. When the tab becomes visible again the watchdog re-arms with a fresh stallTimeoutMs window, and playback resumes automatically if the browser paused it in the background.

Realtime video (WebSocket) reopens the socket on onerror/unexpected onclose, if it fails to open within ~8s, if no decoded frame arrives within stallTimeoutMs (the classic "WAN black screen until refresh"), or on a server reconnect message.

Live audio (WebSocket) reopens on an unexpected close, after stallTimeoutMs without messages, after repeated malformed TLV data, and on a server reconnect message. A server reconnect also re-resolves the A100/DR40 media URIs. Historical Dash.js initialization failure falls back to decoded Opus; historical fetch/decoder errors surface through onError.

Talkback (WebSocket) reconnects only while the user still has active talk intent. Capture frames waiting for the socket are bounded, and every new socket receives a fresh play-ASAP control prefix before PCM. Releasing the control cancels pending recovery immediately.

function CameraWithStatus({ cameraUuid }: { cameraUuid: string }) {
  const [attempt, setAttempt] = useState(0);
  const [error, setError] = useState<Error | null>(null);
  return (
    <div>
      {attempt > 0 ? <div role="status">Reconnecting (attempt {attempt})…</div>
        : error ? <div role="alert">Playback error: {error.message}</div> : null}
      <RhombusBufferedPlayer
        cameraUuid={cameraUuid}
        onReady={() => { setAttempt(0); setError(null); }}
        onError={setError}
        onRecoveryAttempt={setAttempt}
      />
    </div>
  );
}

Backend contract

Token endpoint (always required)

POST your paths.federatedToken route (default /api/federated-token):

  • Request: { "durationSec": number }.
  • Server: forward to Rhombus POST /org/generateFederatedSessionToken with your server-side API key. Include a Rhombus domain so the browser may call api2.rhombussystems.com in direct mode.
  • Response JSON: must include federatedSessionToken. Optionally expiresInSec / expiresAtMs / expiresAt so refresh timing matches your server-enforced cap.

Media-URI endpoint (proxy mode only)

Needed when apiOverrideBaseUrl is set. POST your paths.mediaUris route (default /api/media-uris):

  • Request: { "cameraUuid": string } — plus "deviceType": "doorbell" when the player was given deviceType="doorbell" (a DR40).
  • Server: forward Rhombus POST /camera/getMediaUris — or, when deviceType === "doorbell", POST /doorbellcamera/getMediaUris with { deviceUuid: cameraUuid } — and return the JSON as-is so the relevant fields survive: wanLiveMpdUri / wanVodMpdUriTemplate (WAN DASH), lanLiveMpdUris / lanLiveMpdUri / lanVodMpdUrisTemplates (LAN DASH), wanLiveH264Uri(s) / lanLiveH264Uri(s) (realtime). The same deviceType tag rides along on the /api/footage-seekpoints and /api/presence-windows bodies — route those to /doorbellcamera/getSeekpoints ({ deviceUuid, startTimeSec, durationSecs, includeAnyMotion }) and /doorbellcamera/getPresenceWindows ({ deviceUuid, startTimeSec, durationSec }).

A minimal Express proxy:

app.post("/api/federated-token", async (req, res) => {
  const r = await fetch("https://api2.rhombussystems.com/api/org/generateFederatedSessionToken", {
    method: "POST",
    headers: { "x-auth-apikey": process.env.RHOMBUS_API_TOKEN, "content-type": "application/json" },
    body: JSON.stringify({ durationSec: req.body.durationSec, domain: ".your-domain.com" }),
  });
  res.json(await r.json());
});

app.post("/api/media-uris", async (req, res) => {
  const isDoorbell = req.body.deviceType === "doorbell"; // DR40 video intercom
  const path = isDoorbell ? "/doorbellcamera/getMediaUris" : "/camera/getMediaUris";
  const body = isDoorbell
    ? { deviceUuid: req.body.cameraUuid }
    : { cameraUuid: req.body.cameraUuid };
  const r = await fetch(`https://api2.rhombussystems.com/api${path}`, {
    method: "POST",
    headers: { "x-auth-apikey": process.env.RHOMBUS_API_TOKEN, "content-type": "application/json" },
    body: JSON.stringify(body),
  });
  res.json(await r.json()); // return upstream as-is
});

Audio media-URI endpoint (proxy mode only)

RhombusAudioPlayer posts { source: { type, uuid } } to paths.audioMediaUris (default /api/audio-media-uris). Route type: "audio-gateway" to /audiogateway/getMediaUris with { gatewayUuid: uuid }; route type: "dr40" to /doorbellcamera/getMediaUris with { deviceUuid: uuid }. Return the upstream response unchanged. The SDK selects WAN/LAN fields, adds the A100 /ws suffix, and applies federated auth to the WebSocket, MPD, and every segment.

app.post("/api/audio-media-uris", async (req, res) => {
  const type = req.body?.source?.type;
  const uuid = req.body?.source?.uuid;
  if (!uuid || (type !== "audio-gateway" && type !== "dr40")) {
    return res.status(400).json({ error: "Invalid audio source" });
  }
  const isGateway = type === "audio-gateway";
  const path = isGateway
    ? "/audiogateway/getMediaUris"
    : "/doorbellcamera/getMediaUris";
  const body = isGateway ? { gatewayUuid: uuid } : { deviceUuid: uuid };
  const upstream = await fetch(`https://api2.rhombussystems.com/api${path}`, {
    method: "POST",
    headers: {
      "x-auth-apikey": process.env.RHOMBUS_API_TOKEN,
      "content-type": "application/json",
    },
    body: JSON.stringify(body),
  });
  res.status(upstream.status).json(await upstream.json());
});

Preserve upstream non-2xx status codes in your wrapper. Do not transform or rename wanLiveOpusUri, lanLiveOpusUris, wanVodMpdUriTemplate, or lanVodMpdUrisTemplates.

Audio talkback capability endpoint

RhombusTalkback posts { source } to paths.audioTalkbackCapabilities (default /api/audio-talkback-capabilities). This route is deliberately normalized rather than a raw Rhombus pass-through because it is the application backend's job to combine API-key RBAC, license, device configuration, and connectivity:

type RhombusTalkbackCapability = {
  canTalk: boolean;
  interactionMode: "toggle" | "hold";
  authorized: boolean;
  licensed: boolean;
  speakerEnabled: boolean;
  connected?: boolean;
  reason?:
    | "not-authorized"
    | "license-required"
    | "speaker-disabled"
    | "device-unavailable"
    | "capability-unavailable";
};

For the selected source, use the server-side API key to:

  1. Verify the device is returned by the accessible A100/DR40 inventory for that API key's assigned role. Any device permission accepted by Console—ADMIN, READONLY, or LIVEONLY—is sufficient to talk; ADMIN is required only to change audio settings.
  2. Fetch /audiogateway/getConfig with { audioGatewayUuid } or /doorbellcamera/getConfig with { deviceUuid }.
  3. Fetch /license/getDeviceLicenses and require the matching device's licenseType to be "ENTERPRISE".
  4. Require config.device_speaker_enabled === true and current device availability.
  5. Return "toggle" only when the internal microphone and speaker are selected and audio_internal_mic_aec_enabled is true; otherwise return "hold".

Return a non-revealing not-authorized answer for an inaccessible/unknown UUID rather than leaking cross-organization device existence. Set Cache-Control: no-store: role, license, speaker, and connectivity state can change. Return capability-unavailable for transient configuration, inventory, or license-service failures instead of misreporting them as a policy denial.

Conceptually:

app.post("/api/audio-talkback-capabilities", async (req, res) => {
  const { type, uuid } = req.body.source;
  const isGateway = type === "audio-gateway";

  // All calls use the server-side API key whose assigned role defines access.
  const [config, licenses, accessibleDevices] = await Promise.all([
    rhombusPost(
      isGateway ? "/audiogateway/getConfig" : "/doorbellcamera/getConfig",
      isGateway ? { audioGatewayUuid: uuid } : { deviceUuid: uuid }
    ),
    rhombusPost("/license/getDeviceLicenses", {}),
    listAccessibleAudioDevices(type),
  ]);

  const authorized = accessibleDevices.some(device => device.uuid === uuid);
  const licensed = licenses.deviceLicenses?.some(
    license => license.deviceUuid === uuid && license.licenseType === "ENTERPRISE"
  ) ?? false;
  const speakerEnabled = authorized && config.config?.device_speaker_enabled === true;
  const connected = authorized && getDeviceConnected(accessibleDevices, uuid);
  const effectiveAec =
    config.config?.audio_internal_mic_aec_enabled === true &&
    config.config?.audio_use_external_mic !== true &&
    config.config?.audio_use_external_speaker !== true;
  const canTalk = authorized && licensed && speakerEnabled && connected;

  res.set("Cache-Control", "no-store").json({
    canTalk,
    interactionMode: effectiveAec ? "toggle" : "hold",
    authorized,
    licensed,
    speakerEnabled,
    connected,
    ...(!authorized ? { reason: "not-authorized" }
      : !licensed ? { reason: "license-required" }
      : !speakerEnabled ? { reason: "speaker-disabled" }
      : !connected ? { reason: "device-unavailable" }
      : {}),
  });
});

The capability route controls the SDK UI, but it cannot secure a WebSocket by itself. Ensure the realtime audio handshake also validates the federated token's permission group against the requested device before accepting inbound PCM.

Footage seekpoints (Timeline, optional)

When Timeline/RhombusPlayer fetches seekpoints, it POSTs paths.footageSeekpoints (proxy default /api/footage-seekpoints) with { cameraUuid, startTime, duration, includeAnyMotion } (seconds). Forward to Rhombus POST /camera/getFootageSeekpointsV2 and return the JSON as-is.

Presence windows (footage availability, optional)

When Timeline/RhombusPlayer fetches footage availability (and before every built-in clip export unless requireFootage: "off"), it POSTs paths.presenceWindows (proxy default /api/presence-windows) with { cameraUuid, startTimeSec, durationSec } (seconds). Forward to Rhombus POST /camera/getPresenceWindows with your server-side API key and return the JSON as-is ({ presenceWindows: { VideoCloud: [...], VideoLocal: [...] } }). If the route is missing, availability rendering stays in the legacy mode and the clip pre-check fails open — nothing breaks, you just don't get gap detection.

Clip routes (built-in Save Clip)

Built-in export needs three routes (defaults shown; override via saveClip.paths). All are API-key authed server-side — the federated token is not used here:

Route Method Forwards to Notes
/api/save-clip POST /video/spliceV3 Forward the SDK's body as-is; return { clipUuid }.
/api/clip-progress POST /event/getClipWithProgress Forward { clipUuid }; return the { clip: { status, percentComplete, currentOperation, clipLocation } }.
/api/clip-download GET media host ?clipUuid=…&region=… → stream /media/metadata/{region}/{clipUuid}.mp4 with the API key.

Each route forwards to the Rhombus endpoint with your server-side API key (mirroring the token / media-URI routes above); /api/clip-download resolves the media host + region and streams the file back.

Never put Rhombus API keys in frontend headers.

Exported API surface

Components

  • RhombusMediaPlayer — complete video, A100/DR40 audio, shared timeline, and talkback facade.
  • RhombusPlayer — unified live/VOD player with controls.
  • RhombusAudioPlayer — unified A100/DR40 live and historical audio.
  • RhombusAudioPlayerControls — reusable audio control bar.
  • RhombusTalkback — A100/DR40 browser-microphone talkback engine and default control.
  • RhombusMicrophoneControl — reusable default talkback control for detached/custom layouts.
  • RhombusBufferedPlayer — DASH live & VOD.
  • RhombusRealtimePlayer — realtime H.264 live.
  • RhombusPlayerControls — the default control bar (exported for advanced composition).
  • RhombusDateTimePicker — standalone date/time jump picker (footage-aware disabled days).
  • Timeline — standalone canvas scrubber.

Hooks

  • useRhombusPlaybackController — create the shared epoch-ms playback state and commands used to synchronize one video participant, one audio participant, an optional timeline, and matching talkback policy/echo coordination.

Constants (value and type — usable as named members or plain strings)

  • RhombusPlayerControl — { Play, GoLive, Rewind, Speed, Zoom, Snapshot, SaveClip, Timeline, LiveType, VideoFit, GoToDate }.
  • RhombusAudioPlayerControl — { Play, GoLive, Rewind, Speed, Volume, Timeline }.

Types

  • Player props: RhombusPlayerProps, RhombusBufferedPlayerProps, RhombusRealtimePlayerProps, RhombusPlayerBaseProps, TimelineProps.
  • Audio/controller: RhombusAudioPlayerProps, RhombusAudioPlayerHandle, RhombusAudioPlayerState, RhombusAudioPlayerClassNames, RhombusAudioPlayerControlsProps, RhombusAudioSource, RhombusAudioTransport, RhombusPlaybackController, RhombusPlaybackControllerOptions, RhombusPlaybackControllerState, RhombusMediaBaseProps.
  • Talkback: RhombusTalkbackProps, RhombusTalkbackHandle, RhombusTalkbackState, RhombusTalkbackStatus, RhombusTalkbackCapability, RhombusTalkbackInteractionMode, RhombusResolvedTalkbackInteractionMode, RhombusTalkbackBlockedReason, RhombusTalkbackClassNames, RhombusTalkbackStyles, RhombusMicrophonePermission, RhombusMicrophoneControlProps.
  • Complete media facade: RhombusMediaPlayerProps, RhombusMediaPlayerHandle, RhombusMediaPlayerClassNames, RhombusMediaPlayerStyles, RhombusMediaPlayerVideoProps, RhombusMediaPlayerAudioProps, RhombusMediaPlayerTalkbackProps.
  • Handles: RhombusPlayerHandle, RhombusBufferedPlayerHandle, RhombusRealtimePlayerHandle, TimelineHandle.
  • Unified player: RhombusPlayerState, RhombusPlayerMode, RhombusPlayerClassNames, RhombusLiveTransport, RhombusVideoFit, RhombusSnapshotResult, RhombusClipRange, RhombusClipVisibility, RhombusClipExportOptions, RhombusClipExportPhase, RhombusClipExportStatus, RhombusSaveClipConfig, RhombusPlayerTimelineConfig.
  • Timeline: TimelineMark, TimelineColors, RhombusFootageSeekPoint.
  • Footage availability: RhombusFootageWindow, RhombusFootageAvailability, RhombusFootageGap, RhombusRangeCoverage, FetchPresenceWindowsOptions.
  • Quality / mode: RhombusBufferedStreamQuality, RhombusRealtimeStreamQuality, RhombusConnectionMode, RhombusRealtimeConnectionMode, RhombusPlayerPaths.
  • Misc: FederatedTokenFetchResult, RhombusDashPlayerCallbacks, RhombusDashQualityCallbacks.

Helpers (most apps never need these — the components do all of this internally)

Export Purpose
fetchFederatedSessionToken(url, headers, durationSec, usedDefaultPath) Manually fetch a token.
getFederatedTokenRefreshDelayMs(args) Compute the next refresh delay from TTL + hints.
formatVodMpdUri(template, startTimeSec, durationSec) Fill {START_TIME}/{DURATION} in a VOD template.
getDefaultRhombusDashSettings() / getDefaultRhombusVodDashSettings() The Dash.js settings the SDK uses.
resolveLiveH264WebSocketUrl(options) Resolve the authed realtime socket URL yourself.
startRhombusRealtimeSession(options) Drive the WebSocket + WebCodecs decode loop onto your own canvas.
snapshotCanvasElement(canvas, opts) / snapshotVideoElement(video, opts) Capture a frame → RhombusSnapshotResult.
chooseVodAnchor, isWithinWindow, vodOffsetToWallClock, wallClockToVodOffset, shouldSwitchToLive, isAtLiveEdge Pure VOD time-math helpers used by the switching logic.
requestClipSplice(options) / fetchClipProgress(options) / buildClipDownloadUrl(options) Build your own Save Clip flow.
fetchPresenceWindows(options) / mergeFootageWindows / computeFootageGaps / computeRangeCoverage Footage-availability client + coverage math for custom gap UIs.

Browser support

  • RhombusBufferedPlayer (and RhombusPlayer in buffered mode): any modern browser with MSE (Dash.js). Broadest support.
  • RhombusRealtimePlayer (and RhombusPlayer's default live transport): needs WebCodecs VideoDecoder with H.264 — Chrome, Edge, Safari 16.4+. Firefox H.264 is still limited.
  • RhombusAudioPlayer: live audio and decoded VOD require Web Audio, WebAssembly, and Web Workers. Historical playback prefers Opus/WebM MSE and automatically falls back to decoded Rhombus Opus segments on Safari/iOS and other browsers without that MSE combination.
  • RhombusTalkback: requires a secure context (HTTPS or localhost), getUserMedia, Web Audio, and WebSockets. It prefers AudioWorklet and falls back to ScriptProcessorNode when the worklet is unavailable or blocked by CSP. The browser may show a microphone permission prompt on the first user gesture.

Audio starts muted. Call play(), goLive(), or setMuted(false) from the user's actual click/tap handler; forwarding the action through a shared playback controller preserves that activation for both the Web Audio and Dash.js transports.

RhombusPlayer feature-detects WebCodecs and auto-falls back to buffered live; for the low-level players, detect yourself:

const supportsRealtime = typeof window !== "undefined" && "VideoDecoder" in window;
return supportsRealtime
  ? <RhombusRealtimePlayer cameraUuid={id} connectionMode="wan" />
  : <RhombusBufferedPlayer cameraUuid={id} />;

Production browser and security-policy checklist

Audio playback and talkback span fetch, media, worker, WebSocket, and microphone browser subsystems. A restrictive deployment policy should explicitly allow:

  • connect-src to your application API and the exact HTTPS/WSS media hosts returned by Rhombus media-URI resolution;
  • media-src to the DASH manifest/segment hosts used by video and historical audio;
  • worker-src blob: for the worker-backed Opus decoder;
  • microphone access in the page's Permissions-Policy and, when embedded, the iframe's allow="microphone" attribute; and
  • HTTPS for every non-localhost deployment.

The talkback capture path creates an AudioWorklet module from a short-lived blob URL. When a policy or browser blocks that worklet, the SDK falls back to ScriptProcessorNode; the Opus playback decoder still requires blob workers. Prefer adding the narrow directives above over loosening the entire CSP, and test the final production policy in every supported browser.

Do not request microphone access during page load. RhombusTalkback intentionally waits for startTalking(), toggleTalking(), or requestMicrophonePermission() from a user gesture and releases capture tracks when the action ends.

For Next.js or another SSR framework, place SDK imports in a client-only media module and disable server rendering for the component that imports it. For example:

"use client";

import dynamic from "next/dynamic";

const AudioStation = dynamic(
  () => import("./AudioStation.client").then(module => module.AudioStation),
  { ssr: false }
);

AudioStation.client.tsx can then import @rhombussystems/react normally. Do not use unexported deep import paths to work around SSR; they are not part of the package contract.

Troubleshooting

Symptom Likely cause / fix
404 on /api/federated-token Token route not implemented / wrong path. Implement it or set paths.federatedToken. Check the console [Rhombus…] hint.
CORS / 401 / 403 on getMediaUris (direct mode) Token not minted with a domain authorizing this origin. Add domain server-side, or set apiOverrideBaseUrl to proxy media.
Save Clip button missing Built-in export needs proxy mode — set apiOverrideBaseUrl. Without it, use onClipRangeSelect and export yourself.
Clip download 404 The /api/clip-download route can't resolve the media host/region. Verify the route streams /media/metadata/{region}/{uuid}.mp4 with the API key.
Realtime shows black, then recovers Normal stall-watchdog reconnect. Tune stallTimeoutMs; surface onRecoveryAttempt.
Realtime never renders, no errors Browser lacks WebCodecs H.264 (e.g. Firefox). Use buffered, or let RhombusPlayer fall back.
self, window, or document is undefined during SSR The package is browser-only because Dash.js evaluates browser globals. Load the importing component client-side with SSR disabled.
LAN won't connect Browser can't reach the device host, or mixed content (HTTPS page → HTTP device). Check routing/firewall and protocol.
VOD / timeline empty for a range No recorded footage for that window. Pick a range when the camera was recording.
404 on /api/presence-windows Availability route not implemented / wrong path. Implement it (forward /camera/getPresenceWindows) or set paths.presenceWindows. Harmless otherwise: gap rendering stays off and the clip pre-check fails open.
VOD plays a "VIDEO NOT AVAILABLE" pattern Rhombus serves placeholder frames (HTTP 200) where footage doesn't exist — not a player bug. Enable timeline.fetchAvailability + the presence-windows route to surface those gaps and gate clip exports.
Short blip on quality / token change (realtime) Expected — realtime reconnects the socket. Buffered changes are seamless.
Audio says Ready or shows time but is silent Audio starts muted. Click Unmute (or call setMuted(false) directly in a user gesture), then verify volume, OS output device, and tab/site audio permissions.
Audio media response has no usable WAN/LAN URI The UUID is the wrong device type, inaccessible to this organization, or has no URI for the chosen network. Use an A100 audio-gateway UUID or DR40 device UUID and verify connectionMode.
Audio remains Connecting Check the live WebSocket in DevTools, federated query parameters, CSP connect-src, proxy response, and whether the browser can reach the selected WAN/LAN host.
Historical audio fails only on Safari/iOS The decoded fallback must fetch WASM and Rhombus Opus segments. Allow worker/WASM assets and segment hosts in CSP/CORS, and confirm token query parameters reach each segment request.
Duplicate or echoing DR40 audio Pair RhombusPlayer and RhombusAudioPlayer with the same DR40 UUID and the same controller. Matching buffered/VOD video then owns embedded audio automatically.
404 on /api/audio-talkback-capabilities This is an application-owned proxy route, not a browser-callable Rhombus endpoint. Implement it, verify apiOverrideBaseUrl/paths.audioTalkbackCapabilities, and restart any long-running backend process after deploying the route.
Talkback says Role lacks device access The API key behind the capability proxy cannot access this device through its assigned permission group, or the UUID is wrong. Use the same role/device scope expected in Console.
Talkback says Enterprise license required Assign an Enterprise device license to the selected A100/DR40. The proxy must check /license/getDeviceLicenses.
Talkback says Device speaker disabled Enable the speaker in the device's Audio Controls using an ADMIN role. Refresh/re-resolve capability afterward.
Microphone permission denied Serve the app over HTTPS (or localhost), allow microphone access in browser/OS settings, and call startTalking() or requestMicrophonePermission() directly from a user gesture.
Talk button connects but the device is silent Verify the selected UUID/type, WAN/LAN reachability, capability response, realtime WebSocket frames, and that every new socket sends ctrl before exact 1,920-byte PCM frames.
User hears their own talkback Give RhombusAudioPlayer and RhombusTalkback the same source and playback controller so incoming far-audio echo frames are suppressed during TX.
Talk remains available in history This is the documented default. Set disableTalkbackInVod and provide playbackController or viewingMode to block it.

Migrating from 2.1 → 2.2

2.2 is non-breaking. Existing video and listen-only audio code needs no changes.

The additive talkback surface includes:

  • RhombusMediaPlayer, the optional high-level video/audio/talkback facade;
  • RhombusTalkback and RhombusMicrophoneControl;
  • talkback state, handle, capability, mode, permission, class-name, and style types;
  • RhombusPlayerPaths.audioTalkbackCapabilities;
  • shared-controller far-audio suppression while matching talkback is active.

To enable talkback:

  1. Add the application-owned /api/audio-talkback-capabilities route.
  2. Make that route enforce the API key role's device scope and normalize Enterprise licensing, speaker configuration, connectivity, and effective AEC.
  3. Ensure the realtime audio service validates the federated principal's device access for inbound talkback.
  4. Render RhombusTalkback with the same source/controller as the associated audio player.

The default remains permissive while viewing VOD. Set disableTalkbackInVod for products that require a live visual/audio context before an operator may speak.

Migrating from 2.0 → 2.1

2.1 is non-breaking. Existing video-only code can upgrade without changes.

The new surface is additive:

  • use RhombusAudioPlayer for A100 or DR40 live/historical audio;
  • use useRhombusPlaybackController when video, audio, and an optional standalone Timeline must share play/pause, epoch-ms position, rate, mute, volume, and seeks;
  • pass playbackController to an existing RhombusPlayer without changing its standalone behavior when the prop is absent;
  • Timeline.cameraUuid and Timeline.onSeek are now optional for controller-driven timelines; cameraUuid remains required when fetching seekpoints or availability;
  • RhombusMediaBaseProps contains source-independent auth/network/recovery props, while the existing RhombusPlayerBaseProps name remains exported for video code;
  • RhombusPlayerPaths adds audioMediaUris, audioGatewayMediaUris, and dr40MediaUris.

Add the /api/audio-media-uris proxy route before using proxy-mode audio. Existing /api/media-uris video routes are unchanged.

Migrating from 1.x → 2.0

2.0 is mostly additive — it introduces the unified RhombusPlayer, the standalone Timeline, the RhombusPlayerControl constant, and the snapshot / clip / VOD-time helpers. None of that requires changes to existing 1.x code.

The major bump is warranted by one breaking behavioral change:

⚠️ Breaking: realtime onReady now fires on every (re)connect

RhombusRealtimePlayer's onReady prop — and the onReady option of the exported startRhombusRealtimeSession helper — used to fire once per mount (only the first successful WebSocket connection). In 2.0 it fires every time the socket reaches OPEN — the initial connect and each successful auto-reconnect (after a stall, network drop, or token-refresh reconnect).

This makes it symmetric with onRecoveryAttempt (fire on drop → clear on reconnect), but it means any onReady handler you used for one-time setup will now run repeatedly.

Who is affected: only code using RhombusRealtimePlayer onReady (or startRhombusRealtimeSession onReady) to do something that must happen once. If you only used onReady to hide a "connecting…" indicator, no change is needed — the extra firings are harmless (and arguably better).

How to migrate — guard one-time work yourself:

// 1.x — relied on onReady firing exactly once:
<RhombusRealtimePlayer
  cameraUuid={id}
  connectionMode="wan"
  onReady={runOnceSetup}
/>

// 2.0 — make the once-only intent explicit; do per-connect work freely:
function LiveView({ id }: { id: string }) {
  const didInit = useRef(false);
  return (
    <RhombusRealtimePlayer
      cameraUuid={id}
      connectionMode="wan"
      onReady={() => {
        clearReconnectingBanner();          // fine to run on every (re)connect
        if (!didInit.current) {
          didInit.current = true;
          runOnceSetup();                    // runs only on the first connect
        }
      }}
    />
  );
}

RhombusBufferedPlayer's onReady is unchanged (it fires when Dash.js initializes and the manifest loads). This change is realtime-only. If you specifically need the old fire-once realtime behavior and don't want to guard it yourself, open an issue — a one-shot option could be reintroduced.

Not breaking (no action needed)

  • The auth/endpoint/resilience props were consolidated into a shared RhombusPlayerBaseProps type, but RhombusBufferedPlayerProps / RhombusRealtimePlayerProps keep the same shape.
  • Both players now accept a ref (forwardRef) — additive; existing usage is unaffected.
  • RhombusPlayerPaths gained an optional footageSeekpoints field.
  • RhombusPlayerControl is exported as a named constant and a string union, so existing string literals keep working.

License

MIT

About

Rhombus SDK for React projects

Resources

Readme
Activity
Custom properties

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages