diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..e5fa7a095 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,100 @@ +# Recordly working agreement + +## Scope + +Recordly is an Electron desktop screen recorder/editor. The renderer is React + TypeScript; +PixiJS and canvas compose previews and exported frames. Capture and fast export also have +platform-native helpers. Changes that look renderer-only can therefore affect Windows, +macOS, Linux, MP4, and GIF differently. + +## Toolchain and setup + +- Use Node.js 22, matching `.github/workflows/quality.yml` and build/release CI. +- For quality-only work, install exactly as CI does: `npm ci --ignore-scripts`. +- A normal `npm install` runs `scripts/postinstall.mjs`, which rebuilds `uiohook-napi` and + all platform helpers. Do not use it merely to run TypeScript tests; on Windows it can + invoke Visual Studio/CMake and overwrite tracked helper binaries/manifests. +- `npm run dev` starts the Vite/Electron development app (not verified in this checkout; + camera and runtime smoke require an interactive desktop session). +- `npm run build` builds native helpers, typechecks, bundles, smokes the Electron main + entry, and packages the current platform (not verified locally; this is a heavy + platform build rather than the normal inner-loop check). + +On this machine, the PowerShell `npm` shim may fail with: + +`Cannot find module 'C:\Users\dodzi\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js'` + +That is a broken npm-prefix/shim resolution, not a Recordly failure. Use a repaired Node +22 installation. As a temporary diagnostic workaround, invoke the installed CLI directly: + +`node "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" ` + +## Verified quality commands + +Run these from the repository root: + +- `npx tsc --noEmit` — verified passing on 2026-08-25. +- `npm run lint` — verified exit 0 on 2026-08-25; it currently reports existing hook + dependency warnings, so inspect new warnings in touched files. +- `npm test` — verified on 2026-08-25: 111 files passed, 1020 tests passed, 1 skipped. +- `npm run i18n:check` — verified passing on 2026-08-25. +- `npm run format:check` — verified failing on 2026-08-25 with 122 pre-existing format + errors. CI deliberately treats this check as advisory. Do not bulk-format unrelated + files to make a focused change pass; format touched code and report the baseline debt. + +If Vitest/esbuild fails under an agent sandbox with `Cannot read directory "../../..": +Access is denied` and `Could not resolve ... vitest.config.ts`, rerun the same test outside +the filesystem sandbox. Otherwise a sandbox boundary looks like a code/test failure. + +Windows Application Control can block freshly built executables or test DLLs with +`0x800711C7` / `Application Control policy has blocked this file`. Rebuild and rerun; +changing product code in response can hide an environmental failure. + +## Rendering and export invariants + +- Preview/export parity is a product requirement. Editor webcam preview is a DOM video + layer in `VideoPlayback.tsx`; MP4/WebCodecs uses `modernFrameRenderer.ts`; GIF uses + `frameRenderer.ts`. A webcam visual effect is incomplete until all three agree. +- `ModernVideoExporter` can bypass JavaScript frame composition through native static- + layout exporters. When adding an effect unsupported by the native compositors, add an + explicit native skip reason and test it; otherwise preview is correct but some MP4 + exports silently omit the effect. +- Webcam media is recorded as a separate sidecar with `timeOffsetMs`. Preserve raw media + and the synchronization logic in `videoPlayback/webcamSync.ts`; destructive processing + at capture time removes editability and risks screen/webcam drift. +- Extensions have ordered render hooks, including `post-webcam`. Preserve hook ordering + when moving webcam composition, or extensions will render on the wrong layer. +- Preview and export intentionally have fallback paths for decoder/media-element and + Pixi WebGPU/WebGL failures. New effects must fail soft (unprocessed webcam plus warning) + instead of dropping the webcam or aborting an export. + +## Project and settings compatibility + +- `.recordly` projects persist `WebcamOverlaySettings`. Add new webcam fields with safe + defaults in `types.ts` and normalization in `projectPersistence.ts`; old projects must + continue to load without a version bump unless the wire shape truly becomes incompatible. +- Keep settings non-destructive: raw webcam footage remains the source of truth and editor + controls determine rendering at preview/export time. +- User-visible settings keys must exist in every locale. Run `npm run i18n:check`; missing + parity means some localized settings panels show fallback/missing text. +- `LICENSE.md` and the README identify Recordly as AGPL-3.0; `CONTRIBUTING.md` currently + and incorrectly points contributors to an MIT `LICENSE` path. Treat the canonical license + file as authoritative, and verify the license and redistribution terms of bundled ML + runtimes or model assets before committing them. + +## Native helper rules + +- Windows helper sources and the tracked binaries under `electron/native/bin/win32-x64` + are tied to `helpers-manifest.json` fingerprints. Use the corresponding + `scripts/build-*.mjs` command after source changes; hand-copying a binary without the + manifest makes builds reject or unknowingly reuse stale helpers. +- Full package validation is platform-specific. Follow `.github/workflows/build.yml` for + FFmpeg installation, Electron native dependency rebuilds, packaging targets, and + `npm run smoke:packaged-binaries` rather than inventing a local release sequence. + +## Onboarding provenance + +This guide was derived from repository files and freshly run commands on 2026-08-25. +No Recordly-specific auto-memory entries existed, and the repository owner had just forked +the project, so there is no human-supplied tribal-knowledge tier yet. Add only recurring, +expensive-to-rediscover lessons here as they emerge. diff --git a/docs/superpowers/plans/2026-08-25-webcam-background-blur.md b/docs/superpowers/plans/2026-08-25-webcam-background-blur.md new file mode 100644 index 000000000..d12cc188e --- /dev/null +++ b/docs/superpowers/plans/2026-08-25-webcam-background-blur.md @@ -0,0 +1,20 @@ +# Webcam background blur implementation plan + +1. Add failing tests for setting normalization, project/editor persistence, dirty state, + recording preferences, recording-session snapshots, and manifests. Implement the + shared settings type and optional compatible fields. +2. Add the pinned MediaPipe/TFJS dependencies, build-time asset plugin, asset smoke check, + dynamic segmentation chunk, and packaged Apache notices. Verify dev and production + asset URLs without committing generated binaries. +3. Add a tested lazy blur engine with serialized initialization/inference, 15 FPS preview + throttling, timestamp caching, stale-result suppression, retry/dispose, and raw fallback. +4. Integrate the launch popover and recording HUD through the existing preview stream. + Add localized switch/slider/status controls and preserve the raw MediaRecorder stream. +5. Integrate editor playback using the synchronized webcam element. Refresh on playback, + seeks, source changes, toggles, and strength changes. +6. Feed processed canvases into Canvas2D/GIF and Pixi/WebCodecs webcam composition before + existing presentation effects and hooks. Preflight exports and add the explicit native + skip reason. +7. Run targeted tests after each slice, then full TypeScript, lint, Vitest, i18n, touched + formatting, Vite build, asset smoke, packaged Windows smoke, and interactive camera and + export checks. Commit only after fresh verification. diff --git a/docs/superpowers/specs/2026-08-25-webcam-background-blur.md b/docs/superpowers/specs/2026-08-25-webcam-background-blur.md new file mode 100644 index 000000000..1e016317a --- /dev/null +++ b/docs/superpowers/specs/2026-08-25-webcam-background-blur.md @@ -0,0 +1,51 @@ +# Webcam background blur specification + +## Goal + +Add a non-destructive, local-only webcam background blur to Recordly's setup preview, +recording HUD, editor preview, MP4 export, and GIF export. The raw webcam sidecar remains +unchanged and editable. + +## Product behavior + +- A fresh install starts with blur disabled and strength 12. +- The webcam controls expose a `Blur background` switch and an integer strength slider + from 1 through 20. +- The launch choice is remembered and snapshotted into a new recording session. Editor + changes also become the preference for the next recording. +- Old projects, imported webcam media, and old recording manifests default to blur off. +- While the local model loads, Recordly shows raw webcam video. Model or inference failures + leave the webcam visible and warn once rather than aborting recording or export. +- The effect runs locally from packaged assets. It never uploads frames or downloads a + model at runtime. + +## Technical design + +- Add `WebcamBackgroundBlurSettings { enabled, amount }` under + `WebcamOverlaySettings.backgroundBlur`, recording preferences, and optional version-2 + recording-session manifest data. Project wire versions do not change. +- Use `@tensorflow-models/body-segmentation` with MediaPipe Selfie Segmentation's + landscape model. Lazy-load one serialized segmenter per renderer and composite with + threshold 0.5, edge blur 3, and the selected strength. +- A Vite plugin serves a fixed whitelist of MediaPipe package assets in development and + emits them into `dist` for packaged builds. Third-party notices ship beside them. +- Interactive previews infer at most 15 times per second and reuse the latest processed + frame. Exports process each distinct webcam source timestamp and cache repeated frames. +- Processing happens before existing webcam crop, mirror, shape, shadow, and + `post-webcam` extension hooks. +- Blurred jobs are ineligible for the native static-layout compositor and use the existing + software frame renderer. Native encoders that consume software-rendered frames remain + available. + +## Acceptance + +- Setup preview, HUD, editor, MP4, and GIF visibly agree at the same strength. +- Mirroring, cropping, seeks, speed changes, and webcam time offsets remain correct. +- Saving and reopening a project preserves blur; opening legacy data remains unblurred. +- A packaged Windows build loads the model with networking disabled. +- Missing/corrupt assets and inference errors fall back to raw webcam with one warning. + +## Exclusions + +No virtual camera, transparent background, replacement image, native C++/CUDA model, or +destructive capture-time processing is included. diff --git a/electron/electron-env.d.ts b/electron/electron-env.d.ts index 8a4eedd63..a2808ee91 100644 --- a/electron/electron-env.d.ts +++ b/electron/electron-env.d.ts @@ -718,6 +718,7 @@ interface Window { webcamPath?: string | null; timeOffsetMs?: number; hideOverlayCursorByDefault?: boolean; + webcamBackgroundBlur?: { enabled: boolean; amount: number }; }, options?: { preserveProjectPath?: boolean }, ) => Promise<{ success: boolean }>; @@ -728,6 +729,7 @@ interface Window { webcamPath?: string | null; timeOffsetMs?: number; hideOverlayCursorByDefault?: boolean; + webcamBackgroundBlur?: { enabled: boolean; amount: number }; }; }>; getCurrentVideoPath: () => Promise<{ success: boolean; path?: string }>; @@ -881,6 +883,7 @@ interface Window { microphoneEnabled: boolean; microphoneDeviceId?: string; systemAudioEnabled: boolean; + webcamBackgroundBlur: { enabled: boolean; amount: number }; }>; getRecordingAudioLabConfig: () => Promise<{ browserMicrophoneProfile: string; @@ -890,6 +893,7 @@ interface Window { microphoneEnabled?: boolean; microphoneDeviceId?: string; systemAudioEnabled?: boolean; + webcamBackgroundBlur?: { enabled: boolean; amount: number }; }) => Promise<{ success: boolean; error?: string }>; /** Countdown timer before recording */ getCountdownDelay: () => Promise<{ success: boolean; delay: number }>; diff --git a/electron/ipc/project/session.test.ts b/electron/ipc/project/session.test.ts new file mode 100644 index 000000000..e7268519c --- /dev/null +++ b/electron/ipc/project/session.test.ts @@ -0,0 +1,66 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; + +vi.mock("electron", () => ({ app: { getPath: () => "C:\\RecordlyTest" } })); +import { + getRecordingSessionManifestPath, + persistRecordingSessionManifest, + resolveRecordingSessionManifest, +} from "./session"; + +const temporaryDirectories: string[] = []; + +async function createRecordingFixture() { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), "recordly-blur-session-")); + temporaryDirectories.push(directory); + const videoPath = path.join(directory, "recording.mp4"); + const webcamPath = path.join(directory, "recording-webcam.webm"); + await Promise.all([fs.writeFile(videoPath, "video"), fs.writeFile(webcamPath, "webcam")]); + return { videoPath, webcamPath }; +} + +afterEach(async () => { + await Promise.all( + temporaryDirectories + .splice(0) + .map((directory) => fs.rm(directory, { recursive: true, force: true })), + ); +}); + +describe("recording session background blur", () => { + it("round-trips the optional blur snapshot in a version-2 manifest", async () => { + const { videoPath, webcamPath } = await createRecordingFixture(); + await persistRecordingSessionManifest({ + videoPath, + webcamPath, + timeOffsetMs: 25, + webcamBackgroundBlur: { enabled: true, amount: 17 }, + }); + + await expect(resolveRecordingSessionManifest(videoPath)).resolves.toMatchObject({ + videoPath, + webcamPath, + timeOffsetMs: 25, + webcamBackgroundBlur: { enabled: true, amount: 17 }, + }); + }); + + it("defaults old manifests to blur off", async () => { + const { videoPath, webcamPath } = await createRecordingFixture(); + await fs.writeFile( + getRecordingSessionManifestPath(videoPath), + JSON.stringify({ + version: 2, + videoFileName: path.basename(videoPath), + webcamFileName: path.basename(webcamPath), + }), + "utf-8", + ); + + await expect(resolveRecordingSessionManifest(videoPath)).resolves.toMatchObject({ + webcamBackgroundBlur: { enabled: false, amount: 12 }, + }); + }); +}); diff --git a/electron/ipc/project/session.ts b/electron/ipc/project/session.ts index 3c126e6d6..b7df4203c 100644 --- a/electron/ipc/project/session.ts +++ b/electron/ipc/project/session.ts @@ -1,6 +1,7 @@ import { constants as fsConstants } from "node:fs"; import fs from "node:fs/promises"; import path from "node:path"; +import { normalizeWebcamBackgroundBlurSettings } from "../../../src/lib/webcamBackgroundBlur"; import { RECORDING_SESSION_MANIFEST_SUFFIX } from "../constants"; import type { RecordingSessionData, RecordingSessionManifest } from "../types"; import { normalizeVideoSourcePath, parseJsonWithByteOrderMark } from "../utils"; @@ -15,7 +16,9 @@ export function getRecordingSessionManifestPath(videoPath: string) { return path.join(path.dirname(videoPath), `${baseName}${RECORDING_SESSION_MANIFEST_SUFFIX}`); } -export async function persistRecordingSessionManifest(session: RecordingSessionData): Promise { +export async function persistRecordingSessionManifest( + session: RecordingSessionData, +): Promise { const normalizedVideoPath = normalizeVideoSourcePath(session.videoPath); if (!normalizedVideoPath) { return; @@ -34,6 +37,7 @@ export async function persistRecordingSessionManifest(session: RecordingSessionD videoFileName: path.basename(normalizedVideoPath), webcamFileName: path.basename(normalizedWebcamPath), timeOffsetMs: normalizeRecordingTimeOffsetMs(session.timeOffsetMs), + webcamBackgroundBlur: normalizeWebcamBackgroundBlurSettings(session.webcamBackgroundBlur), }; await fs.writeFile(manifestPath, JSON.stringify(manifest, null, 2), "utf-8"); @@ -51,8 +55,7 @@ export async function resolveRecordingSessionManifest( try { const content = await fs.readFile(manifestPath, "utf-8"); - const parsed = - parseJsonWithByteOrderMark>(content); + const parsed = parseJsonWithByteOrderMark>(content); if (parsed.version !== 1 && parsed.version !== 2) { return null; } @@ -67,6 +70,9 @@ export async function resolveRecordingSessionManifest( videoPath: normalizedVideoPath, webcamPath: null, timeOffsetMs: normalizeRecordingTimeOffsetMs(parsed.timeOffsetMs), + webcamBackgroundBlur: normalizeWebcamBackgroundBlurSettings( + parsed.webcamBackgroundBlur, + ), }; } @@ -80,6 +86,9 @@ export async function resolveRecordingSessionManifest( videoPath: normalizedVideoPath, webcamPath: webcamExists ? webcamPath : null, timeOffsetMs: normalizeRecordingTimeOffsetMs(parsed.timeOffsetMs), + webcamBackgroundBlur: normalizeWebcamBackgroundBlurSettings( + parsed.webcamBackgroundBlur, + ), }; } catch { return null; @@ -136,7 +145,6 @@ export async function resolveRecordingSession( return { videoPath: normalizedVideoPath, webcamPath: linkedWebcamPath, + webcamBackgroundBlur: normalizeWebcamBackgroundBlurSettings(undefined), }; } - - diff --git a/electron/ipc/recordingPreferences.test.ts b/electron/ipc/recordingPreferences.test.ts new file mode 100644 index 000000000..d1f4a2014 --- /dev/null +++ b/electron/ipc/recordingPreferences.test.ts @@ -0,0 +1,28 @@ +import { describe, expect, it } from "vitest"; +import { normalizeRecordingPreferences } from "./recordingPreferences"; + +describe("normalizeRecordingPreferences", () => { + it("defaults legacy files to webcam blur off", () => { + expect(normalizeRecordingPreferences({ microphoneEnabled: true })).toEqual({ + microphoneEnabled: true, + microphoneDeviceId: undefined, + systemAudioEnabled: false, + webcamBackgroundBlur: { enabled: false, amount: 12 }, + }); + }); + + it("normalizes webcam blur without losing audio preferences", () => { + expect( + normalizeRecordingPreferences({ + microphoneDeviceId: "mic-1", + systemAudioEnabled: true, + webcamBackgroundBlur: { enabled: true, amount: 1_000 }, + }), + ).toEqual({ + microphoneEnabled: false, + microphoneDeviceId: "mic-1", + systemAudioEnabled: true, + webcamBackgroundBlur: { enabled: true, amount: 20 }, + }); + }); +}); diff --git a/electron/ipc/recordingPreferences.ts b/electron/ipc/recordingPreferences.ts new file mode 100644 index 000000000..901209158 --- /dev/null +++ b/electron/ipc/recordingPreferences.ts @@ -0,0 +1,27 @@ +import { + DEFAULT_WEBCAM_BACKGROUND_BLUR, + normalizeWebcamBackgroundBlurSettings, + type WebcamBackgroundBlurSettings, +} from "../../src/lib/webcamBackgroundBlur"; + +export interface RecordingPreferences { + microphoneEnabled: boolean; + microphoneDeviceId?: string; + systemAudioEnabled: boolean; + webcamBackgroundBlur: WebcamBackgroundBlurSettings; +} + +export function normalizeRecordingPreferences(value: unknown): RecordingPreferences { + const candidate = value && typeof value === "object" ? (value as Record) : {}; + return { + microphoneEnabled: candidate.microphoneEnabled === true, + microphoneDeviceId: + typeof candidate.microphoneDeviceId === "string" + ? candidate.microphoneDeviceId + : undefined, + systemAudioEnabled: candidate.systemAudioEnabled === true, + webcamBackgroundBlur: normalizeWebcamBackgroundBlurSettings( + candidate.webcamBackgroundBlur ?? DEFAULT_WEBCAM_BACKGROUND_BLUR, + ), + }; +} diff --git a/electron/ipc/register/project.ts b/electron/ipc/register/project.ts index f1fa43e26..a009c85d0 100644 --- a/electron/ipc/register/project.ts +++ b/electron/ipc/register/project.ts @@ -2,6 +2,7 @@ import { randomUUID } from "node:crypto"; import { constants as fsConstants } from "node:fs"; import fs from "node:fs/promises"; import path from "node:path"; +import { normalizeWebcamBackgroundBlurSettings } from "../../../src/lib/webcamBackgroundBlur"; import { BrowserWindow, dialog, ipcMain, shell } from "electron"; import { RECORDINGS_DIR } from "../../appPaths"; import { buildMediaUrl, getMediaServerBaseUrl } from "../../mediaServer"; @@ -608,7 +609,7 @@ export function registerProjectHandlers() { return { success: true, webcamPath: nextSession.webcamPath ?? null } }) - ipcMain.handle('set-current-recording-session', async (_, session: { videoPath: string; webcamPath?: string | null; timeOffsetMs?: number; hideOverlayCursorByDefault?: boolean }, options?: { preserveProjectPath?: boolean }) => { + ipcMain.handle('set-current-recording-session', async (_, session: { videoPath: string; webcamPath?: string | null; timeOffsetMs?: number; hideOverlayCursorByDefault?: boolean; webcamBackgroundBlur?: unknown }, options?: { preserveProjectPath?: boolean }) => { const normalizedVideoPath = normalizeVideoSourcePath(session.videoPath) ?? session.videoPath setCurrentVideoPath(normalizedVideoPath) setCurrentRecordingSession({ @@ -616,6 +617,7 @@ export function registerProjectHandlers() { webcamPath: normalizeVideoSourcePath(session.webcamPath ?? null), timeOffsetMs: normalizeRecordingTimeOffsetMs(session.timeOffsetMs), hideOverlayCursorByDefault: normalizeBoolean(session.hideOverlayCursorByDefault), + webcamBackgroundBlur: normalizeWebcamBackgroundBlurSettings(session.webcamBackgroundBlur), }); await rememberApprovedLocalReadPath(currentRecordingSession!.videoPath) await rememberApprovedLocalReadPath(currentRecordingSession!.webcamPath) diff --git a/electron/ipc/register/settings.ts b/electron/ipc/register/settings.ts index e84f63171..73907edd3 100644 --- a/electron/ipc/register/settings.ts +++ b/electron/ipc/register/settings.ts @@ -19,6 +19,7 @@ import { setCountdownRemaining, setCountdownTimer, } from "../state"; +import { normalizeRecordingPreferences } from "../recordingPreferences"; import { parseJsonWithByteOrderMark } from "../utils"; const BROWSER_MICROPHONE_PROFILE_ENV = "RECORDLY_BROWSER_MIC_PROFILE"; @@ -146,22 +147,9 @@ export function registerSettingsHandlers() { try { const content = await fs.readFile(RECORDINGS_SETTINGS_FILE, "utf-8"); const parsed = parseJsonWithByteOrderMark>(content); - return { - success: true, - microphoneEnabled: parsed.microphoneEnabled === true, - microphoneDeviceId: - typeof parsed.microphoneDeviceId === "string" - ? parsed.microphoneDeviceId - : undefined, - systemAudioEnabled: parsed.systemAudioEnabled === true, - }; + return { success: true, ...normalizeRecordingPreferences(parsed) }; } catch { - return { - success: true, - microphoneEnabled: false, - microphoneDeviceId: undefined, - systemAudioEnabled: false, - }; + return { success: true, ...normalizeRecordingPreferences(undefined) }; } }); @@ -177,6 +165,7 @@ export function registerSettingsHandlers() { microphoneEnabled?: boolean; microphoneDeviceId?: string; systemAudioEnabled?: boolean; + webcamBackgroundBlur?: unknown; }, ) => { try { diff --git a/electron/ipc/types.ts b/electron/ipc/types.ts index 58f5425bd..307040c23 100644 --- a/electron/ipc/types.ts +++ b/electron/ipc/types.ts @@ -1,3 +1,5 @@ +import type { WebcamBackgroundBlurSettings } from "../../src/lib/webcamBackgroundBlur"; + export type SelectedSource = { id?: string; name: string; @@ -48,6 +50,7 @@ export type RecordingSessionData = { webcamPath?: string | null; timeOffsetMs?: number; hideOverlayCursorByDefault?: boolean; + webcamBackgroundBlur?: WebcamBackgroundBlurSettings; }; export type PauseSegment = { @@ -60,6 +63,7 @@ export type RecordingSessionManifest = { videoFileName: string; webcamFileName?: string | null; timeOffsetMs?: number; + webcamBackgroundBlur?: WebcamBackgroundBlurSettings; }; export type ProjectLibraryEntry = { diff --git a/electron/preload.ts b/electron/preload.ts index e55d42cbd..50e930fd2 100644 --- a/electron/preload.ts +++ b/electron/preload.ts @@ -972,6 +972,7 @@ contextBridge.exposeInMainWorld("electronAPI", { microphoneEnabled?: boolean; microphoneDeviceId?: string; systemAudioEnabled?: boolean; + webcamBackgroundBlur?: { enabled: boolean; amount: number }; }) => ipcRenderer.invoke("set-recording-preferences", prefs), getCountdownDelay: () => ipcRenderer.invoke("get-countdown-delay"), setCountdownDelay: (delay: number) => ipcRenderer.invoke("set-countdown-delay", delay), diff --git a/electron/windows.ts b/electron/windows.ts index cccb9e243..5b74bf838 100644 --- a/electron/windows.ts +++ b/electron/windows.ts @@ -289,9 +289,7 @@ function setHudOverlayFallbackExpanded(expanded: boolean) { function setHudOverlayMousePassthrough(ignore: boolean) { hudOverlayIgnoringMouse = - hudOverlaySourceSelectionActive && !hudOverlayRecordingActive - ? true - : ignore; + hudOverlaySourceSelectionActive && !hudOverlayRecordingActive ? true : ignore; if (hudOverlayMouseReassertTimer) { clearTimeout(hudOverlayMouseReassertTimer); @@ -610,9 +608,14 @@ export function createHudOverlayWindow(): BrowserWindow { if (VITE_DEV_SERVER_URL) { win.loadURL(VITE_DEV_SERVER_URL + "?windowType=hud-overlay"); } else { - win.loadFile(path.join(RENDERER_DIST, "index.html"), { - query: { windowType: "hud-overlay" }, - }); + const packagedRendererBaseUrl = getPackagedRendererBaseUrl(); + if (packagedRendererBaseUrl) { + win.loadURL(`${packagedRendererBaseUrl}/?windowType=hud-overlay`); + } else { + win.loadFile(path.join(RENDERER_DIST, "index.html"), { + query: { windowType: "hud-overlay" }, + }); + } } return win; diff --git a/package-lock.json b/package-lock.json index 89ab3d499..d99d4cf45 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,12 @@ "version": "1.3.5-beta.2", "hasInstallScript": true, "dependencies": { + "@mediapipe/selfie_segmentation": "0.1.1675465747", "@phosphor-icons/react": "^2.1.10", + "@tensorflow-models/body-segmentation": "1.0.2", + "@tensorflow/tfjs-backend-webgl": "4.22.0", + "@tensorflow/tfjs-converter": "4.22.0", + "@tensorflow/tfjs-core": "4.22.0", "capturekit": "^1.0.13", "electron-updater": "^6.8.3", "ffmpeg-static": "^5.3.0", @@ -1780,6 +1785,12 @@ "node": ">= 10.0.0" } }, + "node_modules/@mediapipe/selfie_segmentation": { + "version": "0.1.1675465747", + "resolved": "https://registry.npmjs.org/@mediapipe/selfie_segmentation/-/selfie_segmentation-0.1.1675465747.tgz", + "integrity": "sha512-IxYxNhwE5VwOm52L1yoFWYLP7q9Pd+NJjzOC5tlepfvEGaY3o9hslhUrx9BgseqdfZtKSDtd/4NfCSMjNzQalA==", + "license": "Apache-2.0" + }, "node_modules/@nodelib/fs.scandir": { "version": "2.1.5", "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", @@ -3129,6 +3140,94 @@ "node": ">=10" } }, + "node_modules/@tensorflow-models/body-segmentation": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/@tensorflow-models/body-segmentation/-/body-segmentation-1.0.2.tgz", + "integrity": "sha512-sbPiL8wpqfKqh01of6qguZzU9yLWOQDTwiPEIHFPA/EAjz5T51LKHKIySll+mmteRo/TxNED8pxd9VJ6q2r7kg==", + "license": "Apache-2.0", + "dependencies": { + "rimraf": "^3.0.2" + }, + "peerDependencies": { + "@mediapipe/selfie_segmentation": "~0.1.0", + "@tensorflow/tfjs-backend-webgl": "^4.9.0", + "@tensorflow/tfjs-converter": "^4.9.0", + "@tensorflow/tfjs-core": "^4.9.0" + } + }, + "node_modules/@tensorflow/tfjs-backend-cpu": { + "version": "4.22.0", + "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-cpu/-/tfjs-backend-cpu-4.22.0.tgz", + "integrity": "sha512-1u0FmuLGuRAi8D2c3cocHTASGXOmHc/4OvoVDENJayjYkS119fcTcQf4iHrtLthWyDIPy3JiPhRrZQC9EwnhLw==", + "license": "Apache-2.0", + "dependencies": { + "@types/seedrandom": "^2.4.28", + "seedrandom": "^3.0.5" + }, + "engines": { + "yarn": ">= 1.3.2" + }, + "peerDependencies": { + "@tensorflow/tfjs-core": "4.22.0" + } + }, + "node_modules/@tensorflow/tfjs-backend-webgl": { + "version": "4.22.0", + "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-backend-webgl/-/tfjs-backend-webgl-4.22.0.tgz", + "integrity": "sha512-H535XtZWnWgNwSzv538czjVlbJebDl5QTMOth4RXr2p/kJ1qSIXE0vZvEtO+5EC9b00SvhplECny2yDewQb/Yg==", + "license": "Apache-2.0", + "dependencies": { + "@tensorflow/tfjs-backend-cpu": "4.22.0", + "@types/offscreencanvas": "~2019.3.0", + "@types/seedrandom": "^2.4.28", + "seedrandom": "^3.0.5" + }, + "engines": { + "yarn": ">= 1.3.2" + }, + "peerDependencies": { + "@tensorflow/tfjs-core": "4.22.0" + } + }, + "node_modules/@tensorflow/tfjs-converter": { + "version": "4.22.0", + "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-converter/-/tfjs-converter-4.22.0.tgz", + "integrity": "sha512-PT43MGlnzIo+YfbsjM79Lxk9lOq6uUwZuCc8rrp0hfpLjF6Jv8jS84u2jFb+WpUeuF4K33ZDNx8CjiYrGQ2trQ==", + "license": "Apache-2.0", + "peerDependencies": { + "@tensorflow/tfjs-core": "4.22.0" + } + }, + "node_modules/@tensorflow/tfjs-core": { + "version": "4.22.0", + "resolved": "https://registry.npmjs.org/@tensorflow/tfjs-core/-/tfjs-core-4.22.0.tgz", + "integrity": "sha512-LEkOyzbknKFoWUwfkr59vSB68DMJ4cjwwHgicXN0DUi3a0Vh1Er3JQqCI1Hl86GGZQvY8ezVrtDIvqR1ZFW55A==", + "license": "Apache-2.0", + "dependencies": { + "@types/long": "^4.0.1", + "@types/offscreencanvas": "~2019.7.0", + "@types/seedrandom": "^2.4.28", + "@webgpu/types": "0.1.38", + "long": "4.0.0", + "node-fetch": "~2.6.1", + "seedrandom": "^3.0.5" + }, + "engines": { + "yarn": ">= 1.3.2" + } + }, + "node_modules/@tensorflow/tfjs-core/node_modules/@types/offscreencanvas": { + "version": "2019.7.3", + "resolved": "https://registry.npmjs.org/@types/offscreencanvas/-/offscreencanvas-2019.7.3.tgz", + "integrity": "sha512-ieXiYmgSRXUDeOntE1InxjWyvEelZGP63M+cGuquuRLuIKKT1osnkXjxev9B7d1nXSug5vpunx+gNlbVxMlC9A==", + "license": "MIT" + }, + "node_modules/@tensorflow/tfjs-core/node_modules/@webgpu/types": { + "version": "0.1.38", + "resolved": "https://registry.npmjs.org/@webgpu/types/-/types-0.1.38.tgz", + "integrity": "sha512-7LrhVKz2PRh+DD7+S+PVaFd5HxaWQvoMqBbsV9fNJO1pjUs1P8bM2vQVNfk+3URTqbuTI7gkXi0rfsN0IadoBA==", + "license": "BSD-3-Clause" + }, "node_modules/@types/babel__core": { "version": "7.20.5", "resolved": "https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz", @@ -3286,6 +3385,12 @@ "@types/node": "*" } }, + "node_modules/@types/long": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/@types/long/-/long-4.0.2.tgz", + "integrity": "sha512-MqTGEo5bj5t157U6fA/BiDynNkn0YknVdh48CMPkTSpFTVmvao5UQmm7uEF6xBEo7qIMAlY/JSleYaE6VOdpaA==", + "license": "MIT" + }, "node_modules/@types/ms": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/@types/ms/-/ms-2.1.0.tgz", @@ -3303,6 +3408,12 @@ "undici-types": "~7.16.0" } }, + "node_modules/@types/offscreencanvas": { + "version": "2019.3.0", + "resolved": "https://registry.npmjs.org/@types/offscreencanvas/-/offscreencanvas-2019.3.0.tgz", + "integrity": "sha512-esIJx9bQg+QYF0ra8GnvfianIY8qWB0GBx54PK5Eps6m+xTj86KLavHv6qDhzKcu5UUOgNfJ2pWaIIV7TRUd9Q==", + "license": "MIT" + }, "node_modules/@types/plist": { "version": "3.0.5", "resolved": "https://registry.npmjs.org/@types/plist/-/plist-3.0.5.tgz", @@ -3353,6 +3464,12 @@ "@types/node": "*" } }, + "node_modules/@types/seedrandom": { + "version": "2.4.34", + "resolved": "https://registry.npmjs.org/@types/seedrandom/-/seedrandom-2.4.34.tgz", + "integrity": "sha512-ytDiArvrn/3Xk6/vtylys5tlY6eo7Ane0hvcx++TKo6RxQXuVfW0AF/oeWqAj9dN29SyhtawuXstgmPlwNcv/A==", + "license": "MIT" + }, "node_modules/@types/verror": { "version": "1.10.11", "resolved": "https://registry.npmjs.org/@types/verror/-/verror-1.10.11.tgz", @@ -4092,7 +4209,6 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz", "integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==", - "dev": true, "license": "MIT" }, "node_modules/base64-js": { @@ -4694,7 +4810,6 @@ "version": "0.0.1", "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz", "integrity": "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg==", - "dev": true, "license": "MIT" }, "node_modules/convert-source-map": { @@ -4880,18 +4995,6 @@ "node": ">=0.4.0" } }, - "node_modules/detect-libc": { - "version": "2.1.2", - "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", - "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", - "dev": true, - "license": "Apache-2.0", - "optional": true, - "peer": true, - "engines": { - "node": ">=8" - } - }, "node_modules/detect-node": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/detect-node/-/detect-node-2.1.0.tgz", @@ -5884,7 +5987,6 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz", "integrity": "sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw==", - "dev": true, "license": "ISC" }, "node_modules/fsevents": { @@ -6019,7 +6121,6 @@ "resolved": "https://registry.npmjs.org/glob/-/glob-7.2.3.tgz", "integrity": "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q==", "deprecated": "Glob versions prior to v9 are no longer supported", - "dev": true, "license": "ISC", "dependencies": { "fs.realpath": "^1.0.0", @@ -6053,7 +6154,6 @@ "version": "1.1.12", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz", "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==", - "dev": true, "license": "MIT", "dependencies": { "balanced-match": "^1.0.0", @@ -6064,7 +6164,6 @@ "version": "3.1.2", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.2.tgz", "integrity": "sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw==", - "dev": true, "license": "ISC", "dependencies": { "brace-expansion": "^1.1.7" @@ -6370,7 +6469,6 @@ "resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz", "integrity": "sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA==", "deprecated": "This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.", - "dev": true, "license": "ISC", "dependencies": { "once": "^1.3.0", @@ -6624,280 +6722,6 @@ "integrity": "sha512-0/BnGCCfyUMkBpeDgWihanIAF9JmZhHBgUhEqzvf+adhNGLoP6TaiI5oF8oyb3I45P+PcnrqihSf01M0l0G5+Q==", "license": "MIT" }, - "node_modules/lightningcss": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.32.0.tgz", - "integrity": "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ==", - "dev": true, - "license": "MPL-2.0", - "optional": true, - "peer": true, - "dependencies": { - "detect-libc": "^2.0.3" - }, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - }, - "optionalDependencies": { - "lightningcss-android-arm64": "1.32.0", - "lightningcss-darwin-arm64": "1.32.0", - "lightningcss-darwin-x64": "1.32.0", - "lightningcss-freebsd-x64": "1.32.0", - "lightningcss-linux-arm-gnueabihf": "1.32.0", - "lightningcss-linux-arm64-gnu": "1.32.0", - "lightningcss-linux-arm64-musl": "1.32.0", - "lightningcss-linux-x64-gnu": "1.32.0", - "lightningcss-linux-x64-musl": "1.32.0", - "lightningcss-win32-arm64-msvc": "1.32.0", - "lightningcss-win32-x64-msvc": "1.32.0" - } - }, - "node_modules/lightningcss-android-arm64": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.32.0.tgz", - "integrity": "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "android" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-darwin-arm64": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.32.0.tgz", - "integrity": "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "darwin" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-darwin-x64": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.32.0.tgz", - "integrity": "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "darwin" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-freebsd-x64": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.32.0.tgz", - "integrity": "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "freebsd" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-linux-arm-gnueabihf": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.32.0.tgz", - "integrity": "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw==", - "cpu": [ - "arm" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "linux" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-linux-arm64-gnu": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.32.0.tgz", - "integrity": "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "linux" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-linux-arm64-musl": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.32.0.tgz", - "integrity": "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "linux" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-linux-x64-gnu": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.32.0.tgz", - "integrity": "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "linux" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-linux-x64-musl": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.32.0.tgz", - "integrity": "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "linux" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-win32-arm64-msvc": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.32.0.tgz", - "integrity": "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "win32" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, - "node_modules/lightningcss-win32-x64-msvc": { - "version": "1.32.0", - "resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.32.0.tgz", - "integrity": "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MPL-2.0", - "optional": true, - "os": [ - "win32" - ], - "peer": true, - "engines": { - "node": ">= 12.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, "node_modules/lilconfig": { "version": "3.1.3", "resolved": "https://registry.npmjs.org/lilconfig/-/lilconfig-3.1.3.tgz", @@ -6938,6 +6762,12 @@ "deprecated": "This package is deprecated. Use require('node:util').isDeepStrictEqual instead.", "license": "MIT" }, + "node_modules/long": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/long/-/long-4.0.0.tgz", + "integrity": "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA==", + "license": "Apache-2.0" + }, "node_modules/loose-envify": { "version": "1.4.0", "resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz", @@ -7282,6 +7112,26 @@ "license": "MIT", "optional": true }, + "node_modules/node-fetch": { + "version": "2.6.13", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.13.tgz", + "integrity": "sha512-StxNAxh15zr77QvvkmveSQ8uCQ4+v5FkvNTj0OESmiHu+VRi/gXArXtkWMElOsOUNLtUEvI4yS+rdtOHZTwlQA==", + "license": "MIT", + "dependencies": { + "whatwg-url": "^5.0.0" + }, + "engines": { + "node": "4.x || >=6.0.0" + }, + "peerDependencies": { + "encoding": "^0.1.0" + }, + "peerDependenciesMeta": { + "encoding": { + "optional": true + } + } + }, "node_modules/node-gyp": { "version": "12.4.0", "resolved": "https://registry.npmjs.org/node-gyp/-/node-gyp-12.4.0.tgz", @@ -7457,7 +7307,6 @@ "version": "1.4.0", "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==", - "dev": true, "license": "ISC", "dependencies": { "wrappy": "1" @@ -7527,7 +7376,6 @@ "version": "1.0.1", "resolved": "https://registry.npmjs.org/path-is-absolute/-/path-is-absolute-1.0.1.tgz", "integrity": "sha512-AVbw3UJ2e9bq64vSaS9Am0fje1Pa8pbGqTTsmXfaIiMpnr5DlDhfJOuLj9Sf95ZPVDAUerDfEk88MPmPe7UCQg==", - "dev": true, "license": "MIT", "engines": { "node": ">=0.10.0" @@ -8367,6 +8215,22 @@ "node": ">=0.10.0" } }, + "node_modules/rimraf": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-3.0.2.tgz", + "integrity": "sha512-JZkJMZkAGFFPP2YqXZXPbMlMBgsxzE8ILs4lMIX/2o0L9UBw9O/Y3o6wFw/i9YLapcUJWwqbi3kdxIPdC62TIA==", + "deprecated": "Rimraf versions prior to v4 are no longer supported", + "license": "ISC", + "dependencies": { + "glob": "^7.1.3" + }, + "bin": { + "rimraf": "bin.js" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, "node_modules/roarr": { "version": "2.15.4", "resolved": "https://registry.npmjs.org/roarr/-/roarr-2.15.4.tgz", @@ -8504,6 +8368,12 @@ "loose-envify": "^1.1.0" } }, + "node_modules/seedrandom": { + "version": "3.0.5", + "resolved": "https://registry.npmjs.org/seedrandom/-/seedrandom-3.0.5.tgz", + "integrity": "sha512-8OwmbklUNzwezjGInmZ+2clQmExQPvomqjL7LFqOYqtmuxRgQYqOD3mHaU+MvZn5FLUeVxVfQjwLZW/n/JFuqg==", + "license": "MIT" + }, "node_modules/semver": { "version": "7.7.3", "resolved": "https://registry.npmjs.org/semver/-/semver-7.7.3.tgz", @@ -9269,6 +9139,12 @@ "node": ">=8.0" } }, + "node_modules/tr46": { + "version": "0.0.3", + "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", + "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==", + "license": "MIT" + }, "node_modules/truncate-utf8-bytes": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/truncate-utf8-bytes/-/truncate-utf8-bytes-1.0.2.tgz", @@ -9650,6 +9526,22 @@ "pnpm": ">=8" } }, + "node_modules/webidl-conversions": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", + "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==", + "license": "BSD-2-Clause" + }, + "node_modules/whatwg-url": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", + "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==", + "license": "MIT", + "dependencies": { + "tr46": "~0.0.3", + "webidl-conversions": "^3.0.0" + } + }, "node_modules/which": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz", @@ -9723,7 +9615,6 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==", - "dev": true, "license": "ISC" }, "node_modules/xmlbuilder": { diff --git a/package.json b/package.json index f930a7007..315616df3 100644 --- a/package.json +++ b/package.json @@ -39,6 +39,7 @@ "normalize:electron-main-cjs": "node scripts/normalize-electron-main-cjs.mjs", "smoke:electron-main-cjs": "node scripts/smoke-electron-main-cjs.mjs", "smoke:packaged-binaries": "node scripts/smoke-packaged-binaries.mjs", + "smoke:webcam-segmentation-assets": "node scripts/smoke-webcam-segmentation-assets.mjs", "verify:macos-distribution": "node scripts/verify-macos-distribution.mjs", "checksums:release": "node scripts/write-release-checksums.mjs", "release:create": "node scripts/create-release.mjs", @@ -46,7 +47,12 @@ "test:watch": "vitest" }, "dependencies": { + "@mediapipe/selfie_segmentation": "0.1.1675465747", "@phosphor-icons/react": "^2.1.10", + "@tensorflow-models/body-segmentation": "1.0.2", + "@tensorflow/tfjs-backend-webgl": "4.22.0", + "@tensorflow/tfjs-converter": "4.22.0", + "@tensorflow/tfjs-core": "4.22.0", "capturekit": "^1.0.13", "electron-updater": "^6.8.3", "ffmpeg-static": "^5.3.0", diff --git a/public/THIRD_PARTY_NOTICES.txt b/public/THIRD_PARTY_NOTICES.txt new file mode 100644 index 000000000..2ad1fe972 --- /dev/null +++ b/public/THIRD_PARTY_NOTICES.txt @@ -0,0 +1,212 @@ +Recordly webcam background blur includes the following Apache-2.0 components: + +- MediaPipe Selfie Segmentation, Copyright 2020 The MediaPipe Authors + https://github.com/google-ai-edge/mediapipe +- TensorFlow.js Models, Copyright 2018 The TensorFlow Authors + https://github.com/tensorflow/tfjs-models +- TensorFlow.js, Copyright 2018 The TensorFlow Authors + https://github.com/tensorflow/tfjs + +These components are licensed under the following terms: + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + +Copyright [yyyy] [name of copyright owner] + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/scripts/smoke-webcam-segmentation-assets.mjs b/scripts/smoke-webcam-segmentation-assets.mjs new file mode 100644 index 000000000..5715332e8 --- /dev/null +++ b/scripts/smoke-webcam-segmentation-assets.mjs @@ -0,0 +1,30 @@ +import fs from "node:fs"; +import path from "node:path"; +import process from "node:process"; + +const assetDirectory = path.resolve("dist", "webcam-segmentation"); +const expectedFiles = [ + "selfie_segmentation.js", + "selfie_segmentation.binarypb", + "selfie_segmentation.tflite", + "selfie_segmentation_landscape.tflite", + "selfie_segmentation_solution_simd_wasm_bin.data", + "selfie_segmentation_solution_simd_wasm_bin.js", + "selfie_segmentation_solution_simd_wasm_bin.wasm", + "selfie_segmentation_solution_wasm_bin.js", + "selfie_segmentation_solution_wasm_bin.wasm", +]; + +const missing = expectedFiles.filter( + (fileName) => !fs.existsSync(path.join(assetDirectory, fileName)), +); +if (!fs.existsSync(path.resolve("dist", "THIRD_PARTY_NOTICES.txt"))) { + missing.push("../THIRD_PARTY_NOTICES.txt"); +} + +if (missing.length > 0) { + console.error(`Missing packaged webcam segmentation assets: ${missing.join(", ")}`); + process.exit(1); +} + +console.log(`Verified ${expectedFiles.length} webcam segmentation runtime assets and notices.`); diff --git a/src/components/launch/LaunchWindow.tsx b/src/components/launch/LaunchWindow.tsx index cb43ead1b..fd261e77c 100644 --- a/src/components/launch/LaunchWindow.tsx +++ b/src/components/launch/LaunchWindow.tsx @@ -39,6 +39,7 @@ import { MorePopover } from "./popovers/MorePopover"; import { ProjectPopover } from "./popovers/ProjectPopover"; import { SourcePopover } from "./popovers/SourcePopover"; import { WebcamPopover } from "./popovers/WebcamPopover"; +import { WebcamBackgroundBlurPreview } from "@/components/webcam/WebcamBackgroundBlurPreview"; import { RecordingControls } from "./RecordingControls"; import { MarqueeText } from "./SourceSelector"; @@ -75,6 +76,8 @@ function LaunchWindowContent() { setWebcamEnabled, webcamDeviceId, setWebcamDeviceId, + webcamBackgroundBlur, + setWebcamBackgroundBlur, countdownDelay, setCountdownDelay, preparePermissions, @@ -302,6 +305,8 @@ function LaunchWindowContent() { videoDevices={videoDevices} webcamDeviceId={webcamDeviceId} selectedVideoDeviceId={selectedVideoDeviceId} + backgroundBlur={webcamBackgroundBlur} + onBackgroundBlurChange={setWebcamBackgroundBlur} onSelectVideoDevice={(deviceId) => { setWebcamEnabled(true); setSelectedVideoDeviceId(deviceId); @@ -534,8 +539,10 @@ function LaunchWindowContent() { onPointerUp={handleWebcamPreviewPointerUp} onPointerCancel={handleWebcamPreviewPointerUp} > -