Gamelayer

docs/source/api/serving.rst

@@ -41,6 +41,37 @@ The reusable applications provide concrete examples of the shared serving stack:
 - OmniDreams model wiring under ``integrations_v2/omnidreams/`` and serving
   under ``apps/interactive_drive/``.
 
+Application-served browser UI
+-----------------------------
+
+The v2 WebRTC runtime serves one minimal viewer at ``/`` for every application.
+An application that implements ``IWebUiProvider``
+(``flashdreams.api_v2.web_ui``) also has its own page and session endpoints
+served:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 34 66
+
+   * - Route
+     - Serves
+   * - ``GET /request_session``
+     - ``index.html`` from the application's web root.
+   * - ``GET /<file>``
+     - Any other file in that web root, resolved inside it only.
+   * - ``GET /api/session/initial_scene``
+     - The application's scene, verbatim as JSON.
+   * - ``GET /api/session/first_frame``
+     - The session's first frame, or ``404`` before one exists.
+   * - ``POST /api/session/input``
+     - One page submission, answered with the resulting scene.
+
+These routes answer ``404`` for an application that does not implement the
+protocol, so adding it changes nothing for the others. The serving layer never
+inspects a scene: prompt, event, and upload semantics belong to the
+application. ``integrations_v2/lingbot/`` is the reference implementation; see
+:doc:`/models/lingbot_world`.
+
 Launch patterns
 ---------------
 

docs/source/models/lingbot_world.rst

@@ -76,6 +76,35 @@ Application arguments follow ``--``. Run
 ``flashdreams-run-v2 cam2v-lingbot -- --help`` for custom first-frame,
 intrinsics, prompt, and motion-normalizer inputs.
 
+Browser UI
+----------
+
+The run above serves two clients:
+
+- ``http://<host>:8089/`` — the runtime's minimal viewer, offered by every v2
+  application.
+- ``http://<host>:8089/request_session`` — the Lingbot scene UI, with a preset
+  picker, first-frame upload, prompt box, and buttons for the scene's text
+  events.
+
+A scene is a prompt, a first frame, and a catalog of text events. Triggering an
+event replaces the rollout's cross-attention text context in place, so the
+scene changes without restarting the session; clearing it restores the base
+prompt. Changing the prompt — including by picking another preset — swaps that
+context too, so the scene changes while the rollout keeps running. The first
+frame is the exception: a rollout cannot replace the frame it was initialized
+from, so pick a preset before connecting to start from its image.
+
+Append ``?manual`` to skip auto-connect — only one session runs per process, so
+a stray browser tab can otherwise claim it — or ``?preset=<slug>`` to open a
+specific scene, such as ``?preset=water-blaster``.
+
+The page and its presets ship in ``integrations_v2/lingbot/apps/cam2v/web/``.
+It is served because ``LingbotCam2VApplication`` implements
+``flashdreams.api_v2.web_ui.IWebUiProvider``, which also adds the
+``/api/session/initial_scene``, ``/api/session/first_frame``, and
+``/api/session/input`` endpoints the page calls.
+
 Sample data is downloaded from the
 `LingBot-World v2 repository <https://github.com/Robbyant/lingbot-world-v2/tree/main/examples>`_.
 Valid ``--example-idx`` values are ``0, 1, 2, 5``. Note the single GPU command might run

docs/source/quickstart/index.rst

@@ -62,9 +62,10 @@ and streams the generated camera view to a browser over WebRTC:
        interactive-drive-omnidreams --mode webrtc \
        --host 0.0.0.0 --port 8089
 
-Then open ``http://<server-ip>:8089/request_session`` in a browser on the same network
-(use ``localhost`` on the same machine). The first launch spends several
-minutes loading checkpoints and compiling kernels; later launches reuse
+Then open ``http://<server-ip>:8089/`` in a browser on the same network (use
+``localhost`` on the same machine). An application serving its own page, such
+as Lingbot, also offers it at ``/request_session``. The first launch spends
+several minutes loading checkpoints and compiling kernels; later launches reuse
 the cached assets.
 
 Inspect the application arguments without loading checkpoints:

flashdreams/flashdreams/api_v2/loop.py

@@ -5,6 +5,7 @@
 
 from __future__ import annotations
 
+import logging
 import queue
 import threading
 import time
@@ -14,6 +15,7 @@
 from enum import Enum
 from typing import TYPE_CHECKING, Any, Generic, TypeVar, final
 
+from loguru import logger
 from torch import Tensor
 
 from flashdreams.runtime_v2.event_buffer import EventBuffer
@@ -24,6 +26,8 @@
 )
 from flashdreams.runtime_v2.user_input_events import UserInputEvents
 
+_LOGGER = logging.getLogger(__name__)
+
 if TYPE_CHECKING:
     from flashdreams.runtime_v2.presentation_manager import PresentationManager
     from flashdreams.runtime_v2.session_desc import SessionDesc
@@ -471,6 +475,9 @@ def _parse_lifecycle_events(
 ) -> _LoopRunResult | None:
     """Return the terminal lifecycle request, prioritizing close."""
     if any(isinstance(event, CloseUserInputEvent) for event in events):
+        # A client can end the whole run this way, which otherwise looks from
+        # the outside like the server exiting on its own.
+        logger.info("A client sent a close event; stopping the session.")
         return _LoopRunResult(stop_requested=True)
     return None
 

flashdreams/flashdreams/api_v2/web_ui.py

@@ -0,0 +1,72 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+"""Optional application hook for serving an application-owned browser UI.
+
+The v2 WebRTC server ships one minimal viewer for every application. An
+application that wants its own richer page -- a scene picker, an event panel,
+a heads-up display -- implements :class:`IWebUiProvider` and gets its files
+served alongside that viewer, plus three endpoints the page can call.
+
+The serving layer stays generic: it copies :meth:`IWebUiProvider.initial_scene`
+into a JSON response and hands :meth:`IWebUiProvider.apply_session_input` the
+decoded request body without inspecting either. What a "scene" is, and which
+inputs a page may change, belong entirely to the application.
+
+Applications that do not implement this protocol are unaffected: the server
+registers none of these routes and serves its built-in viewer as before.
+"""
+
+from collections.abc import Mapping
+from pathlib import Path
+from typing import Any, Protocol, runtime_checkable
+
+
+@runtime_checkable
+class IWebUiProvider(Protocol):
+    """An application that serves its own browser UI."""
+
+    def web_root(self) -> Path:
+        """Return the directory holding the application's web assets.
+
+        Its ``index.html`` is served at ``/request_session``; the remaining
+        files are served by name. Paths are resolved inside this directory
+        only, so a request cannot escape it.
+        """
+        ...
+
+    def initial_scene(self) -> Mapping[str, Any]:
+        """Return what the page needs to render before any frame arrives.
+
+        Returned verbatim as JSON from ``GET /api/session/initial_scene`` and
+        again from ``POST /api/session/input``, so a page can render the
+        result of its own change without a second request.
+        """
+        ...
+
+    def first_frame(self) -> tuple[bytes, str] | None:
+        """Return the session's first frame as ``(data, content_type)``.
+
+        ``None`` when the session has no first frame to show yet, which the
+        server reports as ``404`` rather than an error.
+        """
+        ...
+
+    def apply_session_input(self, payload: Mapping[str, Any]) -> Mapping[str, Any]:
+        """Apply one page-submitted change and return the resulting scene.
+
+        Args:
+            payload: Decoded request body. Multipart uploads arrive with file
+                parts as ``bytes`` and every other part as ``str``.
+
+        Returns:
+            The scene as :meth:`initial_scene` would report it afterwards.
+
+        Raises:
+            ValueError: The payload is not a change this application accepts.
+                The server reports it as ``400`` with the message.
+        """
+        ...
+
+
+__all__ = ["IWebUiProvider"]

flashdreams/flashdreams/runtime_v2/README.md

@@ -260,6 +260,40 @@ replacement must keep the negotiated layout, frame rates, width, and height.
 What a sink expects of the pixel values it is handed is part of the result
 contract, in [`api_v2`](../api_v2/README.md#what-a-step-returns).
 
+## Serving an application's own browser UI
+
+The WebRTC mode ships one minimal viewer at `/`, which every application gets.
+An application wanting a richer page — a scene picker, an event panel, a
+heads-up display — implements `IWebUiProvider` from `flashdreams.api_v2.web_ui`
+and returns four things: the directory holding its web assets, the scene the
+page renders before any frame arrives, the session's first frame, and what to
+do with one page submission.
+
+Doing so adds five routes to that run:
+
+| Route | Serves |
+| --- | --- |
+| `GET /request_session` | `index.html` from the application's web root |
+| `GET /<file>` | any other file in that web root, resolved inside it only |
+| `GET /api/session/initial_scene` | `initial_scene()`, verbatim as JSON |
+| `GET /api/session/first_frame` | `first_frame()`, or 404 before one exists |
+| `POST /api/session/input` | `apply_session_input()`, then the resulting scene |
+
+The serving layer stays generic: it copies the scene into a JSON response and
+hands the decoded request body back without inspecting either, so what a scene
+is belongs to the application. JSON, form, and multipart bodies all arrive as
+one flat mapping, with uploaded files as `bytes` under their field name. An
+application raising `ValueError` becomes a 400 carrying its message.
+
+The wiring runs through `ClientWindowMode.attach_application`, called once
+after the window is created. Only `_WebRTCMode` overrides it, and only for an
+application implementing the protocol — every other application keeps exactly
+the routes it had before, since the handlers answer 404 without a provider.
+
+`ClientWindowMode.create` builds the window before the application attaches,
+and aiohttp freezes its router at startup, so these routes are registered up
+front and `WebRTCServer.serve_web_ui()` fills in the application behind them.
+
 ## Adding a mode
 
 A mode is one way to watch a run. Subclass `ClientWindowMode` in

flashdreams/flashdreams/runtime_v2/application_runner.py

@@ -9,6 +9,8 @@
 import time
 from collections.abc import Sequence
 
+from loguru import logger
+
 from flashdreams.api_v2.application import IApplication
 from flashdreams.api_v2.client_window import IClientWindow
 from flashdreams.api_v2.output_sink import OutputSink
@@ -82,10 +84,17 @@ def run(
         try:
             self._application.init(commandline_args)
             next_session_desc: SessionDesc | None = session_desc
+            session_count = 0
             while next_session_desc is not None:
                 if deadline is not None and time.monotonic() >= deadline:
+                    logger.info(
+                        "Application run ending: the {}s timeout expired.",
+                        timeout_seconds,
+                    )
                     break
                 session = self._application.create_session(next_session_desc)
+                session_count += 1
+                logger.info("Session {} starting.", session_count)
                 session_run_started = True
                 remaining_seconds = (
                     None if deadline is None else max(0.0, deadline - time.monotonic())
@@ -100,8 +109,27 @@ def run(
                 except BaseException:
                     # ``run_session`` closes the window on every failure.
                     window_needs_close = False
+                    logger.info(
+                        "Session {} failed; the run ends with it.", session_count
+                    )
                     raise
                 window_needs_close = next_session_desc is not None
+                # A run that stops without saying so is indistinguishable from
+                # a crash to whoever is watching the server, so every way out
+                # of this loop says which one it was.
+                if next_session_desc is None:
+                    logger.info(
+                        "Session {} ended and asked for no replacement, so the "
+                        "application run is complete. A session ends when the "
+                        "client closes the window, the model finishes its "
+                        "requested steps, or the UI stops.",
+                        session_count,
+                    )
+                else:
+                    logger.info(
+                        "Session {} ended and requested a replacement.",
+                        session_count,
+                    )
         finally:
             if window_needs_close:
                 _close_client_window(self._client_window)

flashdreams/flashdreams/runtime_v2/cli.py

@@ -79,6 +79,7 @@ def entrypoint(argv: Sequence[str] | None = None) -> None:
         return
     session_desc = _session_desc(application, parsed)
     window = mode.create(parsed)
+    mode.attach_application(window, application)
     _report(mode.starting(window))
     # The session's UI and client input decide when the run ends.
     metrics_output_sink = (

flashdreams/flashdreams/runtime_v2/client_window_factory.py

@@ -14,7 +14,9 @@
 from pathlib import Path
 from typing import TYPE_CHECKING, cast
 
+from flashdreams.api_v2.application import IApplication
 from flashdreams.api_v2.client_window import IClientWindow
+from flashdreams.api_v2.web_ui import IWebUiProvider
 from flashdreams.runtime_v2.mp4_client_window import Mp4ClientWindow
 
 if TYPE_CHECKING:
@@ -49,6 +51,16 @@ def create(self, parsed_args: argparse.Namespace) -> IClientWindow:
             ValueError: Whatever :meth:`check_arguments` reports.
         """
 
+    def attach_application(
+        self, client_window: IClientWindow, application: IApplication
+    ) -> None:
+        """Give the window whatever the application offers this mode.
+
+        Called once, after :meth:`create` and before the run. Most modes want
+        nothing: a file has no place to put an application's browser UI.
+        """
+        del client_window, application
+
     def starting(self, client_window: IClientWindow) -> str | None:
         """Return what to tell the user before the run, such as where to watch."""
         del client_window
@@ -100,6 +112,13 @@ def create(self, parsed_args: argparse.Namespace) -> IClientWindow:
 
         return WebRTCClientWindow(host=parsed_args.host, port=parsed_args.port)
 
+    def attach_application(
+        self, client_window: IClientWindow, application: IApplication
+    ) -> None:
+        """Serve the application's own browser UI when it has one."""
+        if isinstance(application, IWebUiProvider):
+            cast("WebRTCClientWindow", client_window).serve_web_ui(application)
+
     def starting(self, client_window: IClientWindow) -> str | None:
         """Return where to connect, which nobody can guess when the port is free."""
         server = cast("WebRTCClientWindow", client_window).server