origin: announce-only entries, advertise on-demand content without instantiating broadcasts

[kixelated]

Problem

Announcing a path requires instantiating a broadcast. origin.publish / create_broadcast put a live broadcast::Producer behind every announced entry, while origin.dynamic() broadcasts are deliberately never announced (#1772). So "announced" implies "instantiated", and there is no way to advertise content we could create on demand without actually starting to create it.

The concrete victim is any large on-demand catalog. A VOD library or transcoder that can serve 100k recordings must either instantiate 100k broadcast producers just to appear in announce streams, or stay invisible and force consumers to know paths out of band. #2610's dynamic announce interest (requested_announce()) surfaces the demand to the application, but its plan still has handlers respond by creating real broadcasts, which only moves the instantiation cost from attach time to interest time. Interest in a prefix should cost an enumeration, not a fleet of broadcasts.

Design

An announce-only entry: a path in the origin's announce tree with no front behind it.

• It shows up in every announce consumer and on the wire like any other announcement. The wire genuinely cannot tell: announcements never carried broadcast state, and the epoch/ended fields planned in Dynamic announcements, broadcast epochs, and ended (VOD) broadcasts #2610 attach naturally (ended: true is the VOD enumeration case). No wire change.
• A subscribe or request against it falls through to the existing dynamic/request machinery (origin.dynamic(), request_broadcast), which materializes the real broadcast on first demand. The materialized broadcast attaches at the same path and epoch and supersedes the placeholder; when it closes, the announce-only entry remains until its own handle is dropped.

Two API flavors that compose:

1. Standing: Producer::announce(path, ...) -> guard (JS: origin.announce(path)). Advertise regardless of interest; drop the guard to retract. For small known catalogs.
2. Interest-scoped: the requested_announce() handle from Dynamic announcements, broadcast epochs, and ended (VOD) broadcasts #2610 gains an announce(suffix, ...) response method. Entries live only while the interest that solicited them does: the application enumerates its catalog when some peer opens an announce stream for the prefix, and the whole subtree of announce state evaporates when the last matching stream closes.

Together with #2708 (lazy, prefix-scoped announce solicitation) this closes a fully demand-driven loop: interest flows upstream hop by hop as narrow ANNOUNCE_REQUESTs, the application at the origin answers with cheap announce-only entries, and broadcasts only materialize when someone subscribes.

Notes

• The interest ledger proposed on origin: open the announce interest lazily, on first consumer #2708 is the shared foundation; flavor 2 depends on it and on Dynamic announcements, broadcast epochs, and ended (VOD) broadcasts #2610's requested_announce(). Flavor 1 is independent and could land first.
• Model hazard to design around: announce consumers hand out broadcast::Consumer fronts today. An announce-only entry has none, so either the announce event carries a lazily-materializing front (resolving through the request machinery on first track subscribe), or announce events decouple from fronts. The former keeps the consumer API unchanged and is the likely shape.
• Cross-package sync per the table: js/net, moq-ffi and wrappers if the surface reaches them, doc/concept, and drafts/draft-lcurley-moq-lite.md only if any wire clarification falls out (none expected).

Related: #2610 (dynamic announce interest, epochs, ended broadcasts), #2708 (lazy announce solicitation), #2412 (moq-gst publish-on-demand), #1772 (dynamic broadcasts are never announced).

(written by Fable 5)

[coderabbitai]

🔗 Related PRs

#1913 - feat(moq-net): add OriginProducer::dynamic + infallible OriginConsumer::request_broadcast [merged]
#2094 - Route subscribes through dynamic origins [merged]
#2473 - feat(net): fail over across redundant publishers via per-peer route selection [merged]
#2556 - fix(net): prefer the newest route so a reconnect takes over immediately [merged]
#2617 - feat(net): add announcedBroadcast, and resolve lite restarts by publisher identity [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[kixelated]

Settled design: decouple announcements from fronts

Design pass done against dev. Decision (with @kixelated): announcements stop carrying a broadcast handle, so an advertised path costs a tree entry and nothing else. Sized for a large VOD/recording catalog (10k-100k advertised paths, a handful consumed at a time), which rules out the two cheaper-to-build shapes: a front per advertised path costs a broadcast::Producer + FrontState each, and a front + run_front task each costs a parked task per path.

This is a breaking change to OriginAnnounce, a published API, so it targets dev.

What reading the code changed

Three findings that the issue's original sketch did not account for:

1. Demand cannot exist without a front. It is a handle onto kio::Shared<BroadcastState> (model/broadcast.rs), so there is no way to synthesize one for an unmaterialized advertisement. The wire publishers use broadcast.demand() for their idle/restart bookkeeping, so an advertisement has to report "no demand" structurally (Option<Demand>), not by faking one.
2. The wire publishers need a live handle, not a snapshot. lite::Publisher::run_announce keeps the broadcast::Consumer in its watched map to observe route_changed(), and feeds routes() + demand() into select_route for per-peer split horizon. An advertisement has one static local route (no hops, cost 0) that never changes, so those paths need an explicit branch rather than a delegating shim.
3. Materialization needs no new machinery. This is what makes the decoupled shape cheap to build: for a path that is advertised but not instantiated, Consumer::resolve() already returns Missing, so request_broadcast already falls through to the dynamic queue and the application's origin.dynamic() handler. So Advertisement::resolve() is just request_broadcast(path), and a wire SUBSCRIBE against an advertised path already works today with no change at all. No placeholder fronts, no lazy front adoption inside attach_source.

Shape

pub struct OriginAnnounce {
    pub path: PathOwned,
    /// The advertisement, or `None` once the path is no longer available.
    pub active: Option<Advertisement>,
}

pub struct Advertisement { /* live: Option<broadcast::Consumer>, route, origin handle */ }

impl Advertisement {
    /// The live broadcast, when the path has one. `None` for an announce-only entry:
    /// reading this never instantiates anything, which is what lets a relay forward
    /// 100k advertisements for free.
    pub fn live(&self) -> Option<&broadcast::Consumer>;
    /// The advertised routes: the front's for a live broadcast, one local route for an
    /// announce-only entry.
    pub fn routes(&self) -> Vec<broadcast::Route>;
    /// `None` for an announce-only entry: nothing is instantiated, so nothing is read.
    pub fn demand(&self) -> Option<broadcast::Demand>;
    /// Resolve the broadcast, requesting it from the origin's dynamic handler when the
    /// path is advertised but not instantiated. Resolves synchronously for a live front.
    pub fn resolve(&self) -> kio::Pending<Requesting>;
}

AnnounceConsumer gains an origin::Consumer so each Advertisement can resolve; AnnounceProducer::consume builds one too, so the handle is never absent.

Producer side, flavor 1 from the issue:

/// Advertise `path` without instantiating a broadcast. Drop the guard to retract.
pub fn announce(&self, path: impl AsPath) -> Announce;

The tree gains OriginNode.advertised: Option<AdvertEntry> next to broadcast: Option<OriginBroadcast>.

Open decision for the implementer

A live broadcast and an advertisement at the same path. Cheapest correct rule: the front wins and the advertisement stays silent while it exists (announce the advert only when broadcast.is_none(); unannounce the advert when a front is created; re-announce it when the front goes away). Observably equivalent, since resolving an advertisement goes through request_broadcast, which prefers the announced front anyway. The cost is three touch points in the most delicate code in the crate (attach_source's first-source branch, set_announced, remove), which is where the regression risk in this PR concentrates. Worth a regression test per touch point.

Also note advertisements register no ExclusionGuard (there is no FrontState to register against). That is correct rather than a gap: an advertisement is locally originated with an empty hop chain, so it cannot loop back through a peer.

Blast radius

OriginAnnounce's field is destructured in ~20 places: rs/moq-net (lite/publisher.rs x3 + the watched route-tracking, ietf/publisher.rs, announced_broadcast, tests), rs/moq-stats (6), rs/moq-relay (internal.rs, tests/smoke.rs x6), rs/moq-native (1 test), rs/moq-ffi. Most are mechanical (announce.broadcast -> announce.active), and moq-ffi's own surface can stay unchanged by having MoqAnnouncement::broadcast() call resolve() internally, which keeps libmoq and the py/swift/kt/go wrappers out of the diff.

Not started: this is a materially bigger change than #2762 and #2775, and I would rather begin it with a clean run at it than land it half-done.

(written by Fable 5)

[kixelated]

Follow-up from the epoch design pass in #2610: offline VOD advertisement and historical generation discovery should stay separate.

This issue remains the home for cheap announce-only entries and on-demand materialization. Filed #2873 for the distinct history problem: enumerating multiple retained (path, epoch, ended) generations, fetching an explicitly selected historical generation, and rejecting subscriptions to ended content.

The split keeps ordinary announcements simple. They expose only the highest epoch per path, while #2873 defines a separate history/VOD discovery surface that can compose with this issue's cheap advertisements.

(Written by GPT-5)