Request an IDR frame when the FEC queue reports a loss without RFI

[anderizquierdo]

Summary

On a client where reference frame invalidation is not available, a frame loss reported by the video RTP FEC queue leaves the client waiting for an IDR frame that it never requests. Recovery is delayed until CONSECUTIVE_DROP_LIMIT is reached — 120 frames, which is 2 seconds of frozen video at 60 FPS and 4 seconds at 30 FPS.

notifyFrameLost() exists specifically to speed up recovery ("This lets us avoid having to wait until the next received frame to determine that we lost a frame"). In the non-RFI case it currently makes recovery an order of magnitude slower than the frame gap path it was meant to improve on.

What happens

On the setup below this is reproducible every few minutes, on a link that is not losing a single packet:

1. One packet arrives out of order. The last FEC parity shard of a frame is delivered a few microseconds before that frame's first data packet. Nothing is lost or duplicated — the whole frame arrives, just with its final shard first.
2. reconstructFrame() predicts the frame is unrecoverable. With only that one shard present, missingPackets (46) exceeds totalPackets - neededPackets (8 parity shards), so it reports a speculative loss for a frame that completes microseconds later. The code already anticipates being wrong here:

   // If we make it here and reported a lost frame, we lied to the host. This can happen if we
   // happen to get unlucky and this particular frame happens to be the one with OOS data, but
   // it should almost never happen.

3. notifyFrameLost() drops the frame and returns without asking for anything, because with RFI unavailable every other action in that function is behind if (!waitingForIdrFrame).
4. Nothing ever requests the IDR frame the client is now waiting for, because waitingForNextSuccessfulFrame — the flag that triggers the request — is only set by a different code path that did not run.
5. 120 frames later the CONSECUTIVE_DROP_LIMIT safety net finally requests one. That is the freeze.

This PR addresses step 4, which is where the 3 µs fault becomes seconds of frozen video.

Why recovery stalls

notifyFrameLost() calls dropFrameState(), which decides between RFI and IDR recovery:

if (strictIdrFrameWait || !idrFrameProcessed || waitingForIdrFrame) {
    waitingForIdrFrame = true;
}
else {
    waitingForRefInvalFrame = true;
}

with strictIdrFrameWait = !isReferenceFrameInvalidationEnabled();.

Back in notifyFrameLost(), everything else is inside if (!waitingForIdrFrame), so with RFI unavailable the function returns having done nothing but drop the frame:

void notifyFrameLost(unsigned int frameNumber, bool speculative) {
    dropFrameState();
    if (!waitingForIdrFrame) {        // false in non-RFI mode
        ...
        nextFrameNumber = frameNumber + 1;
        connectionDetectedFrameLoss(startFrameNumber, frameNumber);
    }
}                                     // returns without requesting anything

The IDR request would normally come from processRtpPayload():

if (waitingForIdrFrame) {
    Limelog("Waiting for IDR frame\n");
    // We wait for the first fully received frame after a loss to approximate
    // detection of the recovery of the network. ...
    if (waitingForNextSuccessfulFrame) {
        LiRequestIdrFrame();
    }
}

But waitingForNextSuccessfulFrame = true is set in exactly one place — the Network dropped N frames gap detection in processRtpPayload(). A loss reported through notifyFrameLost() never sets it, so the guard is always false and LiRequestIdrFrame() is never called. The client logs Waiting for IDR frame for every subsequent frame until dropFrameState() hits the drop limit and requests one as a last resort.

Note that the sibling loss paths in processRtpPayload() already handle this correctly:

dropFrameState();
if (waitingForIdrFrame) {
    LiRequestIdrFrame();
}
else {
    connectionDetectedFrameLoss(startFrameNumber, frameIndex);
}

The fix

Set waitingForNextSuccessfulFrame in the non-RFI branch, so the existing request path applies. This deliberately keeps the "wait for the first fully received frame" behaviour rather than requesting an IDR frame immediately, to preserve the congestion-collapse protection that comment describes.

Evidence

Client: Raspberry Pi 4 / DietPi, moonlight-qt 6.1.0, EGLFS, HEVC via the kernel V4L2 request decoder, wired gigabit. Host: Sunshine 2026.516.143833, NVENC. Stream 2560x1440.

Symptom: video freezes for 2-4 s while audio and the game continue, roughly every 5-12 minutes.

Every episode has an identical signature, and the frame count is always exactly 120:

19:47:03  Leaving speculative RFI mode after OOS video data at frame 128975
          Waiting for IDR frame                                      x 120
19:47:07  Reached consecutive drop limit
19:47:07  IDR frame request sent

Across a 45-minute session: 5 episodes, Waiting for IDR frame logged 600 times (= 5 x 120 exactly), Sending speculative RFI request 0 times (confirming RFI is off on this client), Network dropped 0 times (confirming the gap path never ran).

A packet capture on the client NIC (tcpdump -s 80, one RX queue, RPS off, GRO off, zero kernel drops) shows nothing was actually lost. Over ~100,000 video packets there were 0 missing and exactly one reordering event per episode:

+ 0.000 ms  seq 40169..40173   frame 128974   (data)
+32.183 ms  seq 40220          frame 128975   <- last parity shard, arrives first
+32.186 ms  seq 40174          frame 128975   <- SOF, 3 us later
+32.188 ms  seq 40175, 40176, ...             <- rest, in order

Impact

Any client without RFI, on any path that occasionally reorders a packet, gets a 120-frame freeze instead of a ~1-frame recovery. It also affects genuinely lost frames reported by the FEC queue, not just this false positive.

Testing

Validated end-to-end on the affected device. The patched moonlight-common-c was built into moonlight-qt v6.1.0 using the official rpi64/trixie packaging container, so the only difference from the released package is this patch.

Two sessions of comparable length, same host, same game, same stream settings:

	v6.1.0 unpatched	v6.1.0 + this patch
Stream time	45 min	47 min
Reordering events on the wire (packet capture)	1 per episode	1 per episode — identical pattern
Packets lost (NIC counters and capture)	0	0
Leaving speculative RFI mode after OOS video data	5	2
Waiting for IDR frame	600 (120 per episode)	4 (2 per episode)
Reached consecutive drop limit	5	0
Connection status update: 1 (connection poor)	5	0
"Frames dropped by your network connection"	0.37%	0.00%
Visible effect	2-4 s of frozen video per episode	none noticed by the user

The trigger still fires with the patch, so this is a like-for-like comparison rather than an absence of events. The capture shows the same thing both times: the last FEC parity shard of a frame arrives ~2 µs before that frame's first data packet (46 sequence positions early), with zero missing and zero duplicate packets.

What changes is the cost. The IDR frame is now requested in the same second the out-of-sequence data is seen, instead of 120 frames later, and Reached consecutive drop limit never appears. Incoming, decoding and rendering frame rates were identical (56.58 / 56.58 / 56.57 FPS), so nothing else regressed.

Build sanity: moonlight-common-c also configures and builds standalone with the change (CMake + GCC 15, Release); VideoDepacketizer.c compiles with no new warnings.