Resource hardening for small hosts colocated with TS6 - #19
Merged
Merged
Conversation
Live measurement on a 4 GB host running the bot next to its TS6 server: one viewer at 720p30 / 4608 kbps pushed the bot's RSS past 750 MB and was still climbing. The TS6 server, sharing the same RAM, started dropping packets - including viewer join requests - and eventually crashed under memory pressure, taking the whole stack with it. Operators reported "I can't reliably get into the stream and the server crashes". Three changes that together keep the colocated case stable: * Encoder defaults dropped to 480p24 / 2500 kbps (from 720p30 / 4608 kbps). Cuts libvpx's reference frame pool roughly in half; expected ~250 MB per viewer instead of ~750 MB. Operators with a beefier host bump SCREEN_WIDTH/HEIGHT + STREAM_BITRATE together in .env. * `STREAM_VIEWER_LIMIT` default 2 (from 4). With the lower defaults that's still 600-700 MB peak for the bot, leaving room for the TS6 server + OS on a 4 GB box. * New `ICE_DROP_NETWORKS` setting (default `172.16.0.0/12`). In host-network mode aiortc gathers a host candidate per docker bridge gateway plus matching srflx, blowing the offer SDP to 5 KB / 11 UDP fragments that some TS6 server builds choke on. We strip those before send_join_response - aiortc keeps them internally, we just don't burden the server with what no external viewer can route to anyway.
3 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
On a 4 GB host running the bot next to its TS6 server (loopback connect), the operator reports:
The live log showed why: one viewer at 720p30 / 4608 kbps pushed the bot's RSS past 750 MB and was still climbing. The TS6 server sharing the same 4 GB then started dropping forwarded
notifyjoinstreamrequestpackets and eventually crashed under memory pressure.Changes
Lower the encoder defaults (
SCREEN_WIDTH=854,SCREEN_HEIGHT=480,SCREEN_FPS=24,STREAM_BITRATE=2500). The reference-frame pool that dominates per-viewer RSS scales roughly withpixels × fps; cutting both halves the per-viewer cost to an expected ~250 MB. Operators who measured headroom can bump it back up in.env.STREAM_VIEWER_LIMITdefault 2 (was 4). With ~250 MB per viewer, two viewers + the TS6 server + the OS fits in 4 GB; four did not.New
ICE_DROP_NETWORKSsetting (default172.16.0.0/12). Innetwork_mode: host, aiortc gathers a host candidate per docker bridge gateway and a matching srflx for each, blowing the offer SDP to ~5 KB / 11 UDP fragments. The TS6 server forwards those fine for tiny offers but visibly struggles with the fragmented case (and our live trace hadrespondjoinstreamrequest fragments=11 total_bytes=4997). The filter strips bridge-gateway candidates beforesend_join_response; aiortc keeps them in its own ICE pool, we just don't burden the server with addresses no external viewer can route to anyway.Test plan
pytest— 234 passed, 1 skipped (7 new tests for_filter_sdp_candidatescovering: drop network match, non-candidate line preservation, CRLF preservation, empty list passthrough, invalid CIDR safe-skip, no-match passthrough, IPv6 networks).mypy --strictclean (37 files).ruff checkclean.STREAM_VIEWER_LIMIT=2andSCREEN_HEIGHT=480in.env, two viewers should stay under ~700 MB combined, the TS6 server should remain alive across multiple join cycles, andstream_publisher.ice_candidates_filteredshould appear in the log on each join showing the dropped 172.x candidates.Operator notes
After pulling and rebuilding, you have two choices for the encoder settings:
SCREEN_WIDTH,SCREEN_HEIGHT,SCREEN_FPS,STREAM_BITRATE,STREAM_VIEWER_LIMITlines from your.env. The bot will pick up the lower defaults..env— theSTREAM_VIEWER_LIMIT=-1etc. you set previously will still apply. Then this PR only fixes the SDP filter; for the memory issue you'd need to manually set the new lower values.Either way after pulling:
https://claude.ai/code/session_016DuCjRJK995Tj9aDhhB9at
Generated by Claude Code