web-services is the Rust workspace for Wavey's transport, proxy, and low-latency delivery services. It combines a reusable multi-protocol server foundation with cache-backed streaming crates and the upload-response request/response pipeline.
See the design and scalability review for current measurements, resolved defects, production gates, and implementation recommendations.
The web-service crate owns protocol plumbing only. Raw TCP helpers expose
generic [u32_be length][payload] frame reads/writes. Callers decide whether a
frame is mesh JSON, media access-unit bytes, or another application payload.
| Crate | Purpose |
|---|---|
web-service |
Core HTTP/1.1, HTTP/2, HTTP/3, WebSocket, WebTransport, QUIC relay, raw TCP/TLS server, and proxy primitives. |
upload-response |
Shared-memory request/response proxy that streams uploads into a cache, lets workers process them, and returns responses to clients. |
hls |
HLS-specific routing and cache-backed manifest/segment delivery. |
chunks |
Simple chunk/part delivery routes backed by the shared cache layer. |
| Path | Notes |
|---|---|
upload-response/tests |
Worker integration tests and protocol throughput benchmarks. |
web-service/tests |
Server and proxy benchmarks. |
examples/obs-rist-llhls |
OBS RIST ingest with the pure Rust RIST receiver and browser LL-HLS playback through hls.js. |
tls |
Local TLS material used by tests and local development. |
pem_to_env.sh |
Helper script for exporting PEM files into environment variables. |
The workspace still depends on several Wavey Git repositories outside crates.io, but Cargo fetches them directly over HTTPS. On a fresh machine, the first build may need network access for repositories such as:
# Build everything
cargo build --workspace
# Run the full workspace test suite
cargo test --workspace
# Run the web-service benchmark harness
cargo test -p av-web-service --release --test benchmark -- --benchmark
# Run upload-response tests and print benchmark output
cargo test -p av-upload-response --release -- --nocapture
# Run the OBS RIST -> LL-HLS browser playback example
cargo run -p obs-rist-llhls -- --rist-bind 0.0.0.0:7000 --http-port 9444Automated tests generate short-lived loopback certificates. Local development material remains under tls/local.wavey.ai.
The detailed upload-response crate documentation now lives here instead of in upload-response/README.md.
upload-response is a high-performance request and response proxy. It streams
requests into a shared-memory cache for external workers. It then returns their
responses to clients.
The shared-memory ChunkCache and slot-based streaming architecture are inspired by Low-Latency HLS partial segment delivery patterns.
| Protocol | Transport | Encryption | Auth | Notes |
|---|---|---|---|---|
| TCP | TCP | TLS | mTLS | Raw bytes, minimal overhead |
| HTTP/1.1 | TCP | TLS | Bearer | Content-Length or chunked |
| HTTP/2 | TCP | TLS | Bearer | Multiplexed streams |
| HTTP/3 | UDP | QUIC | Bearer | Low latency |
| WebSocket | TCP | TLS | Bearer | Binary frames |
| WebRTC | UDP | DTLS | Signaling | Data channels, P2P capable |
| SRT | UDP | AES-128 | Stream ID | Reliable UDP, media ingest |
| RIST | UDP | DTLS/PSK | URL params | Reliable UDP, broadcast ingest via librist C wrapper |
| RIST Pure | UDP | PSK/SRP support in progress | Socket addr | Pure Rust rist-core/rist-mio ingest |
| RTMP | TCP | None | Stream key | Plain TCP, media ingest |
| RTMPS | TCP | TLS | Stream key | TLS-wrapped RTMP |
| UDP+FEC | UDP | None | None | RaptorQ FEC, lowest-latency bounded-loss delivery |
Some protocols require optional crate features such as srt, rist, rist-pure, webrtc, or udp-fec. The default feature set only enables tcp.
WebTransport permits one session on each QUIC connection. Clients open separate connections for concurrent sessions.
H2H3ServerBuilder::with_max_in_flight_requests sets one handler limit across HTTP/1.1, HTTP/2, HTTP/3, WebSocket, and raw TCP. The default is 4,096.
Excess HTTP work receives 503 with Retry-After: 1. The server does not consume rejected request bodies, which preserves transport flow control.
The rist feature keeps the existing librist/C-wrapper backend. The rist-pure feature adds PureRistIngest, backed by the pure Rust rist-core and rist-mio crates from wavey-ai/rist-rs. Pure RIST byte-stream delivery suppresses duplicate arrivals and holds packets behind a sequence gap until retransmission restores wire order. The reorder queue is bounded and fails closed instead of concatenating bytes across an unresolved gap.
Each RIST source address has a separate ordered request. Queue overflow aborts all active requests because the dropped packet owner is not retained.
flowchart TB
subgraph Ingress
Client[Client Request<br/>H1.1/H2/H3/WSS/WebRTC/SRT/RIST/RTMP]
Router[UploadResponseRouter]
end
subgraph Cache["Shared Memory Cache"]
ReqCache[(Request ChunkCache<br/>Slot 1: HPKS Headers<br/>Slot 2..N: Body bytes<br/>Slot N+1: END marker)]
RespCache[(Response ChunkCache<br/>Slot 1: HPKS Headers<br/>Slot 2..N: Body bytes<br/>Slot N+1: END marker)]
end
subgraph Workers["Worker Pool"]
W1[Worker 1<br/>register_reader]
W2[Worker 2<br/>register_reader]
W3[Worker 3<br/>register_reader]
Writer[Response Writer<br/>try_claim_response]
end
subgraph Egress
Watcher[ResponseWatcher]
Response[Client Response]
end
Client --> Router
Router -->|"write slots"| ReqCache
ReqCache -->|"tail_request<br/>(multi-reader)"| W1
ReqCache -->|"tail_request<br/>(multi-reader)"| W2
ReqCache -->|"tail_request<br/>(multi-reader)"| W3
W1 -.->|"processing"| Writer
W2 -.->|"processing"| Writer
W3 -.->|"processing"| Writer
Writer -->|"write slots<br/>(exclusive)"| RespCache
RespCache --> Watcher
Watcher --> Response
- Multiple readers can register on the same stream via
register_reader(stream_id, worker_id). - Only one worker can claim response write access via
try_claim_response(stream_id, worker_id). - Reader presence checks are fast because
has_readers()is backed by atomic counters. - Active streams now have explicit slot ownership. Ingress allocates a real slot, workers discover active
stream_ids from the service, and slot reuse is safe after the previous stream closes.
For split CPU ingress and GPU worker deployments, UploadResponseControlRouter exposes a private HTTP/2 control plane under /_upload_response.
Run this router on a separate private listener. Configure with_bind_address and with_client_ca, enable HTTP/2, and disable HTTP/3.
The listener requires a worker certificate signed by the configured client CA. It rejects HTTP/1.1 and injects the verified certificate fingerprint into each request.
UploadResponseRouter never matches the internal prefix. Public requests to that prefix receive 404 Not Found.
| Method | Path | Purpose |
|---|---|---|
GET |
/_upload_response/streams |
List active streams and their slot/reader/claim state |
GET |
/_upload_response/streams/{stream_id} |
Read metadata for a single active stream |
GET |
/_upload_response/streams/{stream_id}/request/last |
Get the latest request slot id |
GET |
/_upload_response/streams/{stream_id}/request/slots/{slot_id} |
Read a raw request slot |
GET |
/_upload_response/streams/{stream_id}/response/last |
Get the latest response slot id |
GET |
/_upload_response/streams/{stream_id}/response/slots/{slot_id} |
Read a raw response slot |
PUT |
/_upload_response/streams/{stream_id}/readers/{worker_id} |
Register a reader |
DELETE |
/_upload_response/streams/{stream_id}/readers/{worker_id} |
Unregister a reader |
PUT |
/_upload_response/streams/{stream_id}/response/claim/{worker_id} |
Claim exclusive response write access |
DELETE |
/_upload_response/streams/{stream_id}/response/claim/{worker_id} |
Release a response claim |
PUT |
/_upload_response/streams/{stream_id}/response/headers |
Write a raw HPKS response headers frame |
PUT |
/_upload_response/streams/{stream_id}/response/body |
Append raw response body bytes |
PUT |
/_upload_response/streams/{stream_id}/response/end |
Finish the response stream |
This is the intended v1 control plane for Kubernetes pod splits. CPU ingress
and transcode pods own the client connection and cache. GPU workers read request
slots and write response slots through internal H2. A future high-throughput
data plane can use raw TCP/TLS with the same semantics and HPKS framing.
Create worker clients with RemoteIngressClient::new_with_mtls_pem_and_timeouts. Provide the control server CA, worker identity PEM, and the service timeout policy.
The claim response returns a random 256-bit capability in x-upload-response-capability. Only its SHA-256 digest remains in server memory.
External cache readers should use request_lane_handle, response_lane_handle, or stage_lane_handle. Each handle rejects reads after its physical slot is reused.
Response writes require that capability and a positive x-upload-response-sequence. Exact retries are idempotent; conflicting or skipped sequences fail.
Use a Kubernetes NetworkPolicy as a second boundary. Mutual TLS provides the required worker authentication.
Each request and response stream uses a simple slot-based format:
| Slot | Content |
|---|---|
| 1 | HPKS headers frame with method, path, and headers |
| 2..N-1 | Raw body bytes with no framing overhead |
| N | Empty slot end marker |
HPKSis thehttp-packstreaming format for headers.- Body slots are raw bytes and can be read zero-copy from the cache.
- An empty slot signals stream completion.
UDP+FEC, enabled by the udp-fec feature, uses the extracted raptorq-datagram-fec crate for RaptorQ forward error correction over plain UDP. Each datagram carries the current 16-byte sequenced wire header followed by a serialized RaptorQ EncodingPacket.
0 4 8 12 16
+---------------+---------------+---------------+---------------+
| block_id |transfer_length| packet_seq |src_syms|sym_sz |
+---------------+---------------+---------------+---------------+
| RaptorQ EncodingPacket bytes ... |
| Field | Size | Description |
|---|---|---|
block_id |
u32 LE |
Monotonically increasing block counter |
transfer_length |
u32 LE |
Total source bytes in this block |
packet_seq |
u32 LE |
Monotonic datagram sequence used for gap/reorder tracking |
src_syms |
u16 LE |
K, source symbols per block |
sym_sz |
u16 LE |
T, symbol size in bytes, default 1316 |
Defaults: K=4, T=1316, R=1 repair symbol. That recovers any single datagram loss per block with about 80 ms latency overhead at 48 kHz and 960-sample frames. The sequenced header also lets receivers report missing/reordered datagrams while RaptorQ repairs complete blocks. Loss beyond the repair budget is not recovered by FEC alone. Callers that need eventual delivery must add a repair/backfill path.
use upload_response::{UdpFecIngest, UdpFecSender};
// Sender
let mut sender = UdpFecSender::new(target_addr).await?;
sender.send(&audio_frame).await?;
// Receiver (ingest server)
let ingest = UdpFecIngest::new(service.clone());
let shutdown_tx = ingest.start(bind_addr).await?;RTMP streams serialize access units, parsed video or audio frames, to the cache:
[stream_type:1][key:1][id:8][dts:8][pts:8][data_len:4][data:N]
| Field | Size | Description |
|---|---|---|
stream_type |
1 byte | 0x1b for H.264 video, 0x0f for AAC audio |
key |
1 byte | 1 for keyframe, 0 for non-key |
id |
8 bytes | Sequential frame counter, big-endian |
dts |
8 bytes | Decode timestamp, big-endian |
pts |
8 bytes | Presentation timestamp, big-endian |
data_len |
4 bytes | Payload length, big-endian |
data |
N bytes | Video uses Annex-B NALUs, audio uses AAC plus ADTS |
Use rtmp-ingress with the upload-response feature:
use rtmp_ingress::upload::{deserialize_access_unit, RtmpUploadIngest};
let rtmp = RtmpUploadIngest::new(service.clone());
rtmp.start(addr).await?;
// Workers deserialize AccessUnits from body slots
let (au, bytes_consumed) = deserialize_access_unit(&data)?;SRT streams write raw bytes directly to the cache with no additional framing. Workers receive the exact bytes sent by the SRT client. Default live ingest enables timestamp-based packet delivery, too-late packet dropping, and a 120 ms peer recovery window so retransmissions are delivered in stream order. start_high_throughput remains the explicit bulk-transfer mode and does not provide those live pacing semantics.
use upload_response::{
UploadResponseConfig, UploadResponseService, UploadResponseTimeouts,
};
let config = UploadResponseConfig {
num_streams: 4096,
slot_size_kb: 32,
slots_per_stream: 16,
response_timeout_ms: 30000,
};
let capacity = config.validate()?;
let timeouts = UploadResponseTimeouts {
response_deadline_ms: 30_000,
response_idle_timeout_ms: 30_000,
reader_backpressure_timeout_ms: 5_000,
stream_admission_timeout_ms: 1_000,
remote_io_timeout_ms: 15_000,
};
let service = UploadResponseService::try_new_with_timeouts(config, timeouts)?;Validation includes the request lane, response lane, and all 16 possible stage lanes. It rejects more than 512 MiB of estimated metadata or 1 TiB of logical payload capacity.
UploadResponseService::new remains available as a compatibility wrapper. It panics when capacity validation fails.
The compatibility constructors map response_timeout_ms to the response, streaming-idle, backpressure, and admission waits. Remote worker requests retain their previous 60-second default.
| Slot Size | Throughput | Use Case |
|---|---|---|
| 16 KB | ~1400 MB/s | Many small requests |
| 32 KB | ~1374 MB/s | Default, good balance |
| 64 KB | ~1390 MB/s | Fewer slots for larger writes |
| 128-512 KB | ~1410-1430 MB/s | Large uploads |
| 1+ MB | ~1300 MB/s | Slight performance drop |
32 KB is the default because it limits per-stream payload capacity while retaining strong throughput.
Workers consume requests by tailing the request cache and writing a response once they reach the end marker:
use upload_response::{TailSlot, UploadResponseService};
async fn process_requests(service: Arc<UploadResponseService>, stream_id: u64) {
let mut slot_id = 0;
loop {
let current = service.request_last(stream_id).unwrap_or(0);
if current <= slot_id {
tokio::time::sleep(Duration::from_micros(100)).await;
continue;
}
slot_id += 1;
match service.tail_request(stream_id, slot_id).await {
Some(TailSlot::Headers(h)) => {
// h.method, h.path, h.headers
}
Some(TailSlot::Body(data)) => {
// Process body chunk (zero-copy Bytes)
}
Some(TailSlot::End) => {
write_response(service, stream_id, result).await;
break;
}
None => {}
}
}
}
async fn write_response(
service: Arc<UploadResponseService>,
stream_id: u64,
body: Bytes,
) {
let headers = StreamHeaders::Response(StreamResponseHeaders {
stream_id,
version: HttpVersion::Http11,
status: 200,
headers: vec![],
});
service.write_response_headers(stream_id, headers).await.unwrap();
service.append_response_body(stream_id, body).await.unwrap();
service.end_response(stream_id).await.unwrap();
}Public HTTP traffic streams end to end. A worker's response body slots reach the client as it writes them, so time-to-first-byte is the worker's first slot rather than its last, and response size is not capped. The response ring decouples the two: a worker may run a full ring ahead (1024 slots by default) before it waits on the client at all. Past that it paces to the slowest registered reader rather than recycling slots the reader still needs.
Message-oriented consumers still receive a complete CachedResponse: the
WebSocket handler, the WebRTC data channel, and the TCP/RIST ingests. Those are
capped at slot_bytes * slots_per_stream.
Streaming responses are bounded by idle time (response_idle_timeout_ms)
rather than total duration, so a response may take as long to generate as it
needs provided it keeps producing; only a stalled worker is cut off.
Handlers must call StreamWriter::finish; returning without it resets the
stream, so a failure mid-body can never reach a client as a short but
well-formed response.
See docs/response-streaming.md for the
hop-by-hop breakdown, which router hook to use and why, the backpressure
semantics, and the remaining work.
Benchmarks below were captured on Apple M1 in release mode.
=== Slot Size Throughput Benchmark ===
Upload size: 512 MB
Slot Size | Throughput | Slots Used
-------------+--------------+-------------
16 KB | 1397 MB/s | 32768
32 KB | 1374 MB/s | 16384
64 KB | 1390 MB/s | 8192
100 KB | 1424 MB/s | 5242
128 KB | 1412 MB/s | 4096
256 KB | 1411 MB/s | 2048
512 KB | 1430 MB/s | 1024
768 KB | 1418 MB/s | 682
1024 KB | 1322 MB/s | 512
2048 KB | 1307 MB/s | 256
Latency characteristics for streaming and real-time delivery, such as audio frames. This ranking is effectively the inverse of the bulk throughput table.
| Protocol | Latency Source | Worst-case one-way latency |
|---|---|---|
| UDP+FEC | FEC block fill time only, no retransmit | ~20-80 ms, tunable via K |
| Raw UDP | Single network hop | ~1 ms, no loss recovery |
| WebRTC | DTLS, SCTP, and NACK retransmit | ~50-150 ms |
| SRT | ARQ retransmit on loss adds at least one RTT | ~120-200 ms |
| RIST | Same retransmit model as SRT | ~100-200 ms |
| RIST Pure | Same retransmit model as RIST, no librist FFI boundary | ~100-200 ms |
| TCP/TLS | Head-of-line blocking and Nagle effects | Unpredictable |
| HTTP/1.1 | TCP HOL plus framing overhead | Worse than TCP |
| HTTP/2 | Multiplexing plus HOL at the TCP layer | Worse than TCP |
| HTTP/3 | QUIC avoids per-stream HOL, but adds crypto RTT | ~50-100 ms |
SRT and RIST trade latency for reliability through retransmission. A lost packet always costs at least one more RTT. Historical retransmission can recover loss that exceeds a small FEC budget. UDP+FEC pays a deterministic latency cost in advance. For loss within the repair budget, the maximum recovery latency is the block-fill time.
Reliable transport must restore wire order before exposing a byte stream. RIST retransmissions and bonded duplicates are therefore reordered and deduplicated before cache slot batching. SRT live ingest delegates the equivalent ordering to TSBPD. Cache slot size changes batching and backpressure only; it must never be used to hide continuity or decoder errors.
Avoid K=1, R=1 for live media unless you require 100% repair overhead on each
small packet. The reusable RaptorQ crates carry packet sequencing in the compact
FEC header. Media-aware adaptive repair gives more protection to keyframes and
audio. It does not add unnecessary repair to small delta or data packets:
let sender = UdpFecSender::new(target).await?;Benchmarked on Apple M1, --release, loopback, May 15, 2026:
| Protocol | 512 MB | 1 GB | Notes |
|---|---|---|---|
| HTTP/1.1 (chunked) | 743.1 MB/s | 707.1 MB/s | Streaming without Content-Length |
| WebSocket | 640.3 MB/s | 676.7 MB/s | Binary frames |
| HTTP/1.1 | 553.9 MB/s | 417.1 MB/s | Requires Content-Length |
| WebRTC | 527.2 MB/s | 418.6 MB/s | DTLS, SCTP data channels |
| RTMP | 524.9 MB/s | 253.2 MB/s | Plain TCP, access-unit serialization |
| HTTP/2 | 476.7 MB/s | 334.2 MB/s | Multiplexed streams |
| RIST Pure | 234.7 MB/s | 246.2 MB/s | Pure Rust main profile, reliable UDP |
| HTTP/3 | 154.1 MB/s | 157.1 MB/s | QUIC encryption overhead |
| SRT | 133.1 MB/s | 132.6 MB/s | AES-128, reliable UDP with ARQ |
| RIST | 87.1 MB/s | 66.1 MB/s | librist C wrapper, main profile |
| UDP+FEC | 44.0 MB/s | n/a | RaptorQ encode/decode bound, fixed-latency reliable delivery |
HTTP and WebSocket figures measure end-to-end request/response time. SRT, RIST, RIST Pure, RTMP, WebRTC, and UDP+FEC figures measure client send completion.
UDP+FEC is optimized for latency, not bulk transfer. The RaptorQ codec dominates at high data rates, but for audio-sized frames on a ~20 ms cadence the encode overhead is negligible.
========================================
UDP+FEC (RaptorQ) Benchmark (--release, loopback)
========================================
UDP+FEC 100 MB: 44.0 MB/s
UDP+FEC loss-recovery: 20.0 MB/s (20% packet loss, 2 repair symbols)
========================================
Tune K and R to balance latency versus redundancy:
| K (src syms) | R (repair) | Block latency at 48 kHz/960 | Recovers |
|---|---|---|---|
| 4 | 1 | ~80 ms | Any 1 loss per 5 packets |
| 2 | 1 | ~40 ms | Any 1 loss per 3 packets |
| 1 | 1 | ~20 ms | Any 1 loss per 2 packets |
Both SRT and WebRTC expose high-throughput modes for bulk transfer:
// SRT high-throughput mode (adds SendBuffer, zero PeerLatency)
let srt = SrtIngest::new(service);
srt.start_high_throughput(addr).await?;
// WebRTC high-throughput mode (larger SCTP messages, increased MTU)
let (socket, loop_fut) = WebRtcSocketBuilder::new(&url)
.add_channel(ChannelConfig::reliable())
.high_throughput()
.build();Those modes trade latency for better sustained transfer rates.
Single-stream sequential writes:
Upload size: 1024 MB
Slot size: 64 KB
Throughput: ~1390 MB/s
Concurrent multi-stream, 8 streams with 1 writer and 2 readers each:
Write: 1864 MB/s (29,827 ops/s)
Read: 3966 MB/s (63,461 ops/s)
Combined: 5830 MB/s
Massive concurrent reads, 1000 readers with 1 writer:
Read: 22.1M ops/s
Write: 22K ops/s (concurrent with reads)
The shared-memory ChunkCache scales to very high read fan-out because it uses:
- Pre-allocated ring buffers with no per-write allocation
- Per-slot
RwLockinstead of a global lock - Lock-free
last()checks via atomics - Zero-copy reads through
Bytes::slice() - Only 12 bytes of overhead per slot, 4-byte length plus 8-byte xxhash
# Run all upload-response tests
cargo test -p av-upload-response
# Run all protocol benchmarks and print output
cargo test -p av-upload-response --release --features "srt,rist,rist-pure,webrtc,tcp,udp-fec" --test proto_benchmark -- --nocapture --test-threads=1
# Specific worker/cache benchmarks
cargo test -p av-upload-response --release test_slot_size_benchmark -- --nocapture
cargo test -p av-upload-response --release test_gigabyte_upload_benchmark -- --nocapture
# Protocol comparison benchmark
cargo test -p av-upload-response --release --features "srt,rist,rist-pure,webrtc,tcp,udp-fec" --test proto_benchmark test_protocol_comparison -- --nocapture --test-threads=1
# UDP+FEC benchmark
cargo test -p av-upload-response --release --features "srt,rist,rist-pure,webrtc,tcp,udp-fec" --test proto_benchmark test_udp_fec_benchmark -- --nocapture
# Compile the pure Rust RIST backend
cargo check -p av-upload-response --features rist-pureweb-serviceprovides server traits such asRouterandStreamWriter.playlistsprovides the shared-memoryChunkCache.http-packprovides the HPKS framing used for headers.raptorq-datagram-fecis optional and only needed forudp-fec.
A four-vCPU Google Cloud server sustained a one-hour H3 test after ten warmup minutes. The load generator ran on a separate eight-vCPU host.
The long run used revision 782f466. Short H1, H2, and H3 regressions covered the request-limiting changes through 4564584.
| Requests | Errors | Rate | Payload rate | Wire rate | Mean p99 |
|---|---|---|---|---|---|
| 269,074,634 | 0 | 74,734/s | 3.444 Gbit/s | 3.644 Gbit/s | 31.98 ms |
RSS remained between 19.16 MiB and 20.61 MiB. The process retained five threads and 11 file descriptors throughout the measured hour.
This static test passed. The mixed upload, worker, stage, cancellation, and hostile RIST soak remains a production gate.
The playlist renderer now reuses one scratch buffer and precomputes invariant LL-HLS headers. Changing timestamps, durations, and identifiers no longer allocate temporary strings.
| Revision | Workers | Rate | Allocations/write | Reallocations/write |
|---|---|---|---|---|
c5bb8856 |
1 | 250,451/s | 46 | 10 |
e5410022 |
1 | 363,557/s | 1 | effectively 0 |
c5bb8856 |
8 | 354,214/s | 46 | 10 |
e5410022 |
8 | 647,743/s | 1 | effectively 0 |
These sequential one-second local samples are diagnostic controls. Use dedicated-host medians before making a release capacity claim.
The next cache experiment replicated only the latest encoded payload for one hot stream. Eight readers increased from 8.96 million to 38.62 million reads/s. CPU cost fell from 279.9 to 116.5 ns/read.
Eight replicas retained 7,304 encoded bytes instead of 911 bytes. The write diagnostic added nine allocations/write and 9% CPU cost. Replication is therefore opt-in; balanced and write-heavy workloads retain one payload.
Persistent HTTP/3 is still the production target for low-latency delivery. In
the current isolated two-host test, web-service delivered a 5 ms, 16-channel
PCM-shaped workload exactly through 48 simulated customers. It delivered
19,200 responses per second and 884.7 Mbit/s of response payload. The wire
rate was approximately 947.7 Mbit/s. Request-latency p99 was 14.1 ms on a
path with approximately 12.9 ms
ICMP RTT. At 56 customers the test stopped holding the requested cadence.
The capacity gap relative to the TCP controls requires an implementation investigation. It is not a reason to replace H3. The profile and packet capture found much UDP send and kernel work. Each 5,760-byte response used the minimum practical number of QUIC packets. However, UDP transmit segmentation was off on the virtio NIC. Thus, software segmented the batched output.
A dedicated Linode control had the same offload constraint and was not faster than GCP at the same exact workload. The existing H1/H2 figures were collected with a different topology and are controls only. They are not a valid final protocol comparison.
Quinn remains the default and continues to own WebTransport. Enabling the
h3-tokio-quiche Cargo feature adds Cloudflare's tokio-quiche as a selectable
plain-H3 server backend. The same router and a Quinn client can then measure the
two implementations without a copied server. Selecting
tokio-quiche together with WebTransport is rejected until feature parity is
implemented.
The isolation work has produced its first proven capacity fix. Ordinary media
GET responses carried two CORS fields that belong on preflight responses. Tests
removed this repeated header and QPACK work on the same two-vCPU GCP server. The
mean saturated 64-byte H3 response rate increased from 71,946 to 79,702
responses/s (+10.78%). The change reduced p99 from approximately 18.4 ms to
16.1 ms.
It also reduced wire traffic while it served more requests. An immediate restart of the old build reproduced the old ceiling. This is a small-response transport result.
On the full 5,760-byte control, the same change reduced
server CPU by about 2.4% at 40 customers while completing every scheduled
request.
The nearly 1 Gbit/s PCM result above remains the current qualified full-media boundary.
See HTTP/3 capacity investigation for the workload, results, profile, current interpretation, and bug-audit plan.
web-services is available under the MIT License.