feat(cache): add opt-in SSR fragment caching - #707
Draft
m2broth wants to merge 4 commits into
Draft
Conversation
added 3 commits
August 18, 2026 17:08
Adds opt-in, per-app caching of SSR fragment responses to cut redundant
renders under load, via a registry-configured ssr.cache.{enabled,ttlSeconds}.
- Stale-while-revalidate: one background refresh per key; concurrent
requests get the stale copy instantly instead of queuing behind it.
- Single-flight capture with a byte/count/concurrency-bounded, evicting
in-memory store; capacity is injectable and asserted at composition time.
- Strict user isolation: shared renders forward only an anonymized header
set (host/locale) — never cookies, auth, UA, or the query string.
- A response is cached only on a plain 200 with no Set-Cookie and no
Cache-Control directive (bare or field-qualified) forbidding reuse.
- A named refusal taxonomy surfaced through New Relic metrics, an HTML
debug marker, and logs, so "why wasn't this cached" is diagnosable
without reading code.
- A render deadline bounds how long a stale/cold render can buffer before
falling back to a private render, without capping legitimately slow ones.
Adds opt-in, per-app caching of SSR fragment responses to cut redundant
renders under load, via a registry-configured ssr.cache.{enabled,ttlSeconds}.
- Stale-while-revalidate: one background refresh per key; concurrent
requests get the stale copy instantly instead of queuing behind it.
- Single-flight capture with a byte/count/concurrency-bounded, evicting
in-memory store; capacity is injectable and asserted at composition time.
- Strict user isolation: shared renders forward only an anonymized header
set (host/locale) — never cookies, auth, UA, or the query string.
- A response is cached only on a plain 200 with no Set-Cookie and no
Cache-Control directive (bare or field-qualified) forbidding reuse.
- A named refusal taxonomy surfaced through New Relic metrics, an HTML
debug marker, and logs, so "why wasn't this cached" is diagnosable
without reading code.
- A render deadline bounds a cold miss before falling back to a private
render, and separately bounds how long a background refresh may run
before it's abandoned - a stale request itself is never delayed by it.
Known limitations:
- No automated way to exclude price-bearing routes from caching -
enforcement relies on per-app opt-in discipline and the fragment
responding with Cache-Control: no-store (see "Fragments rendering
prices" in docs/ssr_fragment_caching.md). An app serving both a static
and a price-bearing route under the same ssr.cache config has no
per-route exclusion today.
- Per-instance, in-memory cache: no cross-instance sharing, cold after
every deploy/restart.
Coverage ReportIlc/serverCommit SHA:a3eb0f068cde5f47e6293fcd545b980fd2f0aa98 Test coverage results 🧪File details
Ilc/clientCommit SHA:a3eb0f068cde5f47e6293fcd545b980fd2f0aa98 Test coverage results 🧪File details
RegistryCommit SHA:a3eb0f068cde5f47e6293fcd545b980fd2f0aa98 Test coverage results 🧪File details
|
m2broth
marked this pull request as ready for review
August 18, 2026 15:26
m2broth
requested review from
b1ff,
blackrabbit99,
stas-nc,
wRLSS and
yehor-manzhula
August 18, 2026 15:26
stas-nc
reviewed
Aug 19, 2026
stas-nc
left a comment
stas-nc
left a comment
Member
There was a problem hiding this comment.
Two functional findings on the cache module (details in the line comments).
stas-nc
reviewed
Aug 19, 2026
stas-nc
reviewed
Aug 19, 2026
stas-nc
reviewed
Aug 19, 2026
Two correctness defects on the render-deadline path, plus the log volume this feature would have added at production traffic. Capture slot leaked when load() never settled. The pending entry was removed only when the capture promise settled, while the deadline was raced from outside it, so a fragment that never finishes sending response headers held its registry entry - and the concurrent-capture slot it occupies - for the lifetime of the process. The deadline now lives inside the pending call, so the existing settle-cleanup reclaims the slot; a response arriving after the deadline is destroyed rather than buffered, since it no longer holds a slot. Joiners now share one deadline instead of each arming its own. Cold-miss deadline failed the render instead of falling back. The rejection propagated out of cache.get() and was rethrown, failing the user's request as a bare TimeoutError. It is now converted to a named 'render-deadline' refusal (metric, HTML marker, log line, docs table), which the coordinator already answers with a private render. Genuine transport failures still propagate unchanged. Log volume on the hot path. hit/stale/miss moved from info to debug; refuse and error stay at info. `fragmentProxyHeaders` being dropped on cacheable renders is a configuration fact that never changes for a given app, so it now warns once per appId per process. Evictions - a permanently full cache warns on every insert, reachable with the docs' own sizing example - report at most one warn per minute, carrying the count evicted since the previous line, with the first still immediate. The eviction throttle lives in the fragment cache's storage factory, leaving EvictingCacheStorage a generic bounded map for the registry. Adds regression tests for slot reclamation, dropping a late response and the private-render fallback, and corrects the observability doc, which claimed all `[ILC Cache]` entries are emitted at info. Known limitation: a fragment refused for a static config reason (forward-querystring, wrapper-conf, ttl-invalid) still logs one info line per request, because those reasons are re-evaluated per render rather than tombstoned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
m2broth
marked this pull request as draft
August 20, 2026 05:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds opt-in, per-app caching of SSR fragment responses to cut redundant renders under load, via a registry-configured ssr.cache.{enabled,ttlSeconds}.
Known limitation: there's no automated way to exclude price-bearing routes from caching — enforcement relies on per-app opt-in discipline and the fragment responding with Cache-Control: no-store (see "Fragments rendering prices" in docs/ssr_fragment_caching.md). An app serving both a static and a price-bearing route under the same ssr.cache config has no per-route exclusion today.