Skip to content

Fix cache correctness defects in deduplication, namespaces, and lazy chunks - #54

Merged
iazaran merged 5 commits into
mainfrom
fix-cache-correctness
Sep 1, 2026
Merged

Fix cache correctness defects in deduplication, namespaces, and lazy chunks#54
iazaran merged 5 commits into
mainfrom
fix-cache-correctness

Conversation

@iazaran

@iazaran iazaran commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Fixes a set of correctness defects found while auditing the package. Every fix is backward compatible — no signature changes, no config changes, and data already sitting in users' caches is read safely.

vendor/bin/phpunit: 508 passing (486 before this branch, 2 pre-existing skips). The 19 new tests in tests/Unit/BugFixRegressionTest.php were checked against the pre-fix source: 15 of 19 fail there, so they genuinely pin the defects rather than the implementation.

Write deduplication (enabled by default)

put() skipped the store write on any content-hash match, so the expiry was never extended. A key refreshed on every pass with unchanged content expired at the first write's TTL:

SmartCache::put('heartbeat', 'alive', 3600);   // t+3600
// ... 30 minutes later, same value
SmartCache::put('heartbeat', 'alive', 3600);   // skipped — still expires at t+3600

The _sc_dna:{key} record now stores the absolute expiry alongside the xxh128 hash, and a write is skipped only when the stored entry provably outlives the newly requested window.

Two further problems in the same mechanism:

  • It could serve a stale value. forever(), add(), touch() and the SWR regeneration path changed a value or its expiry without maintaining the record, so a later put() compared against content no longer in the store. Writing Y, then forever()-ing X, then writing Y again left X in the cache while put() returned true.
  • It swallowed deletes. A put() with a TTL of 0, a negative TTL, or a past DateTimeInterface is a delete in Laravel. Any live entry trivially satisfies a past expiry, so the delete was skipped and the value survived.

Deduplication expiry maths now uses Carbon::now() instead of time(), matching how the store computes expiries and making the behavior drivable with Carbon::setTestNow().

Upgrade note: records written by earlier releases are a bare hash string. They are read safely, never treated as a skip signal, and upgraded in place on the next write. Old and new code interoperate during a rolling deploy — deduplication is simply inactive for keys whose record was last written by the other version. Expect the cache_write_dedup counter to fall sharply; it was previously inflated by skips that were silently shortening cache lifetimes.

Namespaces

Managed keys are stored fully qualified, but clear(), clearManaged(), cleanupExpiredManagedKeys(), executeStatusCommand() and all of CacheInvalidationService applied the active namespace to them a second time.

clear() deleted nothing under an active namespace. Worse, cleanupExpiredManagedKeys() classified every live namespaced key as expired and dropped it — emptying the index that pattern invalidation, audits and the dashboard depend on. That path is reachable from a plain healthCheck() call.

Lazy chunked collections

ChunkingStrategy chunks with array_chunk(..., preserve_keys: true), so chunk N is keyed by the original offsets. LazyChunkedCollection indexed each chunk by a chunk-relative position, so with strategies.chunking.lazy_loading enabled a 35-item dataset read back as 10 items and 25 nulls. Iteration, offsetGet(), slice(), each(), filter() and map() were all affected.

toArray() also used array_merge(), which renumbered integer keys and diverged from the eager restore path for sparse keys.

The existing tests missed this because their fixtures build 0-based chunks — a shape optimize() has never produced (preserve_keys dates to the initial commit). Those fixtures now mirror the real producer, and toArray() appends on key collision so hand-built collections keep working.

Other fixes

  • Model wildcard invalidation never matched anything. CacheInvalidation::matchesPattern() replaced the bare * after preg_quote() had escaped it, compiling user_* to /^user_\.*$/ — a literal dot. invalidatesPatterns(['user_*']) matched nothing and stale entries survived indefinitely. Now mirrors the correct implementation already present in CacheInvalidationService.
  • CompressionStrategy::restore() corrupted the process-wide error-handler stack. set_error_handler($previous) pushes a frame rather than popping one, so two handlers leaked per call and the application's own restore_error_handler() popped the wrong frame — leaving SmartCache's error-swallowing closure active and silently discarding application warnings.
  • memo() could return the wrong value. MemoizedCacheDriver::get() detected a miss by comparing the fetched value against the caller's $default, so reading a stored false/0/''/[] with a matching default recorded a hit as a miss. It also leaked memory in long-running workers: eviction only measured the value map, leaving accessOrder and the negative-lookup map unbounded.
  • Both destructors could fatal a served request. SmartCache and CostAwareCacheManager persisted to cache from __destruct() unguarded; a backend failure at shutdown turned a 200 into an uncatchable fatal. CostAwareCacheManager::persist() also never wrote an emptied map, so forget() was not durable.
  • A corrupted or foreign entry under _sc_performance_metrics / _sc_cost_metadata raised a TypeError instead of being ignored.

Behavior change

putWithJitter() and rememberWithJitter() now apply the percentage they are given. They previously routed through applyJitter(), which is gated on smart-cache.jitter.enabledoff by default — so both silently stored values with an unmodified TTL. Calling them is the opt-in. applyJitter() keeps its flag-gated behavior for direct callers and the fluent withJitter() modifier is unchanged.

Docs

README, docs/index.html and CHANGELOG updated for 1.13.3. The documented rememberWithJitter() example also passed the callback and percentage in the wrong order and would have raised a TypeError.

Write deduplication (enabled by default) skipped the store write on any
content-hash match. The expiry was therefore never extended, so a key
refreshed on every pass with unchanged content expired at the *first*
write's TTL. The `_sc_dna:{key}` record now carries the absolute expiry
next to the hash, and a write is skipped only when the stored entry
provably outlives the requested window. Legacy bare-hash records are read
safely and upgraded in place.

The same mechanism could also serve a stale value: forever(), add(),
touch() and the SWR regeneration path mutated a value or its expiry
without maintaining the record, so a later put() compared against content
that was no longer in the store. And because a live entry trivially
satisfies a past expiry, a put() with a non-positive TTL — a delete in
Laravel — was skipped too. All of these paths now invalidate the record.

Managed keys are stored fully qualified, but clear(), clearManaged(),
cleanupExpiredManagedKeys() and CacheInvalidationService applied the
active namespace to them a second time. clear() deleted nothing under a
namespace, and cleanupExpiredManagedKeys() classified every live key as
expired — emptying the index that pattern invalidation, audits and the
dashboard rely on, reachable from a plain healthCheck() call.

LazyChunkedCollection indexed each chunk by a chunk-relative position
while ChunkingStrategy chunks with preserve_keys, so every item past the
first chunk read back as null whenever lazy loading was enabled.

Also fixed: model wildcard invalidation never matched (the bare `*` was
replaced after preg_quote had escaped it); CompressionStrategy::restore()
leaked error handlers and left its own error-swallowing closure active in
the application; memo() reported a hit as a miss when the stored value
equalled the caller's default, and leaked two unbounded maps; both
destructors could turn a served request into a fatal at shutdown.

putWithJitter() and rememberWithJitter() now apply the percentage they
are given rather than silently no-opping when the global jitter flag is
off, which is the default.
Restricts the changes to ones that are invisible to a working install: the
dashboard stays opt-in, its middleware default is unchanged, and the set of
keys the console commands act on is identical.

The dashboard rendered the circuit-breaker state straight into the page and
into a CSS class. With circuit_breaker.shared enabled that value is read
from the application cache, so any process able to write that entry could
inject markup. The state is now escaped, the class is restricted to a safe
character set, and CircuitBreaker validates the hydrated state against the
three known values rather than trusting the payload.

Dashboard routes are registered from the service provider, so a route:cache
taken while the dashboard was enabled baked them into the compiled route
file and left all five endpoints serving after the setting was switched back
off. Each endpoint now re-checks dashboard.enabled at request time, and
registration honours routesAreCached() the way Laravel's own
loadRoutesFrom() does.

smart-cache:clear --force and smart-cache:status --force enumerated Redis
with KEYS *, which is O(N) and runs to completion on Redis' single-threaded
event loop. They now use SCAN, returning the same key set, with a fallback
to the previous call for any driver whose SCAN is unusable. The duplicated
enumeration helpers move to a shared trait.

Adds the first HTTP-level test coverage for the dashboard routes, which had
none, and documents that dashboard.middleware ships without authentication.
@iazaran

iazaran commented Aug 31, 2026

Copy link
Copy Markdown
Owner Author

Follow-up push: test counts refreshed across TESTING.md, README.md and docs/index.html (486 → 519 tests, 2,055 assertions), plus the security hardening that could be done without changing behavior for anyone already running the package.

Applied — all invisible to a working install:

  • Dashboard XSS: the circuit-breaker state was interpolated raw into the page body and a CSS class. Under circuit_breaker.shared that value comes from the application cache, so any process able to write that entry could inject markup. Now escaped, with the class restricted to a safe character set, and CircuitBreaker validates the hydrated state against its three known values instead of trusting the payload.
  • Dashboard stayed reachable after being disabled: routes are registered from the service provider, so a route:cache taken while enabled baked them into the compiled route file and all five endpoints kept serving once the setting was switched off. Each endpoint now re-checks dashboard.enabled per request, and registration honours routesAreCached().
  • KEYS *SCAN in smart-cache:clear --force and smart-cache:status --force. KEYS is O(N) and runs to completion on Redis' single-threaded event loop. Same key set, with a fallback to the old call for any driver whose SCAN is unusable — covered by tests against both phpredis- and predis-shaped connections.
  • First HTTP-level tests for the dashboard routes, which previously had none.

Deliberately not changed — each would alter behavior for someone already running this:

  • dashboard.middleware default stays ['web']. Switching it to ['web','auth'] would lock out anyone currently using the dashboard. Documented the exposure instead, in the README, the docs, and the published config.
  • GET /smart-cache/health still performs cleanup. Making it read-only is the right fix for a CSRF-able destructive GET, but it removes a side effect someone may rely on.
  • isSmartCacheInternalKey() still uses str_contains('_sc_'), so app keys like report_sc_2024 remain excluded from --force. Switching to str_starts_with would make the command delete more keys.
  • encrypt_all still does not encrypt values that trip chunking or compression, because the first matching strategy wins. Fixing the strategy composition changes what gets written for those users.
  • The Redis double-prefix bug in --force is untouched: on a prefixed Redis the sweep is currently a silent no-op, and fixing it would start deleting keys that are not being deleted today.

Happy to take any of those in a separate PR if you want them — they each need a deliberate call rather than a silent fix.

…rate

routesAreCached() is not declared on the Application contract - it comes
from CachesRoutes, which is why Laravel's own loadRoutesFrom() guards with
an instanceof check. Calling it unconditionally would raise "Call to
undefined method" on any container that does not implement CachesRoutes.
Guarded the same way Laravel does.

Redis cluster SCAN walks a single node, because PhpRedisClusterConnection
passes a `node` through to RedisCluster::scan. For a sweep that decides
what to delete, a partial view of the keyspace is worse than the previous
behaviour, so cluster connections keep using KEYS.

The dashboard's Hit Rate card always rendered N/A: it read a top-level
hit_rate key, but getPerformanceMetrics() reports the value as
cache_efficiency.hit_ratio. The old key is still honoured.

Test counts refreshed to 522 / 2,060.
@iazaran
iazaran merged commit db56375 into main Sep 1, 2026
24 checks passed
@iazaran
iazaran deleted the fix-cache-correctness branch September 1, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant