Fix cache correctness defects in deduplication, namespaces, and lazy chunks - #54
Merged
Conversation
Write deduplication (enabled by default) skipped the store write on any
content-hash match. The expiry was therefore never extended, so a key
refreshed on every pass with unchanged content expired at the *first*
write's TTL. The `_sc_dna:{key}` record now carries the absolute expiry
next to the hash, and a write is skipped only when the stored entry
provably outlives the requested window. Legacy bare-hash records are read
safely and upgraded in place.
The same mechanism could also serve a stale value: forever(), add(),
touch() and the SWR regeneration path mutated a value or its expiry
without maintaining the record, so a later put() compared against content
that was no longer in the store. And because a live entry trivially
satisfies a past expiry, a put() with a non-positive TTL — a delete in
Laravel — was skipped too. All of these paths now invalidate the record.
Managed keys are stored fully qualified, but clear(), clearManaged(),
cleanupExpiredManagedKeys() and CacheInvalidationService applied the
active namespace to them a second time. clear() deleted nothing under a
namespace, and cleanupExpiredManagedKeys() classified every live key as
expired — emptying the index that pattern invalidation, audits and the
dashboard rely on, reachable from a plain healthCheck() call.
LazyChunkedCollection indexed each chunk by a chunk-relative position
while ChunkingStrategy chunks with preserve_keys, so every item past the
first chunk read back as null whenever lazy loading was enabled.
Also fixed: model wildcard invalidation never matched (the bare `*` was
replaced after preg_quote had escaped it); CompressionStrategy::restore()
leaked error handlers and left its own error-swallowing closure active in
the application; memo() reported a hit as a miss when the stored value
equalled the caller's default, and leaked two unbounded maps; both
destructors could turn a served request into a fatal at shutdown.
putWithJitter() and rememberWithJitter() now apply the percentage they
are given rather than silently no-opping when the global jitter flag is
off, which is the default.
Restricts the changes to ones that are invisible to a working install: the dashboard stays opt-in, its middleware default is unchanged, and the set of keys the console commands act on is identical. The dashboard rendered the circuit-breaker state straight into the page and into a CSS class. With circuit_breaker.shared enabled that value is read from the application cache, so any process able to write that entry could inject markup. The state is now escaped, the class is restricted to a safe character set, and CircuitBreaker validates the hydrated state against the three known values rather than trusting the payload. Dashboard routes are registered from the service provider, so a route:cache taken while the dashboard was enabled baked them into the compiled route file and left all five endpoints serving after the setting was switched back off. Each endpoint now re-checks dashboard.enabled at request time, and registration honours routesAreCached() the way Laravel's own loadRoutesFrom() does. smart-cache:clear --force and smart-cache:status --force enumerated Redis with KEYS *, which is O(N) and runs to completion on Redis' single-threaded event loop. They now use SCAN, returning the same key set, with a fallback to the previous call for any driver whose SCAN is unusable. The duplicated enumeration helpers move to a shared trait. Adds the first HTTP-level test coverage for the dashboard routes, which had none, and documents that dashboard.middleware ships without authentication.
Owner
Author
|
Follow-up push: test counts refreshed across Applied — all invisible to a working install:
Deliberately not changed — each would alter behavior for someone already running this:
Happy to take any of those in a separate PR if you want them — they each need a deliberate call rather than a silent fix. |
…rate routesAreCached() is not declared on the Application contract - it comes from CachesRoutes, which is why Laravel's own loadRoutesFrom() guards with an instanceof check. Calling it unconditionally would raise "Call to undefined method" on any container that does not implement CachesRoutes. Guarded the same way Laravel does. Redis cluster SCAN walks a single node, because PhpRedisClusterConnection passes a `node` through to RedisCluster::scan. For a sweep that decides what to delete, a partial view of the keyspace is worse than the previous behaviour, so cluster connections keep using KEYS. The dashboard's Hit Rate card always rendered N/A: it read a top-level hit_rate key, but getPerformanceMetrics() reports the value as cache_efficiency.hit_ratio. The old key is still honoured. Test counts refreshed to 522 / 2,060.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a set of correctness defects found while auditing the package. Every fix is backward compatible — no signature changes, no config changes, and data already sitting in users' caches is read safely.
vendor/bin/phpunit: 508 passing (486 before this branch, 2 pre-existing skips). The 19 new tests intests/Unit/BugFixRegressionTest.phpwere checked against the pre-fix source: 15 of 19 fail there, so they genuinely pin the defects rather than the implementation.Write deduplication (enabled by default)
put()skipped the store write on any content-hash match, so the expiry was never extended. A key refreshed on every pass with unchanged content expired at the first write's TTL:The
_sc_dna:{key}record now stores the absolute expiry alongside thexxh128hash, and a write is skipped only when the stored entry provably outlives the newly requested window.Two further problems in the same mechanism:
forever(),add(),touch()and the SWR regeneration path changed a value or its expiry without maintaining the record, so a laterput()compared against content no longer in the store. WritingY, thenforever()-ingX, then writingYagain leftXin the cache whileput()returnedtrue.put()with a TTL of0, a negative TTL, or a pastDateTimeInterfaceis a delete in Laravel. Any live entry trivially satisfies a past expiry, so the delete was skipped and the value survived.Deduplication expiry maths now uses
Carbon::now()instead oftime(), matching how the store computes expiries and making the behavior drivable withCarbon::setTestNow().Upgrade note: records written by earlier releases are a bare hash string. They are read safely, never treated as a skip signal, and upgraded in place on the next write. Old and new code interoperate during a rolling deploy — deduplication is simply inactive for keys whose record was last written by the other version. Expect the
cache_write_dedupcounter to fall sharply; it was previously inflated by skips that were silently shortening cache lifetimes.Namespaces
Managed keys are stored fully qualified, but
clear(),clearManaged(),cleanupExpiredManagedKeys(),executeStatusCommand()and all ofCacheInvalidationServiceapplied the active namespace to them a second time.clear()deleted nothing under an active namespace. Worse,cleanupExpiredManagedKeys()classified every live namespaced key as expired and dropped it — emptying the index that pattern invalidation, audits and the dashboard depend on. That path is reachable from a plainhealthCheck()call.Lazy chunked collections
ChunkingStrategychunks witharray_chunk(..., preserve_keys: true), so chunk N is keyed by the original offsets.LazyChunkedCollectionindexed each chunk by a chunk-relative position, so withstrategies.chunking.lazy_loadingenabled a 35-item dataset read back as 10 items and 25nulls. Iteration,offsetGet(),slice(),each(),filter()andmap()were all affected.toArray()also usedarray_merge(), which renumbered integer keys and diverged from the eager restore path for sparse keys.The existing tests missed this because their fixtures build 0-based chunks — a shape
optimize()has never produced (preserve_keysdates to the initial commit). Those fixtures now mirror the real producer, andtoArray()appends on key collision so hand-built collections keep working.Other fixes
CacheInvalidation::matchesPattern()replaced the bare*afterpreg_quote()had escaped it, compilinguser_*to/^user_\.*$/— a literal dot.invalidatesPatterns(['user_*'])matched nothing and stale entries survived indefinitely. Now mirrors the correct implementation already present inCacheInvalidationService.CompressionStrategy::restore()corrupted the process-wide error-handler stack.set_error_handler($previous)pushes a frame rather than popping one, so two handlers leaked per call and the application's ownrestore_error_handler()popped the wrong frame — leaving SmartCache's error-swallowing closure active and silently discarding application warnings.memo()could return the wrong value.MemoizedCacheDriver::get()detected a miss by comparing the fetched value against the caller's$default, so reading a storedfalse/0/''/[]with a matching default recorded a hit as a miss. It also leaked memory in long-running workers: eviction only measured the value map, leavingaccessOrderand the negative-lookup map unbounded.SmartCacheandCostAwareCacheManagerpersisted to cache from__destruct()unguarded; a backend failure at shutdown turned a 200 into an uncatchable fatal.CostAwareCacheManager::persist()also never wrote an emptied map, soforget()was not durable._sc_performance_metrics/_sc_cost_metadataraised aTypeErrorinstead of being ignored.Behavior change
putWithJitter()andrememberWithJitter()now apply the percentage they are given. They previously routed throughapplyJitter(), which is gated onsmart-cache.jitter.enabled— off by default — so both silently stored values with an unmodified TTL. Calling them is the opt-in.applyJitter()keeps its flag-gated behavior for direct callers and the fluentwithJitter()modifier is unchanged.Docs
README,
docs/index.htmland CHANGELOG updated for 1.13.3. The documentedrememberWithJitter()example also passed the callback and percentage in the wrong order and would have raised aTypeError.