feat: implement the fuse ranged read path and the plaintext chunk cache - #1118
Conversation
`crates/fuse` had no read operation at all. Land `OperationCore::read` over the engine's stream primitives, give each handle the stream that pins its content version, and add the bounded plaintext chunk cache blueprint/desktop.md places in the FS core. - `read(handle, offset, size)` serves the window block by block: a cached block answers from memory, a missed one costs exactly the sealed chunks it covers, and a short block ends the window the way `pread` does. - The stream opens lazily on first read, so a write-only handle and a file whose bytes are never read spend neither a resolve nor a slot against the engine's stream ceiling. `release` closes it and drops the plaintext it cached; `unmount` does the same for every open handle. - The cache is keyed by the engine stream handle, which pins one immutable version for its whole life, so a cached block can never be served as a slice of a different version. It is bounded in bytes, evicted least-recently-used, and every block is `Zeroizing`, so zeroization rides `Drop` rather than an unmount callback that a forced teardown would skip. - Opening a stream verifies a head version the kernel's page cache may predate, which is where `Invalidation::Data` is now constructed. The `fuse-op-core` suite gains the read path and a named CI gate, `FUSE Op Core`, as testing law 1 requires. Closes #1109 Part of #648 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WegkkQ3uhNREerTW4MMeY2
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (8)
WalkthroughThe FUSE operation core now supports ranged reads with per-handle streams and a bounded, zeroized plaintext chunk cache. Handle release and unmount clean streams and cached data. Tests and CI now cover this behavior. ChangesFUSE read and cache flow
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant FUSEClient
participant OperationCore
participant ChunkCache
participant StreamHandle
FUSEClient->>OperationCore: request ranged read
OperationCore->>ChunkCache: check stream/block cache
alt cache miss
OperationCore->>StreamHandle: fetch content block
OperationCore->>ChunkCache: cache plaintext block
end
ChunkCache-->>OperationCore: provide plaintext block
OperationCore-->>FUSEClient: return requested range
Possibly related issues
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review findings on the read path. - The buffer a read assembles its window in was a plain `Vec<u8>`: every `extend_from_slice` past capacity freed an allocation still holding decrypted bytes, and the `?` on a failed chunk fetch dropped a partly-filled one. It is now `Zeroizing`, grown through a wiping reserve the way the engine grows its own leaf-assembly buffer, and handed out by `mem::take` so the caller becomes the terminal owner. - `CacheBudget::for_profile` multiplied without a check, so a wrapped product could hand back a plaintext ceiling the mount cannot keep. It is now `checked_mul`, alongside the existing zero-block rejection. - Serve the gateway-URL CID parse from `testkit::requested_cid` rather than a fourth copy of it. - Say plainly that the cache's block size is chosen to match the chunk framing rather than enforced against it — only performance rests on the match, since the engine clamps every window to the pinned version. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WegkkQ3uhNREerTW4MMeY2
crates/fusehad no read operation at all —grep 'fn read'overcrates/fuse/srcreturned onlyreaddir. This lands the ranged read path, the per-handle stream lifetime it needs, and the bounded plaintext chunk cacheblueprint/desktop.mdplaces in the FS core.The design question this issue had to settle
The blueprint says the plaintext cache lives "in the FS core", while the engine already owns
StreamHandlepinning and leaf fetching. The cache sits above the facade, incrates/fuse. Two reasons:StreamHandlethat produced the block. A stream pins one head version for its whole life, so a block cached under its handle can never be served as a slice of a different version. Keying by node id instead would need head-change detection the FS core cannot do, and would reintroduce v1's stale-read class (a per-handle plaintext copy served for the handle's whole lifetime after the inode re-pointed).The cost of stream-scoped keying is that two concurrent handles on one file cache separately. That is the right trade: a media element or a
cpopens once and reads many windows, which is exactly where the cache pays, and version correctness is not something a cache should be able to break.What landed
OperationCore::read(handle, offset, size)serves the window block by block overEngine::read_stream. A cached block answers from memory; a missed one costs exactly the sealed chunks it covers, so the first byte never waits for the last. A block short of the framing width ends the window, the waypreaddoes. A window past the end is empty, not an error.OpenFilenow carriesOption<StreamHandle>. The stream opens lazily on first read, so a write-only handle and a file whose bytes are never read spend neither a resolve nor a slot againstMAX_OPEN_STREAMS.releasecloses the stream and drops the plaintext it cached;unmountdoes the same for every handle still open. Reading through a write-only handle isBadHandle— EBADF, as POSIX has it.crates/fuse/src/cache.rs). Bounded in bytes, evicted least-recently-used, block size matched to the content plane's chunk framing so a cache block maps onto exactly one sealed chunk. Every block isZeroizing<Vec<u8>>, so zeroization ridesDropon every path out — eviction, release, unmount, and a teardown that never callsunmountat all. That last one is deliberate: v1's only zeroization point was adestroy()callback, so a forced unmount left plaintext in RAM.Invalidation::Data { ino }is now constructed. Opening a stream verifies a head version the kernel's page cache for that inode may predate, so the mount says so when the projected size or mtime moved.Zeroization stays at the terminal owner: the cache owns the blocks the engine handed it, so it zeroes them; it never touches the buffer it returns to the caller, and it never reaches into anything the engine still owns.
The named CI gate
The
fuse-op-coresuite rode the workspace sweep with no named gate, whichAGENTS.mdtesting law 1 forbids. This is the first of four slices to touch that suite, so it carries the obligation: a newFUSE Op Corejob runscargo test -p cipherbox-fuse, andblueprint/testing.mdnames it in both the suite map and the PR-gate tier.Tests
crates/fuse/tests/fuse_op_core.rsgains apublishedmodule: a real account fixture — owner root, vault pointer, published record plane — with a file written through the engine's own write plane, so the reads under test are verified reads of real sealed chunks rather than a stub. It covers a ranged read spanning a chunk boundary, the first-byte-before-last-byte property counted in block fetches, cache hits costing no network, the LRU bound holding while eviction really happens,releasefreeing its stream across more rounds thanMAX_OPEN_STREAMS, unmount releasing every stream and every block, two handles reading independently, theDatainvalidation, an unavailable read answeringUnavailablewithout parking, and a write-only handle refusing to read.The never-block law is asserted the way the write plane asserts it: every routing endpoint is failed after the stream opens, so a window that re-resolved, republished, or walked a rotation could not serve at all — and every window still serves.
crates/fuse/src/cache.rsandhandle.rscarry unit suites for the eviction order, the ceiling, per-stream isolation, and the stream's attach/release lifetime.Review gates
Simplify, security, and crypto/privacy all ran on
git diff main...HEAD. Findings folded back in:Vec<u8>grown byextend_from_slicefrees the old allocation on every reallocation, and the?on a failed chunk fetch dropped a partly-filled one. Both reviews landed on this independently, and the engine already solves it one layer down for the same buffer shape. The accumulator is nowZeroizing, grown through a wiping reserve, handed out bymem::takeso the caller becomes the terminal owner. Covered by unit tests on the grow: a fitting grow must not relocate, and one past capacity must carry the bytes into the room it reserved.CacheBudget::for_profilemultiplied unchecked, so a wrapped product could hand back a plaintext ceiling the mount cannot keep — silent in release, exactly the release-active-check rule. Nowchecked_mul, with a test.testkit::requested_cid. Uses the exported helper.Findings verified and not applied, with the reason:
impl Drop for OperationCorecallingunmount()." It would restate what already holds: the core owns the engine, so dropping it drops the cache (every blockZeroizing) and drops the engine, whose ownDropclears the stream table. ADropimpl adds no guarantee and blocks moving fields out.if: always()Result aggregator." That pattern is for matrix jobs and jobs that live in another workflow —Desktop Build,Contract Suite,Web E2E Smoke.FUSE Op Coreis a single job with the same shape and the sameneeds/ifasEngine TestsandCore KATs (native + WASM), neither of which has one.Engineexposes no content-profile accessor and adding one is an engine-side change. The mismatch it would prevent is performance-only, and the host that mounts constructs both from the same profile. Left as is.release_streamthat a future engine-side stream invalidation must also forget the cache." That is prose defending a path that is not in the code, whichAGENTS.mdrules out. The coupling is visible fromrelease_streamitself.The one finding too large for this slice —
mod publishedre-implementscrates/engine/tests/write_plane.rs's account fixture, ~230 lines, becausecrates/engine/src/testkit/is outside this slice's file ownership and the four FUSE slices are sequential overops.rs— is #1121, with a dependency edge back to this issue.Not in scope
Writes, spill files, refusal errnos and freshness — the three sibling slices. No host adapter.
Closes #1109
Part of #648
Note
Implement ranged read path and bounded plaintext chunk cache in the FUSE operation core
ChunkCacheincrates/fuse/src/cache.rs: a bounded LRU in-memory cache keyed by(StreamHandle, block index)that stores plaintext inZeroizingbuffers and evicts least-recently-used blocks when the budget is exceeded.OperationCore::readincrates/fuse/src/ops.rs: frames FUSE read requests into cache-aligned blocks, serves hits fromChunkCache, fetches misses viaengine.read_stream, and assembles output in aZeroizingbuffer usinggrow_wipingto avoid leaving plaintext in freed memory.stream_for, which also detects inode content changes and triggers a host adapterinvalidateif the content has changed since the handle was opened.CacheBudget(withPRODUCTIONandCIpresets) is now part of the public API and must be provided toOperationCore::new.fuse-op-corerunningcargo test -p cipherbox-fuseand comprehensive integration tests incrates/fuse/tests/fuse_op_core.rscovering cache hits, LRU eviction, access checks, cross-chunk reads, and cleanup.OperationCore::newnow requires aCacheBudgetargument; existing callers must be updated.Macroscope summarized 7bba021.
Summary by CodeRabbit
New Features
Bug Fixes
Tests