Skip to content

feat: implement the fuse ranged read path and the plaintext chunk cache - #1118

Merged
FSM1 merged 2 commits into
mainfrom
feat/fuse-ranged-read-and-plaintext-chunk-cache
Aug 6, 2026
Merged

feat: implement the fuse ranged read path and the plaintext chunk cache#1118
FSM1 merged 2 commits into
mainfrom
feat/fuse-ranged-read-and-plaintext-chunk-cache

Conversation

@FSM1

@FSM1 FSM1 commented Aug 6, 2026

Copy link
Copy Markdown
Owner

crates/fuse had no read operation at all — grep 'fn read' over crates/fuse/src returned only readdir. This lands the ranged read path, the per-handle stream lifetime it needs, and the bounded plaintext chunk cache blueprint/desktop.md places in the FS core.

The design question this issue had to settle

The blueprint says the plaintext cache lives "in the FS core", while the engine already owns StreamHandle pinning and leaf fetching. The cache sits above the facade, in crates/fuse. Two reasons:

  1. A cache below the facade would put plaintext in the engine, which the blueprint's "the one place plaintext is cached" sentence forbids. The engine returns plaintext; it retains none.
  2. Keying is only sound above the facade. The cache is keyed by the engine StreamHandle that produced the block. A stream pins one head version for its whole life, so a block cached under its handle can never be served as a slice of a different version. Keying by node id instead would need head-change detection the FS core cannot do, and would reintroduce v1's stale-read class (a per-handle plaintext copy served for the handle's whole lifetime after the inode re-pointed).

The cost of stream-scoped keying is that two concurrent handles on one file cache separately. That is the right trade: a media element or a cp opens once and reads many windows, which is exactly where the cache pays, and version correctness is not something a cache should be able to break.

What landed

  • OperationCore::read(handle, offset, size) serves the window block by block over Engine::read_stream. A cached block answers from memory; a missed one costs exactly the sealed chunks it covers, so the first byte never waits for the last. A block short of the framing width ends the window, the way pread does. A window past the end is empty, not an error.
  • Per-handle stream lifetime. OpenFile now carries Option<StreamHandle>. The stream opens lazily on first read, so a write-only handle and a file whose bytes are never read spend neither a resolve nor a slot against MAX_OPEN_STREAMS. release closes the stream and drops the plaintext it cached; unmount does the same for every handle still open. Reading through a write-only handle is BadHandle — EBADF, as POSIX has it.
  • The bounded plaintext chunk cache (crates/fuse/src/cache.rs). Bounded in bytes, evicted least-recently-used, block size matched to the content plane's chunk framing so a cache block maps onto exactly one sealed chunk. Every block is Zeroizing<Vec<u8>>, so zeroization rides Drop on every path out — eviction, release, unmount, and a teardown that never calls unmount at all. That last one is deliberate: v1's only zeroization point was a destroy() callback, so a forced unmount left plaintext in RAM.
  • Invalidation::Data { ino } is now constructed. Opening a stream verifies a head version the kernel's page cache for that inode may predate, so the mount says so when the projected size or mtime moved.

Zeroization stays at the terminal owner: the cache owns the blocks the engine handed it, so it zeroes them; it never touches the buffer it returns to the caller, and it never reaches into anything the engine still owns.

The named CI gate

The fuse-op-core suite rode the workspace sweep with no named gate, which AGENTS.md testing law 1 forbids. This is the first of four slices to touch that suite, so it carries the obligation: a new FUSE Op Core job runs cargo test -p cipherbox-fuse, and blueprint/testing.md names it in both the suite map and the PR-gate tier.

Tests

crates/fuse/tests/fuse_op_core.rs gains a published module: a real account fixture — owner root, vault pointer, published record plane — with a file written through the engine's own write plane, so the reads under test are verified reads of real sealed chunks rather than a stub. It covers a ranged read spanning a chunk boundary, the first-byte-before-last-byte property counted in block fetches, cache hits costing no network, the LRU bound holding while eviction really happens, release freeing its stream across more rounds than MAX_OPEN_STREAMS, unmount releasing every stream and every block, two handles reading independently, the Data invalidation, an unavailable read answering Unavailable without parking, and a write-only handle refusing to read.

The never-block law is asserted the way the write plane asserts it: every routing endpoint is failed after the stream opens, so a window that re-resolved, republished, or walked a rotation could not serve at all — and every window still serves.

crates/fuse/src/cache.rs and handle.rs carry unit suites for the eviction order, the ceiling, per-stream isolation, and the stream's attach/release lifetime.

Review gates

Simplify, security, and crypto/privacy all ran on git diff main...HEAD. Findings folded back in:

  • The window a read assembles into was itself un-wiped plaintext. A plain Vec<u8> grown by extend_from_slice frees the old allocation on every reallocation, and the ? on a failed chunk fetch dropped a partly-filled one. Both reviews landed on this independently, and the engine already solves it one layer down for the same buffer shape. The accumulator is now Zeroizing, grown through a wiping reserve, handed out by mem::take so the caller becomes the terminal owner. Covered by unit tests on the grow: a fitting grow must not relocate, and one past capacity must carry the bytes into the room it reserved.
  • CacheBudget::for_profile multiplied unchecked, so a wrapped product could hand back a plaintext ceiling the mount cannot keep — silent in release, exactly the release-active-check rule. Now checked_mul, with a test.
  • The gateway-URL CID parse was a fourth copy of testkit::requested_cid. Uses the exported helper.
  • The doc overstated the block-size match. The budget's block size is chosen to match the chunk framing, not enforced against it. Only performance rests on the match — the engine clamps every window to the pinned version, so a mismatch cannot make the EOF logic wrong. Reworded to say what is true.

Findings verified and not applied, with the reason:

  • "Add impl Drop for OperationCore calling unmount()." It would restate what already holds: the core owns the engine, so dropping it drops the cache (every block Zeroizing) and drops the engine, whose own Drop clears the stream table. A Drop impl adds no guarantee and blocks moving fields out.
  • "The new job needs an if: always() Result aggregator." That pattern is for matrix jobs and jobs that live in another workflow — Desktop Build, Contract Suite, Web E2E Smoke. FUSE Op Core is a single job with the same shape and the same needs/if as Engine Tests and Core KATs (native + WASM), neither of which has one.
  • "Derive the cache budget from the engine's content profile instead of taking it." Right idea, but Engine exposes no content-profile accessor and adding one is an engine-side change. The mismatch it would prevent is performance-only, and the host that mounts constructs both from the same profile. Left as is.
  • "Note at release_stream that a future engine-side stream invalidation must also forget the cache." That is prose defending a path that is not in the code, which AGENTS.md rules out. The coupling is visible from release_stream itself.

The one finding too large for this slice — mod published re-implements crates/engine/tests/write_plane.rs's account fixture, ~230 lines, because crates/engine/src/testkit/ is outside this slice's file ownership and the four FUSE slices are sequential over ops.rs — is #1121, with a dependency edge back to this issue.

Not in scope

Writes, spill files, refusal errnos and freshness — the three sibling slices. No host adapter.

Closes #1109
Part of #648

Note

Implement ranged read path and bounded plaintext chunk cache in the FUSE operation core

  • Adds a ChunkCache in crates/fuse/src/cache.rs: a bounded LRU in-memory cache keyed by (StreamHandle, block index) that stores plaintext in Zeroizing buffers and evicts least-recently-used blocks when the budget is exceeded.
  • Implements OperationCore::read in crates/fuse/src/ops.rs: frames FUSE read requests into cache-aligned blocks, serves hits from ChunkCache, fetches misses via engine.read_stream, and assembles output in a Zeroizing buffer using grow_wiping to avoid leaving plaintext in freed memory.
  • Read streams are opened lazily on first read via stream_for, which also detects inode content changes and triggers a host adapter invalidate if the content has changed since the handle was opened.
  • Releasing a handle or unmounting now closes the associated engine stream and evicts all cached plaintext blocks.
  • CacheBudget (with PRODUCTION and CI presets) is now part of the public API and must be provided to OperationCore::new.
  • Adds a CI job fuse-op-core running cargo test -p cipherbox-fuse and comprehensive integration tests in crates/fuse/tests/fuse_op_core.rs covering cache hits, LRU eviction, access checks, cross-chunk reads, and cleanup.
  • Behavioral Change: OperationCore::new now requires a CacheBudget argument; existing callers must be updated.

Macroscope summarized 7bba021.

Summary by CodeRabbit

  • New Features

    • Added ranged file reads across content chunks.
    • Added bounded in-memory caching to improve repeated reads while limiting memory usage.
    • Added lazy stream handling and cleanup when files are released or unmounted.
  • Bug Fixes

    • Improved handling of unavailable, invalid, and write-only file handles.
    • Ensured stale content is invalidated when file data changes.
    • Added secure cleanup of plaintext data removed from memory.
  • Tests

    • Expanded coverage for caching, eviction, ranged reads, stream cleanup, and independent file handles.
    • Added a dedicated CI gate for FUSE operation-core validation.

`crates/fuse` had no read operation at all. Land `OperationCore::read` over
the engine's stream primitives, give each handle the stream that pins its
content version, and add the bounded plaintext chunk cache blueprint/desktop.md
places in the FS core.

- `read(handle, offset, size)` serves the window block by block: a cached block
  answers from memory, a missed one costs exactly the sealed chunks it covers,
  and a short block ends the window the way `pread` does.
- The stream opens lazily on first read, so a write-only handle and a file
  whose bytes are never read spend neither a resolve nor a slot against the
  engine's stream ceiling. `release` closes it and drops the plaintext it
  cached; `unmount` does the same for every open handle.
- The cache is keyed by the engine stream handle, which pins one immutable
  version for its whole life, so a cached block can never be served as a slice
  of a different version. It is bounded in bytes, evicted least-recently-used,
  and every block is `Zeroizing`, so zeroization rides `Drop` rather than an
  unmount callback that a forced teardown would skip.
- Opening a stream verifies a head version the kernel's page cache may predate,
  which is where `Invalidation::Data` is now constructed.

The `fuse-op-core` suite gains the read path and a named CI gate, `FUSE Op
Core`, as testing law 1 requires.

Closes #1109
Part of #648

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WegkkQ3uhNREerTW4MMeY2
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2bae95f7-15f2-4066-842c-f3c144b7b80b

📥 Commits

Reviewing files that changed from the base of the PR and between 0e5a1d6 and 7bba021.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • .github/workflows/ci.yml
  • blueprint/testing.md
  • crates/fuse/Cargo.toml
  • crates/fuse/src/cache.rs
  • crates/fuse/src/handle.rs
  • crates/fuse/src/lib.rs
  • crates/fuse/src/ops.rs
  • crates/fuse/tests/fuse_op_core.rs

Walkthrough

The FUSE operation core now supports ranged reads with per-handle streams and a bounded, zeroized plaintext chunk cache. Handle release and unmount clean streams and cached data. Tests and CI now cover this behavior.

Changes

FUSE read and cache flow

Layer / File(s) Summary
Bounded plaintext cache foundation
crates/fuse/Cargo.toml, crates/fuse/src/cache.rs, crates/fuse/src/lib.rs
Added production and CI cache budgets, zeroizing buffer growth, byte-bounded LRU storage, stream-specific cleanup, and cache accounting.
Stream-aware operation core
crates/fuse/src/handle.rs, crates/fuse/src/ops.rs
Added per-handle stream ownership. OperationCore now fetches cache-aligned blocks, serves ranged reads, invalidates changed projections, and cleans streams and cached plaintext on release or unmount.
Integration validation and CI gate
crates/fuse/tests/fuse_op_core.rs, blueprint/testing.md, .github/workflows/ci.yml
Added coverage for ranged reads, cache hits and eviction, stream cleanup, invalidation, and independent handles. Added a dedicated Rust-gated fuse-op-core CI job.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant FUSEClient
  participant OperationCore
  participant ChunkCache
  participant StreamHandle
  FUSEClient->>OperationCore: request ranged read
  OperationCore->>ChunkCache: check stream/block cache
  alt cache miss
    OperationCore->>StreamHandle: fetch content block
    OperationCore->>ChunkCache: cache plaintext block
  end
  ChunkCache-->>OperationCore: provide plaintext block
  OperationCore-->>FUSEClient: return requested range
Loading

Possibly related issues

Possibly related PRs

  • FSM1/cipher-box#885 — Introduced the OperationCore, ChunkCache, and HandleTable areas extended by this PR.
  • FSM1/cipher-box#1061 — Directly relates to attaching, releasing, and cleaning up FUSE streams.
  • FSM1/cipher-box#607 — Overlaps with the FUSE plaintext caching, stream lifecycle, and operation-core read paths.

Suggested labels: release:cipherbox-fuse:feat

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main changes: FUSE ranged reads and the plaintext chunk cache.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/fuse-ranged-read-and-plaintext-chunk-cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Review findings on the read path.

- The buffer a read assembles its window in was a plain `Vec<u8>`: every
  `extend_from_slice` past capacity freed an allocation still holding decrypted
  bytes, and the `?` on a failed chunk fetch dropped a partly-filled one. It is
  now `Zeroizing`, grown through a wiping reserve the way the engine grows its
  own leaf-assembly buffer, and handed out by `mem::take` so the caller becomes
  the terminal owner.
- `CacheBudget::for_profile` multiplied without a check, so a wrapped product
  could hand back a plaintext ceiling the mount cannot keep. It is now
  `checked_mul`, alongside the existing zero-block rejection.
- Serve the gateway-URL CID parse from `testkit::requested_cid` rather than a
  fourth copy of it.
- Say plainly that the cache's block size is chosen to match the chunk framing
  rather than enforced against it — only performance rests on the match, since
  the engine clamps every window to the pinned version.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WegkkQ3uhNREerTW4MMeY2
@FSM1
FSM1 marked this pull request as ready for review August 6, 2026 14:40
@FSM1
FSM1 merged commit 77ad6a7 into main Aug 6, 2026
38 checks passed
@FSM1
FSM1 deleted the feat/fuse-ranged-read-and-plaintext-chunk-cache branch August 6, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

desktop: implement the fuse ranged read path and the plaintext chunk cache

1 participant