Skip to content

FileCache uses : in cache paths, creating NTFS alternate data streams on Windows #255

Description

@ricochet

wasm_pkg_client::FileCache derives cache paths directly from values containing ::

  • Blobs use digest.to_string(), producing paths such as sha256:<hex>.
  • Releases use <package>-<version>.json, while PackageRef displays as <namespace>:<name>.

On NTFS, the colon does not create an ordinary filename. It creates an alternate data stream:

  • sha256:<hex> becomes stream <hex> on a base file named sha256.
  • wasi:http-0.2.0.json becomes stream http-0.2.0.json on a base file named wasi.

The paths remain directly readable when their full names are known, which can hide the problem. However, normal directory enumeration only returns the base files and does not return the logical cache entries.

Impact

  • read_dir cannot enumerate blob or release entries on Windows.
  • Orphaned or truncated blobs cannot be discovered and removed through portable filesystem APIs.

This caused a Windows CI failure in wasmCloud when cache-integrity cleanup could not find a truncated blob: https://github.com/wasmCloud/wasmCloud/actions/runs/35755462351/job/106840065770

The downstream workaround has to enumerate NTFS streams explicitly: wasmCloud/wasmCloud#5602

Suggested direction

Use a versioned, filesystem-safe cache layout that does not place serialized identifiers directly in filenames. For example:

v2/
  blobs/
    sha256/
      <hex>
  releases/
    <namespace>/
      <package>/
        <version>.json

Blob writes should also use a temporary file followed by an atomic rename so interrupted downloads never leave a partial entry at the final digest path.

Because this is a cache, migration could be handled by using a new layout directory and treating older entries as cache misses. A temporary legacy lookup path could be retained if preserving existing caches is important.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions