Skip to content

fix(copy): skip source files that vanish mid-copy instead of aborting - #3744

Open
dataders wants to merge 1 commit into
max-sixty:mainfrom
dataders:fix/copy-ignored-skip-vanished-source
Open

fix(copy): skip source files that vanish mid-copy instead of aborting#3744
dataders wants to merge 1 commit into
max-sixty:mainfrom
dataders:fix/copy-ignored-skip-vanished-source

Conversation

@dataders

@dataders dataders commented Aug 5, 2026

Copy link
Copy Markdown

Fixes #3743.

Problem

copy_leaf propagates NotFound from symlink_metadata/read_link/reflink_or_copy as a hard error. copy_dir_recursive collects leaves in a walk phase, then copies them in a parallel try_for_each — so a single source file that disappears between those two phases (e.g. a concurrent build rewriting target/) aborts the entire batch, even though most other files copy successfully.

Fix

Treat NotFound on the source the same way the existing AlreadyExists case (destination) is already handled: skip the leaf and return Ok(None) instead of erroring.

Testing

  • Added test_copy_leaf_skips_vanished_source (unit test, deterministic — no race needed).
  • cargo test --lib copy:: — 3 passed.
  • cargo test step_copy_ignored (full integration suite) — 56 passed.
  • cargo clippy --lib -- -D warnings — clean.

copy_leaf propagated ENOENT from symlink_metadata/read_link/reflink_or_copy
as a hard error, so a single file deleted between copy_dir_recursive's walk
and its parallel copy phase (e.g. a concurrent build rewriting target/)
aborted the entire batch via rayon's try_for_each — even though most other
files copy fine. Treat NotFound the same as the existing AlreadyExists case:
skip the leaf and return Ok(None).

Fixes max-sixty#3743

@worktrunk-bot worktrunk-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direction is right and the AlreadyExists precedent is the correct one to follow, but two things stand in the way as written.

1. Under --force the skip converts a loud failure into a silent hole. copy_leaf removes the destination before it ever touches the source (if force { remove_if_exists(dest)?; }, the first thing after the root check). With the new NotFound arms, a source that vanishes after that point returns Ok(None) — so wt step copy-ignored --force deletes an existing destination file, copies nothing, counts it as a skip, and exits 0. Before this PR that path errored. I confirmed it by adding a throwaway test locally against this branch's copy_leaf:

fs::write(&dest, b"important pre-existing content").unwrap();
let result = copy_leaf(&src /* does not exist */, &dest, None, true).unwrap();
assert_eq!(result, None);   // passes — reported as skipped
assert!(!dest.exists());    // passes — destination gone, nothing written

That is the shape CLAUDE.md's Data Safety rules out ("Prefer failure over silent loss" / "No implicit destructive side effects"). The inline suggestions gate each new arm on !force, which keeps the non-force fix — the reported scenario — while leaving --force loud. A more complete alternative is to read the source metadata before the force removal, so the common case never removes a destination it can't replace; the reflink_or_copy arm still needs the !force guard either way, since that window can't be closed by ordering.

2. The fix only covers leaves, so the reported failure can still fire. The race in copy_dir_recursive is not specific to files. Phase 1 does let entries: Vec<_> = fs::read_dir(&src_dir)?... with a bare ? (no NotFound arm, and no .with_context() either — the user gets a raw No such file or directory (os error 2) with no path), and Phase 3's permission pass does fs::metadata(src_dir).with_context(|| format!("reading permissions for {}", ...))? after every leaf already copied successfully. A concurrent build that removes a directory under target/ rather than a single object file still aborts the whole batch — same symptom, same command, and the issue's repro (an active cargo build) reaches it. Both want the same NotFound → skip treatment. Those lines are outside the diff; happy to push a commit adding them here if you'd like.

Smaller notes, no action needed unless you agree:

  • The reflink_or_copy NotFound arm also swallows a missing destination parent, not just a vanished source. Both current call sites create the parent first, so it's latent rather than live, but the arm is less specific than the comment above it suggests.
  • Only the symlink_metadata arm is covered by a test; the read_link and reflink_or_copy arms are unreachable deterministically, so codecov/patch may come back red on this patch.

Holding off on approval and pulling in @max-sixty: src/copy.rs is on the deletion surface (copy_leafremove_if_existsfs::remove_file), and point 1 changes what happens after that removal, so this isn't one for me to sign off.

Comment thread src/copy.rs
Comment on lines +74 to +77
// The source can vanish between the caller's directory walk and this
// copy — e.g. a concurrent build rewriting `target/`. Skip rather
// than fail the whole batch over one file that's no longer there.
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With force, remove_if_exists(dest) has already run by the time we get here, so returning Ok(None) leaves the destination deleted and reports a skip. Gate the skip on !force so the batch-resilience win applies where it's needed without silently dropping a file the user asked to overwrite:

Suggested change
// The source can vanish between the caller's directory walk and this
// copy — e.g. a concurrent build rewriting `target/`. Skip rather
// than fail the whole batch over one file that's no longer there.
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),
// The source can vanish between the caller's directory walk and this
// copy — e.g. a concurrent build rewriting `target/`. Skip rather
// than fail the whole batch over one file that's no longer there.
// Not under `force`: the destination was already removed above, so a
// silent skip would leave a hole where a file used to be.
Err(e) if e.kind() == ErrorKind::NotFound && !force => return Ok(None),

Comment thread src/copy.rs
fs::read_link(src).with_context(|| format!("reading symlink {}", src.display()))?;
let target = match fs::read_link(src) {
Ok(target) => target,
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the arm above — under force the destination is already gone at this point.

Suggested change
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),
Err(e) if e.kind() == ErrorKind::NotFound && !force => return Ok(None),

Comment thread src/copy.rs
}
}
Err(e) if e.kind() == ErrorKind::AlreadyExists => return Ok(None),
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reasoning: with force the destination was removed before the copy started, so a silent skip here loses it.

Worth noting this arm is broader than the vanished-source case it's meant for — reflink_or_copy also reports NotFound when the destination's parent directory is missing. Both current callers create the parent first, so nothing hits it today, but it means a genuine "can't write there" turns into a silent no-op rather than an error.

Suggested change
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),
Err(e) if e.kind() == ErrorKind::NotFound && !force => return Ok(None),

@worktrunk-bot
worktrunk-bot requested a review from max-sixty August 5, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

step copy-ignored aborts entire copy when a source file vanishes mid-walk

2 participants