fix(copy): skip source files that vanish mid-copy instead of aborting - #3744
fix(copy): skip source files that vanish mid-copy instead of aborting#3744dataders wants to merge 1 commit into
Conversation
copy_leaf propagated ENOENT from symlink_metadata/read_link/reflink_or_copy as a hard error, so a single file deleted between copy_dir_recursive's walk and its parallel copy phase (e.g. a concurrent build rewriting target/) aborted the entire batch via rayon's try_for_each — even though most other files copy fine. Treat NotFound the same as the existing AlreadyExists case: skip the leaf and return Ok(None). Fixes max-sixty#3743
worktrunk-bot
left a comment
There was a problem hiding this comment.
The direction is right and the AlreadyExists precedent is the correct one to follow, but two things stand in the way as written.
1. Under --force the skip converts a loud failure into a silent hole. copy_leaf removes the destination before it ever touches the source (if force { remove_if_exists(dest)?; }, the first thing after the root check). With the new NotFound arms, a source that vanishes after that point returns Ok(None) — so wt step copy-ignored --force deletes an existing destination file, copies nothing, counts it as a skip, and exits 0. Before this PR that path errored. I confirmed it by adding a throwaway test locally against this branch's copy_leaf:
fs::write(&dest, b"important pre-existing content").unwrap();
let result = copy_leaf(&src /* does not exist */, &dest, None, true).unwrap();
assert_eq!(result, None); // passes — reported as skipped
assert!(!dest.exists()); // passes — destination gone, nothing writtenThat is the shape CLAUDE.md's Data Safety rules out ("Prefer failure over silent loss" / "No implicit destructive side effects"). The inline suggestions gate each new arm on !force, which keeps the non-force fix — the reported scenario — while leaving --force loud. A more complete alternative is to read the source metadata before the force removal, so the common case never removes a destination it can't replace; the reflink_or_copy arm still needs the !force guard either way, since that window can't be closed by ordering.
2. The fix only covers leaves, so the reported failure can still fire. The race in copy_dir_recursive is not specific to files. Phase 1 does let entries: Vec<_> = fs::read_dir(&src_dir)?... with a bare ? (no NotFound arm, and no .with_context() either — the user gets a raw No such file or directory (os error 2) with no path), and Phase 3's permission pass does fs::metadata(src_dir).with_context(|| format!("reading permissions for {}", ...))? after every leaf already copied successfully. A concurrent build that removes a directory under target/ rather than a single object file still aborts the whole batch — same symptom, same command, and the issue's repro (an active cargo build) reaches it. Both want the same NotFound → skip treatment. Those lines are outside the diff; happy to push a commit adding them here if you'd like.
Smaller notes, no action needed unless you agree:
- The
reflink_or_copyNotFoundarm also swallows a missing destination parent, not just a vanished source. Both current call sites create the parent first, so it's latent rather than live, but the arm is less specific than the comment above it suggests. - Only the
symlink_metadataarm is covered by a test; theread_linkandreflink_or_copyarms are unreachable deterministically, socodecov/patchmay come back red on this patch.
Holding off on approval and pulling in @max-sixty: src/copy.rs is on the deletion surface (copy_leaf → remove_if_exists → fs::remove_file), and point 1 changes what happens after that removal, so this isn't one for me to sign off.
| // The source can vanish between the caller's directory walk and this | ||
| // copy — e.g. a concurrent build rewriting `target/`. Skip rather | ||
| // than fail the whole batch over one file that's no longer there. | ||
| Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None), |
There was a problem hiding this comment.
With force, remove_if_exists(dest) has already run by the time we get here, so returning Ok(None) leaves the destination deleted and reports a skip. Gate the skip on !force so the batch-resilience win applies where it's needed without silently dropping a file the user asked to overwrite:
| // The source can vanish between the caller's directory walk and this | |
| // copy — e.g. a concurrent build rewriting `target/`. Skip rather | |
| // than fail the whole batch over one file that's no longer there. | |
| Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None), | |
| // The source can vanish between the caller's directory walk and this | |
| // copy — e.g. a concurrent build rewriting `target/`. Skip rather | |
| // than fail the whole batch over one file that's no longer there. | |
| // Not under `force`: the destination was already removed above, so a | |
| // silent skip would leave a hole where a file used to be. | |
| Err(e) if e.kind() == ErrorKind::NotFound && !force => return Ok(None), |
| fs::read_link(src).with_context(|| format!("reading symlink {}", src.display()))?; | ||
| let target = match fs::read_link(src) { | ||
| Ok(target) => target, | ||
| Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None), |
There was a problem hiding this comment.
Same as the arm above — under force the destination is already gone at this point.
| Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None), | |
| Err(e) if e.kind() == ErrorKind::NotFound && !force => return Ok(None), |
| } | ||
| } | ||
| Err(e) if e.kind() == ErrorKind::AlreadyExists => return Ok(None), | ||
| Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None), |
There was a problem hiding this comment.
Same reasoning: with force the destination was removed before the copy started, so a silent skip here loses it.
Worth noting this arm is broader than the vanished-source case it's meant for — reflink_or_copy also reports NotFound when the destination's parent directory is missing. Both current callers create the parent first, so nothing hits it today, but it means a genuine "can't write there" turns into a silent no-op rather than an error.
| Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None), | |
| Err(e) if e.kind() == ErrorKind::NotFound && !force => return Ok(None), |
Fixes #3743.
Problem
copy_leafpropagatesNotFoundfromsymlink_metadata/read_link/reflink_or_copyas a hard error.copy_dir_recursivecollects leaves in a walk phase, then copies them in a paralleltry_for_each— so a single source file that disappears between those two phases (e.g. a concurrent build rewritingtarget/) aborts the entire batch, even though most other files copy successfully.Fix
Treat
NotFoundon the source the same way the existingAlreadyExistscase (destination) is already handled: skip the leaf and returnOk(None)instead of erroring.Testing
test_copy_leaf_skips_vanished_source(unit test, deterministic — no race needed).cargo test --lib copy::— 3 passed.cargo test step_copy_ignored(full integration suite) — 56 passed.cargo clippy --lib -- -D warnings— clean.