fix(runtime)!: delete task worktrees from Ultrafuzz once outputs are published (#1227)

[aviggiano]

Problem

With keep_workspaces = false, the workflow engine deleted a successful task's worktree only when git status showed it clean. Ultrafuzz tasks write outputs, test/foundry/<node>/ and foundry.lock into their worktree, so in practice every worktree was left behind (#1227). Reads of verified output also re-checked facts against the task worktree, so deleting it would have made reports unverifiable.

Change

• The engine always runs with SMITHERS_KEEP_WORKTREES=1. The keepWorkspaces plumbing through smithers.ts and start-run.ts is removed.
• The first sync after a run ends deletes worktrees when keep_workspaces = false, read from the launch config. For every terminal attempt that has successful finalization authority, or whose artifacts/<attempt>/ mirror holds no file, it deletes the worktree, its Git registration and its ultrafuzz/<run>/<attempt> branch.
  • A task whose mirror holds a file, such as a rejected output of a continue-policy task, keeps its worktree. Ultrafuzz never deletes the only copy of a rejected output.
• removeRunTaskWorktrees renames a worktree into workspaces/.removing/ before Git forgets it. An interrupted removal therefore never leaves a task path without .git, and the next sync finishes it.
  • It only touches directories that are registered at their own path on their own branch and are not locked.
  • A failure is reported as a TASK_WORKTREE_REMOVAL_FAILED warning.
• Controller finalization still checks against the task worktree. Reads of finalized output pass taskWorktree: "skip" and check published bytes only; this covers report, report bundle, the dashboard, Modal and evals.
  • Skipped on read: the workspace-patch Git binding, the invariant source-proof Git binding, the scan-probe lstat checks, the coverage source inventory, and aggregation source and destination reconciliation.
  • An invariant ledger read under skip needs its durable source-proofs/<attempt>.invariant.json; there is no workspace fallback.
  • Published scan-probe paths are still checked lexically, because the schema does not constrain them.
• Strategy-worktree policy: test/foundry/ and foundry.lock get no special handling. They are deleted with the worktree. This is documented in docs/reference/configuration.md.
• verifyCoverageProductionInventory drops a redundant if (evidence.status === "unavailable") return because the next condition already returns for any status other than "measured". This makes room under the lint complexity cap of 83 for the new skip check. No behaviour change.

Breaking changes

• With the default keep_workspaces = false, task worktrees and their ultrafuzz/<run-id>/<attempt> branches are deleted after the run ends. Undeclared edits, test/foundry/<node>/ and foundry.lock go with them. After upgrading, the next status of a run that ended earlier deletes its worktrees by the same rule.
• keep_workspaces is read from the configuration fixed at launch. Setting it, or ULTRAFUZZ_KEEP_WORKSPACES, for resume has no effect.
• Reads of verified output no longer re-check the worktree-bound facts listed above. Controller finalization still does.

Tests

• stale-worktree-recovery.test.ts:
  • removes a worktree with a submodule, test/foundry/ and foundry.lock;
  • never touches locked, foreign-branch, unregistered or outside directories;
  • finishes interrupted removals;
  • stops at a budget checkpoint between attempts;
  • ignores a non-Git project.
• smithers-artifact-publication.integration.test.ts: a real engine keeps the worktree, and recreates it when the task reruns after Ultrafuzz removed it.
• runtime.test.ts:
  • the sweep keeps a rejected output;
  • it honours keep_workspaces from the launch config;
  • a removal failure reports TASK_WORKTREE_REMOVAL_FAILED.
• artifact-gates.test.ts and verified-output.test.ts: verified reads succeed after the worktree is deleted, including a finalized workspace-patch@1 producer.
• CLI e2e campaign-resume.test.ts: after the first status, workspaces/ is empty and no registration or branch remains. The report is still verified-runtime-report.
• Gates pass: format:check, lint, lint:strict:ci, docs:check, typecheck and knip.

Credit: mrthankyou investigated this in #1259.

🤖 Generated with Claude Code

Confidence Score: 5/5

The latest changes appear safe to merge, although two previously reported, non-blocking cleanup issues remain.

Findings

1.  Git listing failures go unnoticed ▶
2.  Leftover can delete reused branch ▶

Fix with agent prompt

### Issue 1
packages/runtime/src/stale-worktree-recovery.ts:undefined-143
If `git worktree list` fails for an existing Git project, this returns without removing any worktrees or reporting `TASK_WORKTREE_REMOVAL_FAILED`. The status command can appear successful while cleanup remains incomplete, leaving operators without a warning that it needs attention. Report the failure while retaining the intended behavior for non-Git projects.

### Issue 2
packages/runtime/src/stale-worktree-recovery.ts:undefined-163
If an interrupted removal leaves an entry in `.removing` after its registration is gone, and that branch is later reused at another worktree, the next sync deletes the active branch here. No matching registration is required when the original task path is absent. Check that the leftover still owns the branch before deleting it.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR moves task-worktree cleanup from the workflow engine to Ultrafuzz, lets finalized-output reads verify published artifacts without the deleted worktree, and documents the resulting retention behavior. Since the previous review, it also advances the engine dependency-resolution cutoff and updates transitive packages in the lockfile.

• The two previously reported cleanup issues remain in the current code; neither was changed by the latest revision.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Task finishes] --> B[Controller finalizes outputs]
  B --> C[Run ends]
  C --> D[Sync checks sealed keep_workspaces setting]
  D -->|false and disposable| E[Remove task worktree and Git branch]
  D -->|true or rejected output retained| F[Keep worktree]
  E --> G[Read verified published output]

Loading

_{Reviews (2) · Last reviewed commit: "Merge branch 'unstable' into fix/1227-re..."}

[greptile-apps]

Git listing failures go unnoticed

If git worktree list fails for an existing Git project, this returns without removing any worktrees or reporting TASK_WORKTREE_REMOVAL_FAILED. The status command can appear successful while cleanup remains incomplete, leaving operators without a warning that it needs attention. Report the failure while retaining the intended behavior for non-Git projects.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/stale-worktree-recovery.ts
Line: 143

Comment:
**Git listing failures go unnoticed**

If `git worktree list` fails for an existing Git project, this returns without removing any worktrees or reporting `TASK_WORKTREE_REMOVAL_FAILED`. The status command can appear successful while cleanup remains incomplete, leaving operators without a warning that it needs attention. Report the failure while retaining the intended behavior for non-Git projects.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Leftover can delete reused branch

If an interrupted removal leaves an entry in .removing after its registration is gone, and that branch is later reused at another worktree, the next sync deletes the active branch here. No matching registration is required when the original task path is absent. Check that the leftover still owns the branch before deleting it.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/stale-worktree-recovery.ts
Line: 163

Comment:
**Leftover can delete reused branch**

If an interrupted removal leaves an entry in `.removing` after its registration is gone, and that branch is later reused at another worktree, the next sync deletes the active branch here. No matching registration is required when the original task path is absent. Check that the leftover still owns the branch before deleting it.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.