-
Notifications
You must be signed in to change notification settings - Fork 0
Fleet: retry-safe task requests and truthful status #288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
6eeb149
feat(fleet): add retry-safe task requests and truthful status
itsHabib 363fd07
fix(fleet): preserve request replay and observe configured write hooks
itsHabib 9a243af
fix(fleet): retain branch activity and scope legacy maintenance
itsHabib 2caab54
docs(fleet): record final panel residuals for judgment
itsHabib 799313e
Merge remote-tracking branch 'origin/main' into feat/fleet-task-coord…
itsHabib 33d284b
Merge remote-tracking branch 'origin/main' into feat/fleet-task-coord…
itsHabib 8dc96bd
Merge remote-tracking branch 'origin/main' into feat/fleet-task-coord…
itsHabib a7f1ac6
Merge remote-tracking branch 'origin/main' into feat/fleet-task-coord…
itsHabib 977cf74
test(fleet): assert truthful watcher lock refusal after integration
itsHabib 376f3b9
fix(fleet): request keys by git's branch spelling; damaged request ro…
itsHabib bd4ea8d
fix(fleet): replay survives branch deletion under the caller's spelli…
itsHabib 9e0a07a
fix(fleet): replay identity is exact — git's spelling when the ref re…
itsHabib File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| <!-- reaper-work:v1 --> | ||
| # Work: Retry-safe Fleet assignments and truthful observation | ||
|
|
||
| Work-ID: fleet-task-coordination | ||
| Status: active | ||
| Subject: git:f1cea9a96ef77680ab3a6e05ed1614d6b8048e40 | ||
| Stop-at: reviewed-change | ||
|
|
||
| ## Outcome | ||
|
|
||
| A supervisor can retry an assignment without overwriting or duplicating work and | ||
| inspect hook-observed activity without claiming delivery, acceptance or termination. | ||
|
|
||
| ## Preserve | ||
|
|
||
| - Existing branch leases, hooks, Gate authority and legacy dispatch behavior except | ||
| refusing uncorrelated replacement of a request-bound or unreadable assignment. | ||
| - Root cmd/triage/labels/mismatches.jsonl and earlier worktree friction logs are unrelated. | ||
| - One existing dispatch store and hook-owned facts; no second editable ledger. | ||
|
|
||
| ## Change | ||
|
|
||
| - `FOLLOWUPS.md`: final-panel residuals and written deferral rationale. | ||
| - `cmd/fleet/internal/verbs/request.go`: immutable request IDs and serialized replay. | ||
| - `cmd/fleet/internal/verbs/role.go`: subscribe generated hooks to write events. | ||
| - `cmd/fleet/internal/verbs/role_hooks_test.go`: generated subscription and rebind coverage. | ||
| - `cmd/fleet/internal/verbs/status.go`: read-only plain-language request observations. | ||
| - `cmd/fleet/internal/verbs/work.go`: protect request records from legacy mutations. | ||
| - `cmd/fleet/internal/verbs/verbs.go`: command entrypoints before lazy migration. | ||
| - `cmd/fleet/internal/fleet/session.go`: merge per-branch observations under the session lock. | ||
| - `cmd/fleet/internal/codex/task_activity_test.go`: patch adapter lease and evidence proof. | ||
| - `cmd/fleet/internal/fleet/hook.go`: completed write-tool observation in session record. | ||
| - `cmd/fleet/internal/mcp/mcp.go`: equivalent request/status tool entrypoints. | ||
| - `cmd/fleet/internal/verbs/request_test.go`: replay, conflicts, races and read-only proof. | ||
| - `cmd/fleet/internal/fleet/task_activity_test.go`: post-tool activity provenance. | ||
| - `cmd/fleet/internal/mcp/task_test.go`: non-migrating observation through JSON-RPC. | ||
| - `cmd/fleet/README.md`: interface, current limits and next adapter proof. | ||
|
|
||
| ## Prove | ||
|
|
||
| - Green: go test -race ./cmd/fleet/...; go vet ./cmd/fleet/...; golangci-lint run ./cmd/fleet/... | ||
| - Green: both cmd/fleet/testdata/run-suite.sh harness suites; focused real Git CLI exercise. | ||
| - Red: competing requests, changed replay, damaged evidence and late unrelated activity never yield false acceptance. | ||
|
|
||
| ## Stop | ||
|
|
||
| - No live worker launches, stops, lease transfers or installed hook changes from this PR. | ||
| - Delivery, semantic acceptance, correlated answers and effect-safe replacement remain | ||
| follow-on implementation under tsk_01M1ZJVZ1ZDHJC1PR1AZGE47TC, not claimed complete. | ||
|
|
||
| ## Evidence | ||
|
|
||
| - Verified: focused Fleet race tests, root Go vet/lint, and Claude regression scenarios pass. | ||
| - Verified: separate-process replay/conflict tests and Codex regression scenarios pass. | ||
| - Verified: incomplete MCP status remains parseable JSON; compiled-binary fixture smoke passes for both adapter shapes. | ||
| - Verified: initial-head full-module CI and all three configured reviewers completed. | ||
| - Verified: final code head 9a243af has green CI and completed three-member panel. | ||
| - Residual: FOLLOWUPS.md records Windows spelling, incomplete request evidence and lock diagnostics for judgment. | ||
|
|
||
| ## Handoff | ||
|
|
||
| - Last: second panel consolidated; per-branch evidence, MultiEdit subscription and scoped legacy maintenance fixed. | ||
| - Next: judge recorded residuals; no more panel cycles, no merge or live installation. Broader adapter work remains open. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # Fleet task coordination: first implementation increment | ||
|
|
||
| This adds retry-safe local assignments and a read-only observation view. It does | ||
| not yet implement the four-interaction product: launch/delivery, semantic worker | ||
| acceptance, correlated questions, safe stop and replacement remain adapter work. | ||
| Do not activate a live trial or present this as cross-harness lifecycle parity. | ||
|
|
||
| The approved direction is [cc-skills PR #60](https://github.com/itsHabib/cc-skills/pull/60): | ||
| one lead, one active worker, task-owned workspace and natural interaction through | ||
| the supervisor skill. The interfaces below are for the supervisor/adapter, not a | ||
| set of commands the operator should have to learn. | ||
|
|
||
| ## Record once, retry safely | ||
|
|
||
| From the task's repository checkout, with a known worker: | ||
|
|
||
| ```sh | ||
| fleet request my-branch --id navigation-fix-1 --worker SESSION \ | ||
| --for supervisor:ivy --brief 'Reproduce and fix the navigation failure; return focused checks.' | ||
| fleet status | ||
| fleet status --json | ||
| ``` | ||
|
|
||
| Use the discovered executable path if Fleet is not on PATH. The installed Mac | ||
| entrypoint during this build was `/Users/mh/.fleet/bin/fleet`; the new commands | ||
| are not available there until a reviewed release is installed. | ||
|
|
||
| Equivalent MCP tools are `fleet_request` (requires caller cwd) and `fleet_status`. | ||
| The supervisor chooses a stable request ID before calling. Same repo + same ID + | ||
| same branch/worker/lead/brief returns the existing assignment without renewing its | ||
| timestamp, resetting its initial head, posting a message or acquiring a lease. | ||
| Full-ID retries remain valid after branch deletion or session-record cleanup; a | ||
| short session prefix must still resolve uniquely. Supply a branch name, not a | ||
| numbered change. Changing the payload under that ID refuses. A second assignment for the same | ||
| branch refuses, as do unknown ownership, an unavailable worker and an applicable | ||
| stop flag. A recorded assignment is not an execution reservation; the ordinary | ||
| hook/lease guard still controls actual effects. | ||
|
|
||
| Records extend the existing `dispatch` row with `request_id` and `worker`. | ||
| One dispatch-store lock serializes decisions across processes, followed by the | ||
| existing branch lock for ownership inspection. Legacy dispatch/reassign/undispatch | ||
| cannot overwrite or delete these records, including with `--take`. They remain | ||
| retained until a correlated lifecycle operation is implemented. Do not remove | ||
| records manually to reuse IDs. Ordinary legacy records remain supported. | ||
|
|
||
| `request` is effectful and performs the existing lazy key migration before lease | ||
| inspection. Retained collisions refuse; failed requests can leave a migration | ||
| marker/lock but no new assignment. No GitHub write or worker launch occurs. | ||
|
|
||
| ## Observe without claiming more than the evidence | ||
|
|
||
| `status` bypasses migration in both CLI and MCP and restores read-only mode after | ||
| rendering. JSON is `fleet-task-status-v1`, scoped to local request-bound assignments; | ||
| it is not the full portfolio inventory or permission to dispatch. `complete` | ||
| means the assignment sources were readable, not that every task is healthy. | ||
|
|
||
| - **Queued:** the assignment exists; delivery and acceptance are unconfirmed. | ||
| - **Activity observed:** the selected worker has a matching post-dispatch write-tool | ||
| event on the task branch. This is not a claim of successful edits or acceptance. | ||
| - **Status needs checking:** conflicting/unreadable ownership, stop flag or missing | ||
| worker liveness. A stopped/dead session does not establish command quiescence. | ||
|
|
||
| The existing hook-owned session record carries `last_writes`, keyed by branch, | ||
| with time and tool-use ID; `last_write` remains for compatibility. Writes on a | ||
| second branch do not erase the first branch observation. Read-only commands, old activity and another branch do not count. | ||
| JSON keeps IDs and evidence timestamps for debugging; terminal output does not | ||
| require the operator to interpret internal session IDs. No new agent-written | ||
| progress ledger, acceptance claim, done state or automatic takeover is introduced. | ||
|
|
||
| Generated role bindings subscribe Codex write events and supplement Claude | ||
| file-write post-tool events alongside its global Bash hook. Existing bindings | ||
| need regeneration and harness reload when this release is installed. This PR | ||
| does not edit installed hooks. Terminal observations include the activity age. | ||
|
|
||
| ## Verification | ||
|
|
||
| Run `go test -race ./cmd/fleet/...`, `go vet ./cmd/fleet/...` and | ||
| `golangci-lint run ./cmd/fleet/...`, then both `testdata/run-suite.sh` variants | ||
| (default and `codex`). New tests cover real Git state, separate-process replay and | ||
| conflicts, immutable payloads, legacy-writer protection, damaged evidence, | ||
| post-tool provenance and non-migrating JSON-RPC observation. Harness event | ||
| fixtures are not proof of actual live Claude/Codex delivery or stop behavior. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| package codex | ||
|
|
||
| import ( | ||
| "github.com/itsHabib/workbench/cmd/fleet/internal/fleet" | ||
| "os" | ||
| "path/filepath" | ||
| "testing" | ||
| ) | ||
|
|
||
| func TestPatchAdapterEnforcesLeaseAndRecordsActivity(t *testing.T) { | ||
| oldState, oldOrg := fleet.State, fleet.OrgState | ||
| root := t.TempDir() | ||
| fleet.State, fleet.OrgState = filepath.Join(root, "state"), filepath.Join(root, "org") | ||
| t.Cleanup(func() { fleet.State, fleet.OrgState = oldState, oldOrg }) | ||
| repo := filepath.Join(root, "repo") | ||
| if err := os.MkdirAll(filepath.Join(repo, ".git", "refs", "heads"), 0700); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| if err := os.WriteFile(filepath.Join(repo, ".git", "HEAD"), []byte("ref: refs/heads/task\n"), 0600); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| key := fleet.Scope(repo, "task") | ||
| if err := fleet.WriteJSON(fleet.Path("sessions", "holder.json"), fleet.Rec{"session": "holder", "pid": os.Getpid(), "pid_kind": "harness", "last_event_at": fleet.Now()}); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| if err := fleet.WriteLease(key, fleet.LeaseRecord(key, "holder", "", repo, nil)); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| ev := fleet.Event{"session_id": "worker", "cwd": repo, "hook_event_name": "PreToolUse", "tool_name": "apply_patch", "tool_input": fleet.Rec{"command": "*** Begin Patch\n*** Add File: file.txt\n+hello\n*** End Patch"}} | ||
| if v := Run(ev); v.Code != 2 { | ||
| t.Fatalf("patch escaped foreign lease: %v", v) | ||
| } | ||
| if fleet.S(fleet.Lease(key), "session") != "holder" { | ||
| t.Fatal("foreign lease changed") | ||
| } | ||
| ev["session_id"] = "holder" | ||
| if v := Run(ev); v.Code != 0 { | ||
| t.Fatal(v) | ||
| } | ||
| ev["hook_event_name"] = "PostToolUse" | ||
| if v := Run(ev); v.Code != 0 { | ||
| t.Fatal(v) | ||
| } | ||
| write := fleet.M(fleet.M(fleet.SessionRecord("holder"), "last_writes"), key) | ||
| if fleet.S(write, "key") != key || fleet.F(write, "at") == 0 { | ||
| t.Fatalf("patch observation absent: %v", write) | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| package fleet | ||
|
|
||
| import ( | ||
| "os" | ||
| "path/filepath" | ||
| "testing" | ||
| ) | ||
|
|
||
| func TestPostWriteEvidenceOnlyObservedTarget(t *testing.T) { | ||
| oldState, oldOrg := State, OrgState | ||
| root := t.TempDir() | ||
| State, OrgState = filepath.Join(root, "fleet"), filepath.Join(root, "org") | ||
| t.Cleanup(func() { State, OrgState = oldState, oldOrg }) | ||
| repo := filepath.Join(root, "repo") | ||
| if err := os.MkdirAll(filepath.Join(repo, ".git", "refs", "heads"), 0700); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| _ = os.WriteFile(filepath.Join(repo, ".git", "HEAD"), []byte("ref: refs/heads/task\n"), 0600) | ||
| ev := Event{"session_id": "worker", "cwd": repo, "hook_event_name": "PostToolUse", "tool_name": "Read", "tool_input": Rec{"file_path": filepath.Join(repo, "file")}} | ||
| if len(postWriteEvidence(ev, "worker")) != 0 { | ||
| t.Fatal("read counted as write activity") | ||
| } | ||
| ev["tool_name"] = "Edit" | ||
| got := postWriteEvidence(ev, "worker") | ||
| if S(M(got, "last_write"), "key") != Scope(repo, "task") || F(M(got, "last_write"), "at") == 0 { | ||
| t.Fatal(got) | ||
| } | ||
| // Actual hook persists this fact for both adapter faces; it is not an agent-written acceptance. | ||
| if v := Run(ev); v.Code != 0 { | ||
| t.Fatal(v) | ||
| } | ||
| if S(M(SessionRecord("worker"), "last_write"), "key") != Scope(repo, "task") { | ||
| t.Fatal(SessionRecord("worker")) | ||
| } | ||
| ev["tool_input"] = Rec{"file_path": filepath.Join(root, "outside")} | ||
| if len(postWriteEvidence(ev, "worker")) != 0 { | ||
| t.Fatal("outside target attributed to task") | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For Codex installations generated by
fleet role,cmd/fleet/internal/verbs/role.go:140registersPostToolUseonly for^Bash$. Consequently this new evidence path is never invoked for ordinaryEdit/Write/apply_patchactivity, so a Codex worker can modify the assigned branch whilefleet statusremainsQueued. Broaden the installed post-tool matcher to include the write tools handled here.Useful? React with 👍 / 👎.