Every fleet spawn is dispatched twice (8/8 today); spawn survives only because agent names collide

[khaliqgant]

Every fleet spawn is dispatched twice — 8 of 8 today

Every spawn action issued in one session today produced two distinct invocation IDs. The second consistently fails with spawn_agent_name_in_use.

Sample pairs (first succeeded, second refused):

inv_216916774968586240  ->  {"spawned": true}
inv_216916775043461120  ->  spawn_agent_name_in_use

inv_217025730618630144  ->  {"spawned": true}
inv_217025730630524928  ->  spawn_agent_name_in_use

inv_217248326977896448  ->  {"spawned": true}
inv_217248327020621824  ->  spawn_agent_name_in_use

The IDs are near-adjacent, consistent with two near-simultaneous dispatches of one caller action rather than a retry after a failure — no failure precedes the second.

8 spawn calls, 8 duplicate pairs, 0 exceptions. Both local (chief-broker) and remote (sf-mini, daytona-1538-verify-0817b) targets.

Why this is currently harmless, and why that is the problem

Nothing bad happens for spawn — the agent-name uniqueness constraint refuses the second attempt, so exactly one agent starts and the name is never burned. I verified the roster holds exactly one record per name each time.

That safety is incidental. It comes from a name-collision check that exists for a different reason, not from the dispatch path being idempotent. So the open question is not "is spawn broken" — it is:

Are other fleet actions dispatched twice as well, and do any of them lack an equivalent guard?

An action without a natural uniqueness key — anything that appends, increments, charges, enqueues, or writes — would execute twice with no signal. That is the same class of defect as AgentWorkforce/factory#211 (duplicate dispatch across surfaces), which was merged today. This one sits in the transport underneath it.

What would settle it

1. Whether the duplication is in the caller (MCP tool invocation), the relay dispatch path, or the node handler.
2. An inventory of fleet actions and which have a natural idempotency guard. Spawn happens to. release, and anything that mutates counters or queues, may not.
3. If duplication is intended (at-least-once delivery), then it should be stated, and every action handler should be idempotent by contract rather than by luck.

Reproduction

Issue any spawn through the MCP fleet action and watch the invocation stream. Two action.completed / error events arrive with different invocation_ids for a single call.

Filed as an observation with complete evidence rather than a diagnosis — I have not read the dispatch path, and I am not claiming to know where the fork is.

[khaliqgant]

Version pinning, and a check against latest — this is NOT fixed in 11.8.2

I filed this without stating the version, which is the exact "deployed artifact vs latest" gap that cost this org four days on a separate incident. Correcting that here.

Observed on: agent-relay 11.8.1, agent-relay-broker 11.8.1 (both read back from the installed binaries, not inferred from a lockfile).

Latest published: agent-relay 11.8.2, published 2026-08-22T11:53:26Z. So the observation is one version behind current.

Does 11.8.2 fix it? No.

Only two commits separate the two versions:

09a136b01  fix(mcp): stop send_dm echoing the message body back to the sender (#1595)
9e5a4247f  fix(broker): bump relaycast to 7.0.0 and reclaim agent names by audited takeover (#1596)

Neither touches action dispatch or invocation handling. A search of commits since 2026-08-18 for duplicate|double|idempot|invocation returns nothing relevant — the only hit is an unrelated observer-link feature.

Control on that search: the same instrument over the same range returns the two commits above for a plain range query, so it is not silently empty.

So there is no reason to believe upgrading resolves this, and the 9-of-9 duplication observed on 11.8.1 should be expected to reproduce on 11.8.2.

One thing in 11.8.2 that is adjacent and worth flagging

#1596 — "reclaim agent names by audited takeover" — changes agent-name reclaim behaviour. The only reason the duplicate spawn is currently harmless is that the second attempt collides on the agent name and is refused with spawn_agent_name_in_use. If takeover semantics change how a name collision resolves, the duplicate could stop being refused and start succeeding.

That is speculation — I have not read #1596 — but it is the kind of interaction worth checking before upgrading, because it could convert a silently-harmless duplicate into a silently-successful one. The guard that makes this benign is not the dispatch path being idempotent; it is a uniqueness check that #1596 touches the neighbourhood of.

Restating the ask

The question is not "is spawn broken" — it is not. It is: are other fleet actions dispatched twice, and do any of them lack spawn's accidental name-collision guard? Anything that appends, increments, charges, or enqueues would run twice with no signal.

[khaliqgant]

Confirmed retry path that can create a second invocation

Read-only source audit on current Relay main plus the exact locked @relaycast/sdk@8.0.7 tarball found a concrete at-least-once path. This does not by itself prove which response failed in the 8/8 observation, but it proves the current client contract can duplicate any non-idempotent action.

• packages/cli/src/cli/agent-relay-mcp.ts:1225 calls the agent-scoped actions.invoke("spawn", actionInput) once.
• Relaycast 8.0.7 implements that as POST /v1/actions/{name}/invoke.
• The same 8.0.7 HttpClient.request() retries every method up to three times after a network exception and on 429/500/502/503/504. There is no method gate, idempotency key, or reuse of an invocation ID.
• Therefore, if the engine commits attempt 1 but the client sees a failed/ambiguous response, attempt 2 is a new POST and the engine is free to mint a second invocation ID. Spawn happens to collide on name; an append/charge/enqueue action does not.

Positive control: the packed 8.0.7 source contains maxRetries: 3, the shared while-true retry loop around fetch, and actions.invoke delegating to client.post(). This is not inferred from the repo lock or the locally installed 8.0.0 artifact.

Fix contract

The durable fix is an idempotency key accepted by the action-invoke endpoint and forwarded by clients, with the engine returning the original invocation for a replay. A client-only stopgap is to disable automatic retries for action invocation (and other non-idempotent POSTs), but that converts ambiguous committed writes into surfaced unknown outcomes; it does not provide safe retry. Add a must-not-duplicate test where attempt 1 commits and its response is lost, then the same key is retried. Inventory the other POST mutations behind this shared retry loop.

[khaliqgant]

Upstream durable-fix issue filed: AgentWorkforce/relaycast#354. It records the exact @relaycast/sdk@8.0.7 retry path, the distinct-invocation-ID evidence, the idempotency-key contract, and why relaycast#282 (same invocation ID dispatched twice inside the engine) is a separate failure mode. This gives #1604 an owning repository rather than leaving the transport fix parked in Relay.