Two processes on one machine share a lease name, so a routine can fire twice

[zopeVaibhav]

Every process that takes work off the shared queue names itself after its hostname, and the queue decides whether an item is still yours by comparing that name and nothing else. Two processes on one machine therefore carry the same name, and the lease stops meaning anything between them: one whose lease has lapsed is still told the item is its own, and both go on to dispatch it.

What the queue actually compares

ours in server/src/work/queue.ts:195 is the guard behind renew, finish and release:

and(
  eq(workItems.kind, kind),
  eq(workItems.key, key),
  eq(workItems.claimedBy, owner),
  isNull(workItems.finishedAt),
)

There is no lease comparison, deliberately — claim already refuses a row whose lease_until is in the future, so ownership is meant to be settled by the name. The name is built in six places, all the same shape:

server/src/index.ts:878,1025,1064      handoff/ reaper/ summariser/  ${process.env.HOSTNAME ?? randomUUID().slice(0, 8)}
server/scripts/cull-idle-computers.ts  culler/
server/scripts/fire-routines.ts        routines/
worker/src/env.ts:70                   routines/${host || generateId()}

The random part is a fallback for a missing hostname, not a per-process identity. With a hostname — every ordinary case — two processes on one machine are indistinguishable to ours.

The five that use ?? have a second problem: it falls back only on undefined, so HOSTNAME="" makes the owner the constant handoff/, shared by every replica in the deployment rather than merely by one machine. That is the empty-string trap #96, #114, #312, #343, #374 and #395 each found elsewhere, landing here on the one value that keeps two replicas apart.

Reproduction

Against the real queue and Postgres, one item, two claimants, lease allowed to lapse between them:

same owner ("routines/laptop" twice)
  B took it after the lapse:    true
  A still believes it holds it: true      <- both dispatch

distinct owners ("routines/pod-a", "routines/pod-b")
  B took it after the lapse:    true
  A still believes it holds it: false     <- A is told it lost the lease

The only variable is whether the two processes derive the same name.

What a second dispatch costs

renew returning true is exactly what dispatchClaimedRoutines treats as permission to act; its own comment says a false there exists so there is "no dispatch, because that replica is already running this firing and a second dispatch is a second message to a person". insertRun (server/src/routines/store.ts:724) mints a fresh id and inserts unconditionally, so two dispatches are two run rows, two turns, and the same scheduled message delivered twice.

Why two workers on one machine is an ordinary thing to have

The worker binds no port, so there is no "address already in use" to stop a second one. scripts/start.sh does guard, with pgrep -f "bun worker/src/index.ts" — but bun run dev starts the worker as bun --watch src/index.ts from the worker directory, which that pattern does not match. So bun run dev followed by start.sh leaves two workers running, silently. The worker's own header says what it is for: "it is somebody's laptop, left running".

Severity

Moderate, and quiet. A hosted deployment on Kubernetes gets a unique hostname per pod and is unaffected unless HOSTNAME is set to an empty string, which the ?? above would then turn into one owner for every replica. What is reachable today, on a developer's machine or any single host running two of these processes, is a routine that fires twice and a person who receives the same summary twice, with both runs looking correct in the trail.