Skip to content

Stale credential-home lock is never reclaimed when owner.json names a live or unusable pid, and the wait is silent and unbounded #228

Description

@rohanpoudel2

Summary

recoverStaleCredentialHomeLock decides whether a credential-home lock is stale from process.kill(owner.pid, 0) alone. Two separate problems follow:

  1. Values that are not process identifiers at all — 0, -1, fractional, out-of-range — are treated as "owner is alive" or throw a raw argument error, so the lock is never reclaimed.
  2. A genuinely reused pid is indistinguishable from the original owner, so a lock left by a killed scan is never reclaimed either.

In both cases acquireCodexSecurityCredentialHomeLock spins at CREDENTIAL_LOCK_POLL_MILLISECONDS (25 ms) forever, with no message and no timeout. Every other stale path — missing or corrupt owner.json — has a 30 s escape hatch; the pid branch has none.

Environment

  • @openai/codex-security 0.1.5, current main at a8fc009
  • macOS 15 (Darwin 25.5.0), Node.js 24.11.1, Bun 1.3.14
  • Reproduced against the exported acquireCodexSecurityCredentialHomeLock; no network or credentials required

The code

sdk/typescript/src/runtime.ts:

if (isRecord(owner) && typeof owner["pid"] === "number") {
  try {
    process.kill(owner["pid"], 0);
    return false;
  } catch (error) {
    if (nodeErrorCode(error) !== "ESRCH") {
      if (nodeErrorCode(error) === "EPERM") return false;
      throw error;
    }
  }
} else if (Date.now() - metadata.mtimeMs < INCOMPLETE_CREDENTIAL_LOCK_MILLISECONDS) {
  return false;
}

typeof … === "number" admits 0, -1, 0.5 and 2 ** 53. POSIX gives the first two special meanings — 0 signals the caller's own process group and -1 every process it may signal — so both always succeed and always report a live owner. A fractional or out-of-range value makes process.kill throw ERR_INVALID_ARG_TYPE, which is neither ESRCH nor EPERM and is rethrown raw out of a public API.

Note also that the age check is in the else branch, so a lock naming any accepted pid is exempt from it no matter how old it is.

Reproduction

Plant a lock directory with a chosen owner.json, age it, and call the public acquire with a 1.5 s abort:

dead pid, 1h old (control)                     ACQUIRED in 2ms
no owner.json, 1h old (control)                ACQUIRED in 1ms
reused pid (live), 24h old                     ABORT_ERR: hung 1501ms
pid 1 -> EPERM, 24h old                        ABORT_ERR: hung 1502ms
pid 0 -> process group, 24h old                ABORT_ERR: hung 1500ms
pid -1 -> every process, 24h old               ABORT_ERR: hung 1502ms
pid 0.5 -> ERR_INVALID_ARG_TYPE                ERR_INVALID_ARG_TYPE: hung 1ms

Only the abort ended the hang; a 24-hour-old lock is never reclaimed.

The existing test "recovers credential-home locks left by exited processes" only covers a genuinely exited pid, so none of this is pinned.

Why pid reuse is reachable, especially in the container

The lock is held for the whole scan (acquired in #run before runtime initialization, released in the outer finally), so it survives as long as the scan does — and outlives it if the process is SIGKILLed by docker stop, an OOM kill, or a host reboot.

For the shipped container the collision is close to guaranteed:

  • Dockerfile sets CODEX_SECURITY_STATE_DIR=/output/.codex-security-state
  • compose.yaml bind-mounts /output, so the lock persists across container runs
  • compose.yaml sets init: true, so container pids restart from 1 each run

A leftover {"pid": 7} from a killed run therefore names a live process in the next run, and every subsequent scan hangs with no output.

This may be related to #108 ("Run without output directory hangs computer … Randomly success, randomly failure"), whose randomness matches whether the leftover pid happens to collide. I have not confirmed that link.

Suggested direction

Three separable pieces:

  1. Validate the pid — only consult process.kill for Number.isSafeInteger(pid) && pid > 0, otherwise fall through to the existing age check. This is unambiguous and I have opened a PR for it.
  2. Survive pid reuse — the pid alone cannot prove identity. The usual fix is a heartbeat: have the holder refresh the lock directory's mtime while it works, and reclaim a lock whose mtime has not advanced for some multiple of that interval. A plain absolute age bound is not safe here, because the lock is legitimately held for the entire scan and scans can run well over an hour (see Standard full-repository scan shows only generic heartbeat for 40+ minutes — expected behavior? #70, Detect and warn from HEAD drifting earlier #164), so a fixed ceiling would let one scan steal another's lock. This needs a design decision, so I have deliberately left it out of the PR.
  3. Make the wait observable — right now the user sees nothing at all. Even without (2), reporting which pid holds the lock after a few seconds, and where the lock directory is, would turn an unexplained hang into something actionable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions