macOS launchd: ocx service crash-loops with OpenAiTierBackupCollisionError when current config differs from .pre-openai-tiers-v2.bak rollback

[lidge-jun]

Client or integration

Other

Installed macOS launchd service com.opencodex.proxy (ocx service). The same throw also aborts ocx start.

Area

Service lifecycle

Summary

On OpenCodex 2.13.0, an installed launchd service can report "registered / loaded" while never serving port 10100. Startup dies with OpenAiTierBackupCollisionError: Existing OpenAI tier backup differs from the current config when current ~/.opencodex/config.json differs from ~/.opencodex/config.json.pre-openai-tiers-v2.bak and that backup is still classified as a rollback snapshot (openaiProviderTierVersion !== 2).

KeepAlive then turns a single throw into a crash loop. ocx service status says the job is installed, but ocx health --json is {"ok":false,"pid":null,"port":null} and GET /healthz is connection-refused.

This is not a reopen of closed #257. #257 was the ocx init path writing a fresh config without openaiProviderTierVersion, which made startup treat a current 3-tier config as legacy and collide with an existing v2 backup. That init-path gap was fixed in 2aa4430c by stamping the current tier version on fresh/re-init configs, and the close-out explicitly kept rollback-backup immutability. The remaining hole is the start / launchd path: if a later rewritten config still needs startup migration and a rollback-classified .pre-openai-tiers-v2.bak is present, backupConfigBeforeOpenAiTierMigration() throws instead of preserving-then-continuing. ocx init already has cleanupOpenAiTierBackupAfterInit() (src/cli/init.ts) that copies the snapshot to config.json.pre-openai-tiers-v1-rollback.<ts>.bak and unlinks the blocking v2 path. ocx start / the LaunchAgent do not.

Expected: keep the rollback snapshot (do not silently overwrite it), but do not crash-loop the installed service. ocx start / ocx service repair should preserve-then-continue, or fail with an actionable service-status message instead of throwing inside keepalive.

Reproduction

1. Install OpenCodex 2.13.0 and register the macOS service (ocx service / com.opencodex.proxy, KeepAlive + RunAtLoad).
2. Leave an existing ~/.opencodex/config.json.pre-openai-tiers-v2.bak that parses as pre-migration / rollback (openaiProviderTierVersion: 1, defaultProvider: openai-multi).
3. Rewrite current ~/.opencodex/config.json to a smaller current-tree config that still triggers startup migration (observed: kimi-only, no openaiProviderTierVersion).
4. Let launchd start/restart the service, or run ocx start.
5. Observe OpenAiTierBackupCollisionError in ~/.opencodex/service.log, launchd last exit code 1, and port 10100 never coming up.

Operator workaround that recovered the live host (do not delete the rollback):

1. Copy the blocking backup to ~/.opencodex/config.json.pre-openai-tiers-v1-rollback.<ts>.bak.
2. Remove only the blocking .pre-openai-tiers-v2.bak path.
3. Run /opt/homebrew/bin/ocx service repair.
4. Service returned to GET http://127.0.0.1:10100/healthz → 200.

Version

2.13.0

CLI binary: /opt/homebrew/bin/ocx → @bitkyc08/opencodex 2.13.0.
LaunchAgent exec: source checkout bun.exe + src/cli/index.ts (OCX_SERVICE=1, start --port 10100).

Operating system

macOS Darwin 27.0.0 arm64 (persistent LaunchAgent install; host had been up 8 days when the service was found dead).

Provider and model

Not request-path specific. Crash happens before the proxy binds. Live config at failure time was kimi-only; colliding backup was defaultProvider=openai-multi.

Logs or error output

ocx status
❌ Proxy: not running
   Health: http://127.0.0.1:10100/healthz unreachable
   Service: installed and loaded (launchd) — registered but NOT serving
   ↳ re-run 'ocx service repair'

ocx health --json
{"ok":false,"pid":null,"port":null}

curl http://127.0.0.1:10100/healthz
Failed to connect to 127.0.0.1 port 10100: Couldn't connect to server

launchctl print gui/$UID/com.opencodex.proxy
state = spawn scheduled
last exit code = 1
runs = 19
properties = keepalive | runatload

Repeated ~/.opencodex/service.log tail:

OpenAiTierBackupCollisionError: Existing OpenAI tier backup differs from the current config
      at backupConfigBeforeOpenAiTierMigration (src/config.ts:464:15)
      at runOpenAiTierStartupMigration (src/providers/openai-tier-startup.ts:23:8)
      at startServer (src/server/index.ts:493:51)
      at handleStart (src/cli/index.ts:241:16)

After preserving the rollback and ocx service repair:

{"status":"ok","service":"opencodex","version":"2.13.0","pid":12831,"port":10100}
GET /healthz -> 200
launchd state = running

Screenshots and supporting files

None. No secrets, tokens, emails, or account identifiers attached.

Redacted configuration

Current ~/.opencodex/config.json at failure time (shape only):

{
  "port": 10100,
  "defaultProvider": "kimi",
  "googleAntigravityStaticCatalogVersion": 2,
  "providers": { "kimi": { "adapter": "openai-chat" } }
}

Size 3118 bytes. No openaiProviderTierVersion.

Colliding ~/.opencodex/config.json.pre-openai-tiers-v2.bak (shape only):

{
  "defaultProvider": "openai-multi",
  "openaiProviderTierVersion": 1,
  "providers": ["openai", "openai-multi", "anthropic", "xai", "cursor", "google-antigravity", "opencode-go", "kimi", "opencode-free"]
}

Size 25439 bytes. classifyOpenAiTierBackup() treats this as "rollback" because openaiProviderTierVersion !== 2.

Related code:

• src/config.ts classifyOpenAiTierBackup() / backupConfigBeforeOpenAiTierMigration() — rollback-classified divergence throws OpenAiTierBackupCollisionError
• src/providers/openai-tier-startup.ts — startup migration calls backup then save
• src/cli/init.ts cleanupOpenAiTierBackupAfterInit() — already preserve-then-unlink on the init path only

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.

[coderabbitai]

🔗 Related PRs

#387 - feat: ship packaged macOS menu bar companion [open]
#770 - fix: research-batch reliability and security-boundary issues [closed]
#991 - fix(service): complete the repair migration — docs, startup health, doctor, and update advice [merged]
#1160 - fix(service): preserve supervision across restart [open]
#1202 - fix(codex): stop reporting every history failure as a Codex DB lock [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[LeoWang331]

I'm working on a focused fix for #1599: preserve a rollback-classified OpenAI tier backup before retrying startup migration, with regression coverage. I'll keep this scoped to the startup migration/backup boundary and open a Draft PR as soon as the first reviewable slice is ready.

[lidge-jun]

Fixed on dev in 98bdc4d (Merge pull request #1605).

Your diagnosis held up exactly: backupConfigBeforeOpenAiTierMigration threw OpenAiTierBackupCollisionError when the current config still needed tier migration and config.json.pre-openai-tiers-v2.bak was a legitimate pre-migration rollback snapshot. That abort happened before the proxy bound its port, so a LaunchAgent with KeepAlive restarted it forever while ocx service status still reported the service as loaded.

The fix preserves rather than discards. preserveOpenAiTierRollbackSnapshot copies the rollback snapshot to a unique config.json.pre-openai-tiers-v1-rollback.<timestamp>.bak path using no-replace publication, re-reads the copy and byte-compares it against the source, and only then unlinks the blocking v2 name. Startup catches the collision, preserves, and retries the backup, so migration proceeds and the port binds. The same helper backs ocx init cleanup so the two paths cannot drift apart.

Verification on the merged dev: bun run typecheck clean, bun run test 11527 pass / 0 fail across 717 files, with new coverage in tests/openai-provider-option-startup.test.ts and tests/init-backup-cleanup.test.ts.

Closing manually — PRs here target dev, so GitHub's auto-close does not fire.