Flaky: auth.test.ts "releases native login pipes when the cross-platform fallback fires"

[rohanpoudel2]

Summary

tests-ts/auth.test.ts — "releases native login pipes when the cross-platform fallback fires" failed once during a full-suite run and passed on every rerun afterwards, both in isolation and in the full suite. Filing it because an intermittent failure in the suite is worth a look before it lands in CI, not because I can reproduce it on demand.

Environment

• @openai/codex-security 0.1.5, on a branch based on main at ea19f24
• macOS 15 (Darwin 25.5.0), Bun 1.3.14, Node.js 24.11.1
• bun test --timeout 30000 ./tests-ts

Observed

One failure in one full-suite run. Immediately rerunning the file alone passed, and three subsequent full-suite runs passed. I did not capture the assertion output before the rerun overwrote it, which is a gap in this report.

Why it is plausibly real rather than machine noise

The test name describes releasing OS pipes when a fallback path fires, which is the shape of thing that tends to be sensitive to process/descriptor timing rather than to logic. Bun runs test files in one process, so descriptor pressure or ordering from a neighbouring file is a candidate. Two suite features make that worth checking specifically:

• other tests manipulate process.umask and restore it in finally, so they share mutable process state
• other tests spawn subprocesses and pack tarballs, so descriptor counts vary between runs

Suggested direction

Run the file under repetition and under load to see whether it is order-dependent or descriptor-dependent, e.g. a loop of full-suite runs, and separately the file alone with an artificially lowered descriptor limit. If it is a genuine race in the fallback's cleanup, the fix belongs in the code under test rather than the assertion. If it turns out to be a test-only ordering assumption, an explicit wait on the release rather than a timing-dependent assertion would settle it.

Happy to dig into it if a maintainer confirms it is worth chasing — I did not want to change a passing test on the strength of a single unreproduced failure.

[Souptik96]

Ran the repetition experiment you suggested, on a different OS/runtime. No reproduction, but with a caveat that weakens the result — flagging both.

What I ran

Windows 11, bun 1.3.13, main @ ea19f24, deps via bun install:

Run	Result
auth.test.ts alone	16 pass, 1 skip, 0 fail
-t 'releases native login pipes', 20 repetitions	0/20 failures
Full ./tests-ts, 3 runs	the named test passed in all 3

The caveat

My full-suite runs are not a clean baseline: 77 tests failed deterministically — the same 77, identically, in all three runs (656 pass / 77 fail each time). They look environmental rather than new:

• symlink tests (rejects a repository-controlled output symlink…, …npm-style bin when Node preserves main symlinks) — Windows symlink creation needs privilege
• the GitHub release workflow safeguards block — consistent with Windows release tests invoke WSL bash instead of Git Bash when WSL is first on PATH #206's bash-resolution problem; I have WSL present but with no distro installed
• Node 20.18.1 locally vs engines: ^22.13.0 || ^24 || ^26

Since Bun runs test files in one process, those failures alter exactly the descriptor pressure and ordering that your hypothesis points at. So "did not reproduce here" is weak evidence — a clean suite on macOS/Linux is where a repetition loop would actually be informative.

What would settle it

The cheapest decisive run is in your own CI: the same file under -t in a loop, plus one full-suite loop, on a runner where the suite is otherwise green. If it never fires there either, the single observed failure was probably descriptor pressure from a neighbouring file rather than a race in the fallback's cleanup.

I did not touch the test — agreed with your instinct not to change a passing assertion on one unreproduced failure.