teardownHarper waits DEFAULT_PORT_RELEASE_TIMEOUT_MS (default 5s) for all fixed Harper
ports to free, and on expiry it does not release the loopback address
(dist/harperLifecycle.js:618-629 in 0.7.1): the slot stays parked under the current
process's PID and the warning says "The slot will be reclaimed when this process exits."
There is no later re-check. Retirement is permanent for the process lifetime, however briefly
the ports were actually held.
That interacts badly with two facts:
- Under
--isolation=none (harper-integration-test-run) every suite in the run shares one
process, so "reclaimed when this process exits" means "never, for the rest of this run".
- On stock macOS only 127.0.0.1 is bindable (127.0.0.2+ need aliases), so a practical local
pool is one address (HARPER_INTEGRATION_TEST_LOOPBACK_POOL_START=1, COUNT=1).
After one retirement, the next suite's startHarper enters
getNextAvailableLoopbackAddress and spins forever: the free-slot scan finds nothing and
removeDeadProcessesFromPool (dist/loopbackAddressPool.js:414-425) cannot reclaim a slot
whose holder PID is the caller's own live process. The wait is unbounded and silent.
Measured sequence, one macOS host, 2026-08-14, three consecutive
npm run test:integration invocations of the same two-file suite:
| Run (local time) |
PORT_RELEASE_TIMEOUT_MS |
Suite 1 |
Then |
| 15:42 |
5000 (default) |
green, 15.2s total |
wait expired, warning logged, address retired |
| 15:53 |
5000 (default) |
green, 13.4s total |
same |
| 16:05 |
60000 (override) |
green, 68.1s total |
60s wait also expired; suite 2 never acquired an address; the harness had to be killed externally and $TMPDIR/harper-integration-test-loopback-pool.json reset by hand |
Raising the timeout 12x changed only the stall length, which separates the two defects: why
the ports were held is its own issue (on this host an unrelated resident MQTT broker held
1883, filed separately), but the one-shot retirement is what turns any expiry into a wedged
run. In the case the code comments actually contemplate (a slow Harper child that exits a few
seconds after the deadline), the ports free and the address stays retired anyway; that case
follows directly from the code path (no re-check exists) and is expected rather than
separately measured here.
A second, related defect: the pool persists holder PIDs in
$TMPDIR/harper-integration-test-loopback-pool.json across processes
(dist/loopbackAddressPool.js:23). A run killed with SIGKILL leaves its PID parked; after
the 16:05 run was killed, the pool file still carried that run's slot state and was reset to
[] by hand before anything else ran (measured). What happens without that reset is a code
read, not a measurement: reclaim runs only when the pool is exhausted and tests liveness
with process.kill(pid, 0), so a dead run's slot is reclaimed eventually, unless the OS has
reused the PID for any live process, in which case the slot stays parked indefinitely and
later runs are poisoned by a run that no longer exists.
Suggested fixes, independently useful:
- On each acquisition attempt inside the same process, re-probe addresses this process
previously retired instead of skipping them for the process lifetime. The conflict canary
in getNextAvailableLoopbackAddress already exists to make handing out a still-held
address safe; a freed address should return to service on the next probe.
- Stamp pool entries with the holder's process start time (or a timestamp plus a nonce) so
removeDeadProcessesFromPool can distinguish a dead run from a reused PID.
- Name the still-bound ports in the teardown warning (today it prints none), so an operator
can see in one line whether the holder is a Harper child or an unrelated service.
Reproduction
- macOS with no loopback aliases;
HARPER_INTEGRATION_TEST_LOOPBACK_POOL_START=1,
HARPER_INTEGRATION_TEST_LOOPBACK_POOL_COUNT=1.
- Ensure any listener occupies one fixed Harper port on 127.0.0.1 for the duration (e.g.
nc -l 127.0.0.1 1883 &), so the port-release wait must expire.
- Run two Harper suites via
harper-integration-test-run --isolation=none.
- Observed: suite 1 passes; teardown logs
Harper ports on 127.0.0.1 still in use after teardown (5000ms); NOT recycling the address;
suite 2 blocks forever in getNextAvailableLoopbackAddress with no output. Kill the run;
the pool JSON still holds the dead PID.
Measured on
| Component |
Version |
| @harperfast/integration-testing |
0.7.1 (dist/harperLifecycle.js, dist/loopbackAddressPool.js) |
| harper (system under test) |
5.2.1 |
| Node |
v24.16.0 |
| OS |
macOS 26.5.2 (arm64, Darwin 25.5.0), stock loopback (127.0.0.1 only) |
| suites |
datadog-agent-binary bbeb99a integration tests, --isolation=none |
teardownHarperwaitsDEFAULT_PORT_RELEASE_TIMEOUT_MS(default 5s) for all fixed Harperports to free, and on expiry it does not release the loopback address
(
dist/harperLifecycle.js:618-629in 0.7.1): the slot stays parked under the currentprocess's PID and the warning says "The slot will be reclaimed when this process exits."
There is no later re-check. Retirement is permanent for the process lifetime, however briefly
the ports were actually held.
That interacts badly with two facts:
--isolation=none(harper-integration-test-run) every suite in the run shares oneprocess, so "reclaimed when this process exits" means "never, for the rest of this run".
pool is one address (
HARPER_INTEGRATION_TEST_LOOPBACK_POOL_START=1,COUNT=1).After one retirement, the next suite's
startHarperentersgetNextAvailableLoopbackAddressand spins forever: the free-slot scan finds nothing andremoveDeadProcessesFromPool(dist/loopbackAddressPool.js:414-425) cannot reclaim a slotwhose holder PID is the caller's own live process. The wait is unbounded and silent.
Measured sequence, one macOS host, 2026-08-14, three consecutive
npm run test:integrationinvocations of the same two-file suite:PORT_RELEASE_TIMEOUT_MS$TMPDIR/harper-integration-test-loopback-pool.jsonreset by handRaising the timeout 12x changed only the stall length, which separates the two defects: why
the ports were held is its own issue (on this host an unrelated resident MQTT broker held
1883, filed separately), but the one-shot retirement is what turns any expiry into a wedged
run. In the case the code comments actually contemplate (a slow Harper child that exits a few
seconds after the deadline), the ports free and the address stays retired anyway; that case
follows directly from the code path (no re-check exists) and is expected rather than
separately measured here.
A second, related defect: the pool persists holder PIDs in
$TMPDIR/harper-integration-test-loopback-pool.jsonacross processes(
dist/loopbackAddressPool.js:23). A run killed with SIGKILL leaves its PID parked; afterthe 16:05 run was killed, the pool file still carried that run's slot state and was reset to
[]by hand before anything else ran (measured). What happens without that reset is a coderead, not a measurement: reclaim runs only when the pool is exhausted and tests liveness
with
process.kill(pid, 0), so a dead run's slot is reclaimed eventually, unless the OS hasreused the PID for any live process, in which case the slot stays parked indefinitely and
later runs are poisoned by a run that no longer exists.
Suggested fixes, independently useful:
previously retired instead of skipping them for the process lifetime. The conflict canary
in
getNextAvailableLoopbackAddressalready exists to make handing out a still-heldaddress safe; a freed address should return to service on the next probe.
removeDeadProcessesFromPoolcan distinguish a dead run from a reused PID.can see in one line whether the holder is a Harper child or an unrelated service.
Reproduction
HARPER_INTEGRATION_TEST_LOOPBACK_POOL_START=1,HARPER_INTEGRATION_TEST_LOOPBACK_POOL_COUNT=1.nc -l 127.0.0.1 1883 &), so the port-release wait must expire.harper-integration-test-run --isolation=none.Harper ports on 127.0.0.1 still in use after teardown (5000ms); NOT recycling the address;suite 2 blocks forever in
getNextAvailableLoopbackAddresswith no output. Kill the run;the pool JSON still holds the dead PID.
Measured on
dist/harperLifecycle.js,dist/loopbackAddressPool.js)bbeb99aintegration tests,--isolation=none