Skip to content

Teardown's port-release wait probes all five fixed Harper ports unconditionally; an unrelated resident service on any one of them (here: MQTT on 1883) makes the wait unwinnable and every suite retires its address #28

Description

@deliciousmonster

teardownHarper waits on ALL_HARPER_PORTS = [9925, 9926, 9927, 1883, 8883]
(dist/harperLifecycle.js:29) on the instance's loopback address, regardless of which ports
this Harper instance actually bound. waitForPortsFree (dist/portUtils.js:36) polls a
plain bind on each port and returns only when every one succeeds. A resident service that
permanently holds one of those ports on that address makes the wait structurally unwinnable:
it is not slow, it can never return true, at any timeout.

Measured this session: this development host runs a resident MQTT broker (yeti, pid 63153)
listening on 127.0.0.1:1883 (lsof, verified live). On stock macOS the practical pool is
127.0.0.1 (127.0.0.2+ unbindable without aliases), so every harness instance runs on the
same address the broker occupies. Every single teardown then logged:

Harper ports on 127.0.0.1 still in use after teardown (5000ms); NOT recycling the address
(a Harper child outlived the kill). The slot will be reclaimed when this process exits.

including a run with the timeout raised to 60000ms; the suites themselves were green (13-15s
each). The parenthetical is a misdiagnosis on this host: no Harper child outlived anything,
and nothing in the message names which port failed, so the operator is pointed at Harper's
process tree when the cause is an unrelated broker that predates the run. Whether the
instance under test even had an MQTT listener is not established here; the probe does not
care, which is the defect. The downstream consequence (the retired address deadlocks the
rest of a 1-address-pool run) is filed separately.

Suggested fix: scope the wait to ports Harper's own children bound.

  • Cheapest: derive the port list from the config the harness itself passed to startHarper
    (it constructs --HTTP_PORT/--OPERATIONSAPI_NETWORK_PORT/--MQTT_NETWORK_PORT/... at
    dist/harperLifecycle.js:426-435, so it already knows which listeners were requested).
  • Stricter: snapshot which of the fixed ports the instance's process tree actually held at
    kill time (the harness already enumerates the tree to signal it) and wait only on those.
  • Either way, print the ports still failing the probe in the warning, and drop the
    "a Harper child outlived the kill" attribution unless a tree PID is confirmed as the
    holder.

The check's stated purpose (catch an escaped Harper child before recycling the address under
SO_REUSEPORT) survives both variants; a port this instance never bound cannot be held by its
escaped child.

Reproduction

  1. Any host where a resident service holds one fixed Harper port on the pool address. Stock
    macOS with a local MQTT broker on 127.0.0.1:1883 reproduces it as found; otherwise
    nc -l 127.0.0.1 1883 & before the run with
    HARPER_INTEGRATION_TEST_LOOPBACK_POOL_START=1.
  2. Run any suite through startHarper/teardownHarper.
  3. Observed: the suite passes; teardown always waits the full PORT_RELEASE_TIMEOUT_MS
    (5s default; 60s when overridden) and then logs the warning above; the address is retired
    every time. lsof -nP -iTCP:1883 -sTCP:LISTEN shows the holder is the resident broker,
    not a process from Harper's tree.

Measured on

Component Version
@harperfast/integration-testing 0.7.1 (dist/harperLifecycle.js:29,618-629, dist/portUtils.js:36)
harper (system under test) 5.2.1
Node v24.16.0
OS macOS 26.5.2 (arm64, Darwin 25.5.0), stock loopback
resident holder yeti pid 63153, LISTEN on 127.0.0.1:1883 (lsof, 2026-08-14)
suites datadog-agent-binary bbeb99a integration tests

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions