Description
While verifying PR #284 (stdio drain-on-EOF fix), the CI `Test (Python 3.12)`
job's "Check advertised test count" step failed because `pytest --collect-only -q`
on the CI runner (ubuntu-latest, `pip install --require-hashes -r
requirements/ci-postgresql.txt`) collects 6582 tests, while the same
command against the locally-reused `.venv` (macOS, built earlier via `uv
sync`) collects only 6572 — a delta of exactly 10.
Confirmed this is not a Python-version artifact: the `Test (Python 3.13)`
leg on the same CI run reports `6577 passed, 5 skipped` = 6582 total,
matching the 3.12 leg's collected count exactly. So the divergence is
between "CI's Linux + hash-pinned `requirements/ci-postgresql.txt`
environment" and "this local macOS dev venv", not between Python versions.
Ruled out during investigation:
- Setting `CI=true DATABASE_URL=postgresql://cortex:cortex@localhost:5432/cortex`
(CI's exact env vars) locally does NOT change the local count (still 6572) —
so it is not gated on those variables.
- `psycopg` (3.3.3) and `sqlite-vec` (0.1.9) are both installed locally, and
`sqlite3.Connection.enable_load_extension` works on the local interpreter
(uv-managed `cpython-3.12.12-macos-aarch64-none`) — so the "macOS lacks
extension-capable Python" pattern (documented in an earlier lesson) does not
directly apply to this interpreter.
- A likely lead not yet chased to ground: `pgvector` is 0.4.2 locally vs.
the CHANGELOG's stated lock-pinned 0.5.0 ("the version whose psycopg
loader change `pg_store.vector_to_bytes` is written for") — the local venv
was built at an earlier point in time and may have drifted from `uv.lock`.
Several `tests_py/infrastructure/test_pg*.py` files use
`pytest.importorskip`, which (unlike a runtime `skip`) removes a module's
tests from `--collect-only`'s count entirely on import failure — a
plausible mechanism for a "same code, different collected count" gap, but
not confirmed against this exact version delta.
Reproduction
```
Local (macOS, reused venv):
cd Cortex && pytest --collect-only -q | tail -1
=> 6572 tests collected
CI (ubuntu-latest, Test (Python 3.12) job, run 30521851911):
=> "collected 6582 tests" (Check advertised test count step)
```
Acceptance criteria
Non-blocking note
PR #284 works around this by using CI's own verified collected count (6582)
directly, since CI is the environment the doc-claims gate is actually
enforcing parity with; this issue tracks the underlying local/CI parity gap
itself, which pre-dates that PR and is outside its blast radius.
Description
While verifying PR #284 (stdio drain-on-EOF fix), the CI `Test (Python 3.12)`
job's "Check advertised test count" step failed because `pytest --collect-only -q`
on the CI runner (ubuntu-latest, `pip install --require-hashes -r
requirements/ci-postgresql.txt`) collects 6582 tests, while the same
command against the locally-reused `.venv` (macOS, built earlier via `uv
sync`) collects only 6572 — a delta of exactly 10.
Confirmed this is not a Python-version artifact: the `Test (Python 3.13)`
leg on the same CI run reports `6577 passed, 5 skipped` = 6582 total,
matching the 3.12 leg's collected count exactly. So the divergence is
between "CI's Linux + hash-pinned `requirements/ci-postgresql.txt`
environment" and "this local macOS dev venv", not between Python versions.
Ruled out during investigation:
(CI's exact env vars) locally does NOT change the local count (still 6572) —
so it is not gated on those variables.
`sqlite3.Connection.enable_load_extension` works on the local interpreter
(uv-managed `cpython-3.12.12-macos-aarch64-none`) — so the "macOS lacks
extension-capable Python" pattern (documented in an earlier lesson) does not
directly apply to this interpreter.
the CHANGELOG's stated lock-pinned 0.5.0 ("the version whose psycopg
loader change `pg_store.vector_to_bytes` is written for") — the local venv
was built at an earlier point in time and may have drifted from `uv.lock`.
Several `tests_py/infrastructure/test_pg*.py` files use
`pytest.importorskip`, which (unlike a runtime `skip`) removes a module's
tests from `--collect-only`'s count entirely on import failure — a
plausible mechanism for a "same code, different collected count" gap, but
not confirmed against this exact version delta.
Reproduction
```
Local (macOS, reused venv):
cd Cortex && pytest --collect-only -q | tail -1
=> 6572 tests collected
CI (ubuntu-latest, Test (Python 3.12) job, run 30521851911):
=> "collected 6582 tests" (Check advertised test count step)
```
Acceptance criteria
macOS venv does not (or vice versa if the direction is reversed).
0.5.0), a different missing/extra package, or a platform-conditional
collection path.
guarantee parity with `requirements/ci-postgresql.txt`, or fix the
test-collection gate so it is not silently sensitive to this kind of
drift (e.g. an explicit test asserting the two known extras produce the
same collected set, mirroring the existing pyright-env-parity test).
Non-blocking note
PR #284 works around this by using CI's own verified collected count (6582)
directly, since CI is the environment the doc-claims gate is actually
enforcing parity with; this issue tracks the underlying local/CI parity gap
itself, which pre-dates that PR and is outside its blast radius.