ci(bench): stream the per-shape search attribution instead of slurping it

[dougc95]

Follow-up to #1544, which is now merged into ci/bench-host-hardening (#1543). This PR's base is that branch.

Problem

"Attribute search latency per query shape" ran jq -rs over bench-results/<backend>/search-points.json. That file is k6's --out json stream, one line for every metric of every search request. -s loads the whole stream into a single in-memory array, so memory grows with the file size, not with the ~20 query shapes being summarised.

This step has died from memory exhaustion three times:

• run 30550776427;
• run 33515369645;
• run 36423973848, sqlite leg on github-agent3. The log shows 109920 Killed jq -rs after 3m47s. The dispatch was backend=all, tests=prewarm,search, so the leg searched a near-empty database: 2.2M requests in 136 s, which made a 9.17 GB (8.5 GiB), 26.3M-line search-points.json. The runner itself survived. The || fallback printed the warning, and the job then ended. ci(bench): harden shared-host cleanup in fhir-benchmark.yml #1543's teardown ordering had already stopped the containers.

In the two earlier runs, the job was cancelled in this step and every later step was skipped.

Every new leg added by #1475 runs this step, which makes it more likely to hit.

Fix

• New .github/scripts/fhir-bench/search_by_shape.py reads the file line by line and keeps only per-shape duration arrays (array('d')). It produces the same table as the jq program:
  • jq's //: null and false count as missing, while an empty string does not;
  • jq's round, half away from zero;
  • the same median, p95 and max indices;
  • sort_by(-.p95) with ties in ascending shape order, as group_by gives.
• The step calls it with the same fallback warning. It gets timeout-minutes: 15, and matrix.backend moves to env: so the run block holds no expressions.

Verification

• Equivalence with the old jq program, extracted verbatim from the workflow and run with jq 1.8.2:

  • Mixed fixture (ties, missing/null/false/empty tags, non-Point lines, other metrics, n=1/n=2, .5 values): byte-identical.
  • Empty file: identical (the header lines only).
  • With one malformed line: jq exits 5 and prints nothing. The script skips the line and prints the same table.
  • 3,000,000-line fixture (365 MB, 300k matching points across 20 shapes): byte-identical output.

  	Wall time	Peak memory
  search_by_shape.py	~5 s	~16 MB
  jq -rs	~68 s	~4.7 GB

• actionlint 1.7.12: clean.

• Live run: the same dispatch that lost the runner (-f backend=all -f tests=prewarm,search) is re-running from this branch as 36427203450. Results will be added here.

Refs #1475

🤖 Generated with Claude Code

https://claude.ai/code/session_013NudzWDu2yTGExYaxdTQYJ