Carry the review notes and Bend's train-overlap flag into the published benchmark

[jonfroehlich]

Two label-side gaps left over from #21, which shipped records, native, 4096x2048 and galleries (9 splits each, 11.41 GB) to rampnet-benchmark. Both are fixable with a records-only rebuild (179 KB) plus a card re-push — no imagery moves, and export_benchmark.py adopt (added in #125) makes a partial rebuild safe.

1. The reviewer's notes do not travel with the data

verdicts.json carries a top-level review_notes block (what fought the rubric, how confident the reviewer is in their own pass) and an optional per-pano note. score_validation.py prints review_notes above the numbers on purpose — so a caveat reaches whoever quotes a precision figure. The published parquet rows carry neither, and the card points back at benchmark/ in git instead.

That is exactly backwards for the audience that matters here: someone who does load_dataset(...) and never opens the repo. benchmark/README.md has asked for this since the bundle format was written — "whoever finishes #21 should make the caveats travel with the data".

Budapest is the split this exists for: its ground truth was reviewed at LOW confidence, and nothing on the Hub says so.

2. Bend's four training-overlap panoramas are unflagged

Bend is one of the paper's three training cities. An exact-id check on 2026-07-22 found 4 of its 110 reviewed panoramas in rampnet-dataset's train/val splits:

6WC0hdAYRsSAcluKSs5iRg  9kW9cxpuj7q8DMzf-ClrQQ
DJ8Zp111zu6KnMZz-0PHgQ  VgWpqFkTwCIROvM0z-DkOw

The superseded scripts/build_benchmark_dataset.py kept those rows but flagged them train_overlap = true, so a consumer could score either way. scripts/export_benchmark.py — the exporter that actually published the dataset — has no such column, so the published records config cannot be filtered, and neither the card nor benchmark/README.md mentioned the overlap until now.

Measured 2026-08-18 (score_validation.py on the committed bundle with those 4 panoramas removed):

bend	precision	recall
all 110 reviewed panos (published)	0.954	0.758
106, overlap removed	0.956	0.753
unbiased 105 (published)	0.972	0.738
unbiased 101, overlap removed	0.976	0.731

So this is an honesty/filterability fix, not a numbers fix — every delta is well inside the Wilson intervals (±~5 pt at these counts), and no conclusion in docs/ rests on it. It is still the kind of thing a benchmark must state rather than leave for someone to discover.

Two related gaps worth closing at the same time:

• The overlap check has not been re-run since 2026-07-22, and its id list survives only as a constant in a script nothing calls.
• No other split has been checked. Only the GSV splits can overlap at all (bend, paterson, gainesville, sao_paulo — Mapillary ids are a different id space), and of those only Bend is a training city, so the expected answer is zero. That is a prediction, not a measurement.

Shape of the fix

• build_records() gains train_overlap (bool, from a committed id list rather than a script constant), review_notes (per split) and note (per pano).
• Re-run the check for all four GSV splits and commit the result — including the zeros.
• python scripts/export_benchmark.py adopt → build --benchmark benchmark → verify → push --message ..., then regenerate the card so the records column table documents the new fields.
• scripts/build_benchmark_dataset.py and tests/test_benchmark_dataset.py retire once train_overlap lives in the real exporter.

🤖 Generated with Claude Code (claude-opus-5[1m])