You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The 360° seam is unsealed end to end. Four layers of the same defect, each one hiding the next:
Stage 1's peak extraction does not wrap → seam ramps get double-labelled in the published training data → the model is trained on defective supervision at the seam → the model finds less than half the ramps that sit on it → and the scorer cannot see any of this, because the matcher does not wrap either and manual_gold's GT double-marks the same ramps in the same place.
Every number below is derived from committed caches and committed labels, or from the published HF dataset's label columns. No new imagery, no GPU. Reproduction snippets are at the bottom.
This grew out of #130, which covers the scoring-side half (the 11 double-marked GT pairs and the non-wrapping matcher). #130 stays as-is and remains the fix ticket for the scorer. This issue is the whole chain, including two things #130 does not contain: a measured detection deficit in the model, and the same defect in the published Stage 1 dataset.
0. What is measured and what is not
Stated up front, because the interesting claims here are of different strengths and a reviewer should not have to reverse-engineer which is which.
Measured, with the confounds controlled:
The model's recall inside ~4° of the seam is about half what it is elsewhere (§2).
The published Stage 1 training dataset contains seam-crossing duplicate label pairs far above chance (§3).
Which code paths wrap and which do not (§4) — read off the source.
Hypothesis, not yet tested:
That the recall deficit is caused by receptive-field truncation at the image border. The evidence is consistent with it (the deficit is confined to a narrow border strip and is flat everywhere else) but the causal test has not been run. §7 is that test.
That defective Stage 1 supervision at the seam contributes to the deficit. Plausible, untested, and separable from the above.
The "dropout" mode in §3 — seam ramps getting no label rather than two. Suggested by the azimuth histogram, but inside natural variation. Not established.
Explicitly not claimed: that any conclusion in the model comparison, the YOLO baseline, or the operating-point work changes. The aggregate effect is about 0.17 recall points. The reason to care is that it is systematic, permanent, at a known location, and it sits in a published dataset.
1. Why this was invisible
The four layers cancel. In manual_gold, a seam ramp is labelled twice — once per edge. The model, if it fires at all, emits one detection. The non-wrapping matcher scores that as one TP and one unclaimable FN. Fix the GT alone and the answer depends on which duplicate you delete; fix the matcher alone and nothing moves, because the near-side duplicate always gives the detection something to claim. The two only become well-defined together, which is the argument in #130.
And the deficit itself was hidden a second way: scripts/gt_gallery.py, the point-adjudication viewer every verdict split was reviewed in, clamps its crop window instead of wrapping it (left = int(min(max(px - half, 0), sx - CROP_SIZE)), line 237). A seam ramp's two halves land in separate 512 px crops taken 3,584 px apart. A reviewer had no way to see them as one object. The nine verdict-reviewed splits hold only 4 GT points within 11 px of the seam, against 18 in the independently-labelled manual_gold — consistent with reviewers under-marking exactly the region the viewer renders badly.
2. Finding 1 — the model loses about half the ramps on the seam
RampNet at the deployed 0.30, pooled over all ten benchmark splits, GT binned by distance to the nearest seam edge.
Three corrections applied first, so this measures the model and not our own bugs:
The matcher wraps. Otherwise a detection on one side of the seam cannot claim a GT point on the other.
Seam-crossing duplicate GT pairs are merged (second member dropped). Only seam-crossing pairs — genuinely adjacent ramps away from the seam are left in place, which is the conservative choice: merging them too would strip hard cases out of the baseline and inflate the deficit.
Range is controlled by direct standardization on cy. Seam-band GT sits at systematically lower cy (further away) than the rest of the pano, and recall falls off with range independently of anything to do with the seam. exp below is what a population with this band's own range profile would score at baseline rates.
distance from seam
GT
found
observed recall
expected (range-standardized)
Δ
z
0–11 px (≤3.9° of azimuth)
25
8
0.3200
0.7757
−0.4557
−5.67
11–22 px
65
49
0.7538
0.7650
−0.0111
−0.22
22–45 px
270
213
0.7889
0.7714
+0.0175
+0.71
45–90 px
729
602
0.8258
0.8168
+0.0090
+0.65
The deficit is confined entirely to the innermost band. Every other band is flat (|z| < 1). That shape is the point: a broad gradient would suggest range or occlusion; a sharp step at the border is what an edge artifact looks like.
The circularity objection, and the split that answers it
GT on the nine verdict splits is built from model detections a reviewer judged, plus misses the reviewer added. Recall measured against it is therefore partly definitional. manual_gold is not — its GT comes from the independent 1,000-pano YOLO labelling pass, which never saw model output. Split by provenance:
GT provenance
band
GT
found
obs recall
exp recall
Δ
z
manual_gold (independent)
0–11 px
19
7
0.3684
0.8488
−0.4804
−5.88
manual_gold
11–22 px
23
17
0.7391
0.8051
−0.0660
−0.82
nine verdict splits
0–11 px
4
1
0.2500
0.7406
−0.4906
−2.29
Same magnitude both ways, and it is strongest on the independent GT. The verdict splits agree in direction and size but carry almost no weight at n=4.
Note also which way the remaining bias runs: if reviewers under-marked seam ramps through the clamping viewer, those ramps are absent from GT and cannot be counted as misses. That inflates measured seam recall. The deficit is measured against a bias that works to hide it.
Size of the prize
About 25 GT points of 6,549 sit in the affected band, and roughly 11 of them are lost — ~0.17 recall points in aggregate. Small. It is worth fixing anyway because it is not noise: it is a fixed, reproducible blind spot at a known azimuth in every panorama the model has ever run on, and it is one of the few error sources here with a deterministic fix rather than a data-collection fix.
3. Finding 2 — the published Stage 1 dataset double-labels seam ramps
stage_one/dataset_generation/download_dataset.py builds a combined 4096×2048 equirectangular heatmap by projecting each per-azimuth crop-model heatmap back with perspective_to_equirectangular (line 57), then extracts labels with:
peak_local_max treats column 0 and column 4095 as maximally distant. On a cyclic image they are adjacent. So a ramp on the seam can emit a peak on each edge, and min_distance=40 will never suppress one against the other. That is the code that generated projectsidewalk/rampnet-dataset.
Scanning the published parquet label columns directly — curb_ramp_points_normalized is 0.003 MB per row group against 203 MB for image, so column projection makes this a ~12 MB read rather than 463 GB:
panos
labels
seam-crossing label pairs within R
expected under uniform-azimuth null
enrichment
first 8 of 384 files
1,344
5,312
60
3.08
19.5×
first 144 of 384 files
40,209
156,027
1,543
88.1
17.5×
p ≈ 0 at both sample sizes. The full 384-file scan is running; final numbers will be posted as a comment on this issue and this table updated.
That is roughly 1% of the training labels being two marks on one physical ramp — each one becoming two Gaussian targets 40+ px apart in the Stage 2 training objective.
The unproven half. The azimuth histogram also shows the two seam-adjacent bins low (0.66× and 0.69× of the mean), which would be the opposite failure — a split response falling under threshold_abs and producing no label at all. But interior bins range 24–133 against a mean of 83, so that deficit sits inside natural azimuth variation. GT azimuth is genuinely non-uniform in this data (χ² = 193 on 15 df in the benchmark GT — US grid cities put roads at predictable bearings relative to north), so a histogram dip is not evidence on its own. Doubling is established. Dropout is not.
4. Finding 3 — the audit: every site that touches cyclic pano geometry
Must wrap, currently does not:
site
what it is
measured cost today
rampnet/metrics.py:35
greedy_match — the shared matching core every evaluator uses
On the extraction row: across all committed caches at the 0.05 floor there are 0 seam-crossing prediction pairs against 1.58 expected, so there is no evidence the model currently double-peaks at the seam. But P(observe 0 | null) = 0.21, so that is underpowered — "no measured cost", not "proven absent". It also cannot be cleanly tested while the model under-detects there at all.
Already correct, no change needed:scripts/analysis/miss_taxonomy.py:120, scripts/analysis/silent_activation.py (all three sites — heat[row][col % W], nearest_peak, null_azimuths), scripts/model_comparison/equirect_tiling.py:167 (dedup_points), scripts/analysis/crop_window_eval.py:1013, scripts/box_gallery.py:211, and scripts/analysis/miss_gallery.py (reprojects through equirect_tiling.View, so it is structurally immune).
Worth noting that equirect_tiling.dedup_points has had the correct wrapping distance all along. The repo already contains the right code; the scorer just does not call it.
5. The trap in the fix: one caller must NOT wrap
stage_one/crop_model/ps_and_manual_model/evaluate.py:206 calls the same match_predictions with scale_x=341/4. That is crop space (1024×352 crops), where x is genuinely not cyclic. An unconditional wrap would silently corrupt the crop model's numbers.
So the fix is an explicit wrap_x parameter that pano callers opt into, not a blanket change:
This needs a regression test pinning the crop path as not wrapping, or a future consistency cleanup re-breaks it.
6. How to seal it
Three independent layers. The order matters because the first is the measuring instrument for the other two.
Layer 1 — scoring. Wrapped matching plus wrapped GT dedup (#130). Changes nothing user-facing; it is what makes layers 2 and 3 measurable at all.
Layer 2 — extraction. Wrapped NMS: merge peaks across the seam in peaks_to_dets / extract_peaks_from_heatmap, and in the Stage 1 label generator. Cheap, no retrain, kills the doubling including in any regenerated training data. Does not fix truncated context.
Layer 3 — the model. This is the actual seal. The convolutions pad the left and right borders with zeros, so a ramp on the seam is processed with half its context replaced by nothing. Three options, in cost order:
option
retrain?
cost
notes
Circular input pre-padding — wrap-pad the 4096-wide input by m columns from the opposite edge, run, crop the heatmap back
no
~2m/4096 (m=256 → +12.5% inference)
no weight change, no architecture change; the published checkpoint works as-is
Roll-by-180° TTA — run twice, second pass on a rolled pano, combine
no
2×
complete context everywhere; also the diagnostic, see §7
Circular padding in the convs, then retrain
yes
a training run
correct by construction; the RampNet 2.0 answer
This is not the flip-TTA that #78 measured and rejected. That was generic augmentation with no target and it lost on 4 of 5 US splits. This is aimed at a specific measured defect at a known location, and it has to be measured on its own terms rather than inheriting #78's result.
7. The decisive experiment
Roll-by-180° is both a candidate fix and the causal test, which is why it is worth running first.
Take the panoramas holding the ~25 seam-band GT points, roll each by half its width, re-run the published checkpoint, roll the heatmap back, and score the same GT.
The lost ramps come back → context truncation confirmed, and circular pre-padding is the cheap production fix.
They do not → the range standardization in §2 missed a confound, and the deficit needs a different explanation before anyone builds on it.
Minutes on one GPU, against panoramas already in benchmark/*/panos. Nothing downstream should be built on §2 until this has run.
8. What this touches
The published dataset.projectsidewalk/rampnet-dataset carries the duplicate labels. Fixing means either regenerating with wrapped extraction or shipping a documented erratum.
The paper. An amendment covering the Stage 1 label defect and the seam recall deficit. Aggregate effect on the headline numbers is small (~0.17 recall points), but the dataset defect is a property of a published artifact and should be on the record.
Full Stage 1 scan result posted here, and a decision on regeneration vs erratum
Guard at GT ingest: surface within-radius wrapped pairs for a human decision, never auto-merge (see §10)
10. What I would most like challenged
Listed because they are the weakest links, not as a formality.
n = 25 in the affected band. z = −5.67 is decisive against the null, but it is 25 points. Is direct standardization on cy adequate, or does the seam band differ on something else — occlusion, vehicle position, hood/nadir proximity — that cy does not capture?
§3's dropout claim is unproven and I have said so. If someone can think of a way to separate a real label deficit from azimuth non-uniformity in the published dataset, that would settle it.
The auto-merge question.360 seam, in the scoring code: 11 double-marked ramps in manual_gold, and a matcher that does not wrap #130's guard proposal was to merge GT points within the wrapped radius. Measuring the base rate says do not: manual_gold holds 234 within-radius GT pairs away from the seam, 87 of them with near-identical cy at the horizon — genuinely adjacent far-field ramps. The uniform-azimuth null predicts 3.16 of the 11 seam pairs arise by chance. So roughly 3 of the 11 may be real pairs, and auto-merge would silently delete real ramps in the direction that flatters our recall. Hence the blind adjudication deck rather than a code rule. Is that the right call?
Refs #130 (the scoring-side fix), #43 (the reviewer rule and the GT viewer), #51 (the baseline comparison that reads manual_gold), #46 (where this surfaced), #48 (multi-view, which inherits the blind spot), #78 (flip-TTA, which this is not), #101 (the range-proxy caveat in §10).
The 360° seam is unsealed end to end. Four layers of the same defect, each one hiding the next:
Stage 1's peak extraction does not wrap → seam ramps get double-labelled in the published training data → the model is trained on defective supervision at the seam → the model finds less than half the ramps that sit on it → and the scorer cannot see any of this, because the matcher does not wrap either and
manual_gold's GT double-marks the same ramps in the same place.Every number below is derived from committed caches and committed labels, or from the published HF dataset's label columns. No new imagery, no GPU. Reproduction snippets are at the bottom.
This grew out of #130, which covers the scoring-side half (the 11 double-marked GT pairs and the non-wrapping matcher). #130 stays as-is and remains the fix ticket for the scorer. This issue is the whole chain, including two things #130 does not contain: a measured detection deficit in the model, and the same defect in the published Stage 1 dataset.
0. What is measured and what is not
Stated up front, because the interesting claims here are of different strengths and a reviewer should not have to reverse-engineer which is which.
Measured, with the confounds controlled:
Hypothesis, not yet tested:
Explicitly not claimed: that any conclusion in the model comparison, the YOLO baseline, or the operating-point work changes. The aggregate effect is about 0.17 recall points. The reason to care is that it is systematic, permanent, at a known location, and it sits in a published dataset.
1. Why this was invisible
The four layers cancel. In
manual_gold, a seam ramp is labelled twice — once per edge. The model, if it fires at all, emits one detection. The non-wrapping matcher scores that as one TP and one unclaimable FN. Fix the GT alone and the answer depends on which duplicate you delete; fix the matcher alone and nothing moves, because the near-side duplicate always gives the detection something to claim. The two only become well-defined together, which is the argument in #130.And the deficit itself was hidden a second way:
scripts/gt_gallery.py, the point-adjudication viewer every verdict split was reviewed in, clamps its crop window instead of wrapping it (left = int(min(max(px - half, 0), sx - CROP_SIZE)), line 237). A seam ramp's two halves land in separate 512 px crops taken 3,584 px apart. A reviewer had no way to see them as one object. The nine verdict-reviewed splits hold only 4 GT points within 11 px of the seam, against 18 in the independently-labelledmanual_gold— consistent with reviewers under-marking exactly the region the viewer renders badly.2. Finding 1 — the model loses about half the ramps on the seam
RampNet at the deployed 0.30, pooled over all ten benchmark splits, GT binned by distance to the nearest seam edge.
Three corrections applied first, so this measures the model and not our own bugs:
cy. Seam-band GT sits at systematically lowercy(further away) than the rest of the pano, and recall falls off with range independently of anything to do with the seam.expbelow is what a population with this band's own range profile would score at baseline rates.The deficit is confined entirely to the innermost band. Every other band is flat (|z| < 1). That shape is the point: a broad gradient would suggest range or occlusion; a sharp step at the border is what an edge artifact looks like.
The circularity objection, and the split that answers it
GT on the nine verdict splits is built from model detections a reviewer judged, plus misses the reviewer added. Recall measured against it is therefore partly definitional.
manual_goldis not — its GT comes from the independent 1,000-pano YOLO labelling pass, which never saw model output. Split by provenance:manual_gold(independent)manual_goldSame magnitude both ways, and it is strongest on the independent GT. The verdict splits agree in direction and size but carry almost no weight at n=4.
Note also which way the remaining bias runs: if reviewers under-marked seam ramps through the clamping viewer, those ramps are absent from GT and cannot be counted as misses. That inflates measured seam recall. The deficit is measured against a bias that works to hide it.
Size of the prize
About 25 GT points of 6,549 sit in the affected band, and roughly 11 of them are lost — ~0.17 recall points in aggregate. Small. It is worth fixing anyway because it is not noise: it is a fixed, reproducible blind spot at a known azimuth in every panorama the model has ever run on, and it is one of the few error sources here with a deterministic fix rather than a data-collection fix.
3. Finding 2 — the published Stage 1 dataset double-labels seam ramps
stage_one/dataset_generation/download_dataset.pybuilds a combined 4096×2048 equirectangular heatmap by projecting each per-azimuth crop-model heatmap back withperspective_to_equirectangular(line 57), then extracts labels with:peak_local_maxtreats column 0 and column 4095 as maximally distant. On a cyclic image they are adjacent. So a ramp on the seam can emit a peak on each edge, andmin_distance=40will never suppress one against the other. That is the code that generatedprojectsidewalk/rampnet-dataset.Scanning the published parquet label columns directly —
curb_ramp_points_normalizedis 0.003 MB per row group against 203 MB forimage, so column projection makes this a ~12 MB read rather than 463 GB:p ≈ 0 at both sample sizes. The full 384-file scan is running; final numbers will be posted as a comment on this issue and this table updated.
That is roughly 1% of the training labels being two marks on one physical ramp — each one becoming two Gaussian targets 40+ px apart in the Stage 2 training objective.
The unproven half. The azimuth histogram also shows the two seam-adjacent bins low (0.66× and 0.69× of the mean), which would be the opposite failure — a split response falling under
threshold_absand producing no label at all. But interior bins range 24–133 against a mean of 83, so that deficit sits inside natural azimuth variation. GT azimuth is genuinely non-uniform in this data (χ² = 193 on 15 df in the benchmark GT — US grid cities put roads at predictable bearings relative to north), so a histogram dip is not evidence on its own. Doubling is established. Dropout is not.4. Finding 3 — the audit: every site that touches cyclic pano geometry
Must wrap, currently does not:
rampnet/metrics.py:35greedy_match— the shared matching core every evaluator usesrampnet/detection_eval.py:224scripts/gt_gallery.py:237scripts/analysis/operating_point_curve.py:642scripts/analysis/size_analysis.py:128stage_two/evaluate.py:78,stage_two/demo.py:110,scripts/analysis/threshold_sweep.py:97peak_local_maxon the pano heatmapstage_one/dataset_generation/download_dataset.py:64On the extraction row: across all committed caches at the 0.05 floor there are 0 seam-crossing prediction pairs against 1.58 expected, so there is no evidence the model currently double-peaks at the seam. But P(observe 0 | null) = 0.21, so that is underpowered — "no measured cost", not "proven absent". It also cannot be cleanly tested while the model under-detects there at all.
Already correct, no change needed:
scripts/analysis/miss_taxonomy.py:120,scripts/analysis/silent_activation.py(all three sites —heat[row][col % W],nearest_peak,null_azimuths),scripts/model_comparison/equirect_tiling.py:167(dedup_points),scripts/analysis/crop_window_eval.py:1013,scripts/box_gallery.py:211, andscripts/analysis/miss_gallery.py(reprojects throughequirect_tiling.View, so it is structurally immune).Worth noting that
equirect_tiling.dedup_pointshas had the correct wrapping distance all along. The repo already contains the right code; the scorer just does not call it.5. The trap in the fix: one caller must NOT wrap
stage_one/crop_model/ps_and_manual_model/evaluate.py:206calls the samematch_predictionswithscale_x=341/4. That is crop space (1024×352 crops), where x is genuinely not cyclic. An unconditional wrap would silently corrupt the crop model's numbers.So the fix is an explicit
wrap_xparameter that pano callers opt into, not a blanket change:rampnet/detection_eval.py::score_panostage_two/evaluate.py:231(scale_x=heatmap_w)stage_one/dataset_evaluation/evaluate.py:132(PANO_SCALE_X)stage_one/crop_model/ps_and_manual_model/evaluate.py:202(scale_x=341/4)This needs a regression test pinning the crop path as not wrapping, or a future consistency cleanup re-breaks it.
6. How to seal it
Three independent layers. The order matters because the first is the measuring instrument for the other two.
Layer 1 — scoring. Wrapped matching plus wrapped GT dedup (#130). Changes nothing user-facing; it is what makes layers 2 and 3 measurable at all.
Layer 2 — extraction. Wrapped NMS: merge peaks across the seam in
peaks_to_dets/extract_peaks_from_heatmap, and in the Stage 1 label generator. Cheap, no retrain, kills the doubling including in any regenerated training data. Does not fix truncated context.Layer 3 — the model. This is the actual seal. The convolutions pad the left and right borders with zeros, so a ramp on the seam is processed with half its context replaced by nothing. Three options, in cost order:
This is not the flip-TTA that #78 measured and rejected. That was generic augmentation with no target and it lost on 4 of 5 US splits. This is aimed at a specific measured defect at a known location, and it has to be measured on its own terms rather than inheriting #78's result.
7. The decisive experiment
Roll-by-180° is both a candidate fix and the causal test, which is why it is worth running first.
Take the panoramas holding the ~25 seam-band GT points, roll each by half its width, re-run the published checkpoint, roll the heatmap back, and score the same GT.
Minutes on one GPU, against panoramas already in
benchmark/*/panos. Nothing downstream should be built on §2 until this has run.8. What this touches
projectsidewalk/rampnet-datasetcarries the duplicate labels. Fixing means either regenerating with wrapped extraction or shipping a documented erratum.manual_gold's row wherever it is quoted, including the RampNet-vs-YOLO comparison in Train a supervised YOLO baseline (YOLO11 / YOLO26) on the RampNet dataset — isolate architecture vs. data #51.9. Plan
rampnet/geometry.py— one wrapping distance and one wrapping crop box, norampnetimports so bothmetricsanddetection_evalcan use itwrap_xexplicit per §5scripts/analysis/seam_review.py— blind adjudication deck for the 360 seam, in the scoring code: 11 double-marked ramps in manual_gold, and a matcher that does not wrap #130 GT pairs, with matched controls10. What I would most like challenged
Listed because they are the weakest links, not as a formality.
cyadequate, or does the seam band differ on something else — occlusion, vehicle position, hood/nadir proximity — thatcydoes not capture?cyas a range proxy. It is the vertical position in the pano, which is monotone in range on flat ground. Flat-ground distance carries a ~11% range-proportional scale error, not just scatter — multi-view self-consistency measures it and DA3 can anchor it #101 measured a ~11% range-proportional scale error on exactly that assumption. Does that matter here?manual_goldholds 234 within-radius GT pairs away from the seam, 87 of them with near-identicalcyat the horizon — genuinely adjacent far-field ramps. The uniform-azimuth null predicts 3.16 of the 11 seam pairs arise by chance. So roughly 3 of the 11 may be real pairs, and auto-merge would silently delete real ramps in the direction that flatters our recall. Hence the blind adjudication deck rather than a code rule. Is that the right call?Reproduction — §2, the recall deficit (committed caches only; no GPU, no network, no imagery)
Reproduction — §3, the Stage 1 dataset scan (network; reads ~12 MB of label columns, not the images)
Refs #130 (the scoring-side fix), #43 (the reviewer rule and the GT viewer), #51 (the baseline comparison that reads
manual_gold), #46 (where this surfaced), #48 (multi-view, which inherits the blind spot), #78 (flip-TTA, which this is not), #101 (the range-proxy caveat in §10).🤖 Generated with Claude Code (claude-opus-5[1m])