The RampNet-vs-YOLO gap at matched operating points is 0.039, not 0.252 — plus the label rescue and the equirect control (#51)

[jonfroehlich]

Three connected things for #51, in the order they had to happen: rescue the training data a purge had deleted, then answer the standing equirect objection by scoring the tiles arms that objection exists for, then fix the comparison itself — which turned out to have been reading the two models at operating points chosen by different procedures.

Net effect on the headline: the RampNet-vs-YOLO residual is 0.039 F1, not 0.252 and not 0.160.

1. The comparison was not at parity, and that is most of the gap

model_scoreboard.md and yolo_geometry_51.md compared rows whose operating points were chosen differently: RampNet at 0.55 (its shipped deployment threshold), every YOLO leg at 0.25 — the Ultralytics predict() default that nobody selected. So the published gap mixed which model is better with whose default happened to suit F1 on this benchmark.

Give each model one uniform threshold, picked the same way, on a dev split the headline is never reported over (sao_paulo), macro-meaned over the seven pooled US splits:

model	sel thr	P	R	F1	ΔF1 vs RampNet	at published op	gain from parity
RampNet	0.30	0.896	0.797	0.843	—	0.824	+0.018
y11x_tiles (ep44)	0.10	0.875	0.749	0.804	−0.039	0.667	+0.137
y11x_pano (ep38)	0.10	0.787	0.726	0.752	−0.091	0.623	+0.129
y11x_pano_h200 (ep60)	0.10	0.839	0.683	0.751	−0.092	0.575	+0.176

The asymmetry in the last column is the whole finding. 0.55 was nearly right for RampNet; 0.25 was 0.137 F1 away from YOLO's own optimum. The gap holds at 0.032–0.039 across all three candidate dev splits, so the choice of dev split is not carrying it. Selection never touches a reported split, and test_no_candidate_dev_split_is_ever_reported_over asserts that rather than trusting review.

Two things reviewers should see before merging

RampNet loses manual_gold at parity — 0.902 against the tiles arm's 0.911 — and loses morgantown (0.845 vs 0.862) and ties bend. manual_gold is the only split whose ground truth was labelled independently of RampNet's own outputs, the least circular comparison in the benchmark, and the supervised baseline takes it. This is the first split any non-RampNet model has won outright at a defensible operating point.

AP is unmoved and still favours RampNet, 0.849 vs 0.773. A correction travels with that: an earlier reading of this analysis claimed the AP columns were asymmetrically floored, and that was wrong — op_cache meta.score_floor and YoloDetector.score_threshold are both 0.05, so that column was always like-for-like. The honest summary is that RampNet has the better curve, while at each model's own best point the F1s are much closer than published.

2. The equirect objection, measured

#51's claim is that the architecture is what RampNet contributes. The standing objection is that the YOLO arms are handicapped — fed 2048×4096 equirectangular panoramas, where straight edges bow and a far ramp is a few pixels. The tiles arms exist to answer exactly that, and no tiles checkpoint had ever been scored, so the control for the headline was unmeasured.

At the pre-registered conf 0.25, pooled macro-mean F1:

leg	epoch	geometry	F1	vs RampNet
y11x_tiles	44	perspective, imgsz 1024	0.667	−0.160
y11x_pano	38	whole-pano, imgsz 1280	0.623	−0.204
y11x_pano_h200	60	whole-pano, imgsz 1280	0.575	−0.252 (the published arm)

The raw +0.092 tiles-minus-published difference would overclaim, because ep44-vs-ep60 mixes geometry with budget. Split on the pano lineage, where only budget moves:

• over-training (pano ep60 → ep38): +0.048, same geometry
• geometry (pano ep38 → tiles): +0.044, roughly matched budget

So the handicap is real and worth about 0.044 F1 — half the recoverable difference, not all of it. As with the Vistas parity arm, the geometry half is recall (0.474 → 0.520) at flat precision.

The control reproduced exactly. y11x_pano_h200 was published 2026-08-14 against a repo predating the #132 seam fix, which changed how the matcher wraps the 360° seam. Re-scored here under the same commit as the two new legs it returns P 0.969 / R 0.416 / F1 0.575 / AP 0.730 — identical to three decimals. That validates the comparison and means the published #51 table needs no seam amendment. The assertion is in the script, so the day it stops holding it fails loudly.

A result nobody was looking for: more training made the pano arm worse out of distribution. ep38 beats ep60 on 6 of 7 city splits while the ordering reverses in-distribution (manual_gold 0.840 vs 0.851). That lines up with the already-recorded finding that the 1-epoch RampNet release beats all three 60-epoch YOLO arms, and it is the main reason the open ~400 GPU-hour ep60 run still does not look worth it.

The budget term is confounded — y11x_pano and y11x_pano_h200 are divergent continuations of one lineage on different hardware, not one run read at two epochs. Stated next to the number, in the doc.

3. The data rescue that had to come first

Every label directory under /gscratch/scrubbed/jfroehli/yolo/ was empty — all five dataset variants, train and val — while every image directory was intact. That is why y11x_tiles (38612069) and its two retries died at dataset init with No labels found in .../tiles/labels/train.cache.

The mechanism is worth naming because it will recur: scrubbed purges by access time, and the .cache files are precisely what stopped anything from reading the individual label .txt files. Their atime froze while the images kept being read every epoch, so the purge took the labels and left the images. The cache that made training fast is what got the labels deleted — same shape as the partially-purged conda package cache earlier in #51. A cache is not a backup, and its presence is not evidence its source still exists.

Recovery needed no GPU and no re-derivation: Ultralytics' cache stores parsed labels (cls and normalized-xywh bboxes), which is the on-disk .txt format itself — 968,227 boxes over 557,413 train records. The writer is lifted from prepare_yolo_dataset.py::_write_pair so rebuilt files are byte-identical for anything surviving the float32 round trip, including the zero-byte file a background tile gets — load-bearing, since Ultralytics counts a missing label as nm and an empty one as nf, and the error above fires when nf == 0.

Then the same failure was closed off for the arms actually under discussion: the pano label caches had no durable copy at all, and the checkpoint snapshot was 14 days stale on every tiles arm (y11x_tiles ep44 vs ep32 saved, y11x_pano ep38 vs ep30, y26_tiles ep12 vs ep9, y11l_tiles ep11 vs ep9) — so the checkpoints an evaluation would score existed at their current epoch in exactly one place, and it was the volume with the demonstrated purge behaviour. The rescue is a committed script rather than hand-assembly, because a hand-assembled backup cannot be re-run by anyone else.

The epoch-curve follow-up is retracted

The geometry write-up closed by recommending the cheap follow-up — score the tiles checkpoints that already exist at several epochs — to turn the confounded budget term into a measured one. Those checkpoints never existed. Every arm trained save_period: -1, so Ultralytics kept only best.pt and last.pt; find over both the durable snapshot and /gscratch/scrubbed/jfroehli/yolo_runs returns zero epoch*.pt, and best is 0–4 epochs from last on every arm. y11x_pano_h200 was a resume, so it inherited save_period=-1 and Tillicum's SAVE_PERIOD=5 never applied. retarget_yolo_checkpoint.py already documented exactly this, so the recommendation contradicted a committed docstring in this repo. The budget term cannot be de-confounded cheaply: full retrain or nothing.

Gaps, stated

• One seed per cell. This is now the binding limitation, not a footnote. At a 0.160 gap seed variance was irrelevant; at 0.039 it is the entire question, and Train a supervised YOLO baseline (YOLO11 / YOLO26) on the RampNet dataset — isolate architecture vs. data #51's own rule is that differences under ~0.02 should not be read. Three seeds of y11x_tiles is the experiment that would settle whether the architecture claim survives — and it should come before the ~400 GPU-h ep60 run.
• The tiles arm's parity F1 is a lower bound. With budapest_district5 as dev its selected threshold is 0.05, the cache floor — the edge of what was ever measured. Closing this means re-dumping both models at a lower floor; doing one would reintroduce the asymmetry this PR removes.
• y11x_tiles is at ep44 of a 60-epoch schedule, but the pano lineage suggests more epochs would hurt it out of distribution, so this is not simply "the undertrained arm".
• Only the x architecture is swept; y11l_* and y26_* have no committed sweep.
• The VLM and pointing challengers cannot be included at all — no calibrated confidence, so one operating point rather than a curve. Their rows in model_comparison.md are unaffected in either direction.

Reproducing

CPU only, no GPU, no network — everything it reads is committed:

python scripts/analysis/operating_point_parity_51.py --sensitivity
python scripts/analysis/operating_point_parity_51.py --check   # fails if the artifact drifted
pytest tests/test_operating_point_parity_51.py tests/test_yolo_geometry_51.py -q

The control worth keeping. RampNet at 0.55 re-scores to 0.824 from op_cache against the bundle-derived published 0.827. That −0.0025 is peak extraction, not a floor effect (the bundle floor is 0.5501, i.e. above 0.55): peak_local_max at a 0.05 floor finds a different peak set than at 0.55, because newly admitted low peaks change which maxima survive min_distance. The script asserts the two stay within 0.005 and fails loudly if that stops being true. Never assert equality between the two sources.

Related: #151 found that the RampNet–YOLO ordering flips between imagery rigs over the same ground, which qualifies this result further.

🤖 Generated with Claude Code (claude-opus-5[1m])

[jonfroehlich]

Deep review

Re-ran everything from the branch head (92bc1d6) in a clean detached worktree on Windows, CPU only. The measurement is sound and reproduces exactly, and the headline — the published RampNet-vs-YOLO gap is mostly an operating-point artifact — survives. What does not survive is the explanation the PR gives for its own control, and the branch will go red the moment it meets #152/#153.

• operating_point_parity_51.py --sensitivity and yolo_geometry_51.py both rewrite their JSON byte-for-byte identical to the committed copies (sha256 9da31321…, 678ae904…); both --check commands exit 0.
• pytest tests/test_operating_point_parity_51.py tests/test_yolo_geometry_51.py: 37 passed. Full suite: 1359 passed, 1 skipped in 121 s.
• Every number in the PR body and both docs that I could trace matches the artifacts: the parity table (0.30/0.10; 0.843/0.804/0.752/0.751; −0.039/−0.091/−0.092; 0.824/0.667/0.623/0.575; +0.018/+0.137/+0.129/+0.176), the sensitivity band (0.032 / 0.039 / 0.038 for the three dev splits), the per-split table, the geometry table (0.667/0.623/0.575, AP 0.773/0.752/0.730), the decomposition (+0.048 / +0.044 / +0.092), the recall term (0.4743 → 0.5197), "ep38 beats ep60 on 6 of 7" (paterson is a tie at 0.635), and the control leg (0.969 / 0.416 / 0.575 / 0.730, and the test really does fail when a split is perturbed).
• Merge-tree is clean against main and every other open branch. The head did not move during the review.

---

1. High — the scripts and tests read the live US_SPLITS, so this PR and #152/#153 cannot both be green on main

Both scripts import ALL_SPLITS / US_SPLITS from analysis.low_floor_sweep (operating_point_parity_51.py:93, yolo_geometry_51.py:44) and macro_at / pooled return None unless every pooled split has a sweep. #152 adds laurens to US_SPLITS; #153 renames it laurens_mapillary and adds laurens_gsv to CITY_SPLITS. No YOLO report exists for either.

I built a scratch merge of this branch with fix/laurens-challenger-coverage (no textual conflicts — git merge-tree is clean) and ran the two test files on it: 12 of 37 fail, and both --check commands fail. Worse, the parity script does not just fail — it re-pools RampNet over eight splits (laurens has an op_cache), drops all three YOLO legs silently, and prints RampNet at F1 0.820 against a "published" 0.789. The control catches it (test_control_agrees_on_the_committed_data fails), which is the right outcome, but whichever of these PRs merges second turns main red.

This is exactly the trap #152's body describes for farfield_forensics.py and silent_activation.py ("a frozen study reading the live US_SPLITS"), and it applied the fix there: an explicit split tuple in the script with the reason. Same fix here.

Fix: pin POOLED_SPLITS = ("richmond", …, "gainesville") and ALL_SPLITS_AS_RUN (the ten in env.txt) as module constants in both scripts with a comment saying they are the population the 2026-08-30 run covered and are not to follow the registry; derive NON_POOLED from those; make test_pooled_uses_exactly_the_seven_us_splits and test_every_split_has_all_three_legs assert against the pinned tuples rather than mod.US_SPLITS / mod.ALL_SPLITS.

2. High — the control's stated mechanism is wrong, and docs/operating_point.md already says what the real one is

The PR body, operating_point_parity_51.md:153–158, the script docstring (:51–70) and test_control_delta_is_the_known_extraction_difference all say the −0.0025 between RampNet's op_cache re-score and its bundle at 0.55 "is peak extraction: peak_local_max at a 0.05 floor finds a different peak set than at 0.55, because newly admitted low peaks change which maxima survive min_distance."

That cannot be the mechanism. Lowering threshold_abs only adds candidates; a pixel that is the maximum of its min_distance neighbourhood at 0.55 is still that maximum at 0.05. A lower floor can never move or remove a ≥ 0.55 peak.

Measured per split, bundle (≥ 0.55) against op_cache (≥ 0.55), matching detections within 0.004 normalized:

split	bundle	op_cache	in bundle, not in cache	at the seam	in cache, not in bundle	ΔF1 @0.55
richmond, clovis, morgantown, annapolis, budapest_district5	—	—	0	0	0	0.0000
bend	265	257	35	0	27	+0.0034
paterson	284	281	36	0	33	−0.0048
gainesville	205	197	33	1	25	−0.0163
sao_paulo (the dev split)	251	228	53	1	30	−0.0189
manual_gold	3610	3487	495	26	372	−0.0095

Five splits agree detection-for-detection (tp/fp/fn identical). The other five differ by 13–14 % of their detections, mostly at different locations (bend: 8 of 35 are the same point at a different confidence; the rest are elsewhere). That is two different heatmaps, not one heatmap read at two floors. And the repo has already measured why — docs/operating_point.md:80–90: manual_gold's bundle was exported with horizontal-flip TTA ("3610 vs 3487 at ≥ 0.55", the exact counts above), while op_cache is the no-TTA deployment path; and for the GSV splits (bend, paterson, gainesville, and sao_paulo) the production bundle came from a different resample of the panorama than the native-res cache, with detections displaced up to 0.44 R. The five that agree are the Mapillary splits, which #152 records as bit-exact between the two paths.

Consequences for this PR:

• The 0.005 tolerance passes on the pooled macro only by cancellation (+0.003, −0.005, −0.016 across the three GSV splits in the pool). A per-split control would have failed on gainesville.
• The dev split has the largest discrepancy of the ten (−0.019). RampNet's threshold was selected on the split where its op_cache curve is least like its shipped detections. (The selection is probably robust — the sensitivity table lands on 0.30–0.35 either way — but that is an argument the doc has to make, not one it can skip.)
• On manual_gold the TTA-vs-no-TTA delta (0.0095) is the same size as the margin by which the tiles arm "wins" it (0.009). See finding 3.

Fix: replace the peak-extraction paragraph in the doc, the script docstring, and the test docstring with the documented causes and a pointer to operating_point.md's parity gate; add the per-split Δ table above to the artifact (control.per_split) and print it; keep the macro assertion but add a per-split one with its own tolerance, including the dev split and manual_gold, so cancellation cannot hide a split-level divergence again.

3. Medium — the per-split "wins" are read below the doc's own noise rule and inside the source discrepancy

operating_point_parity_51.md "Gaps" says differences under ~0.02 F1 should not be read. Then §2 reads 0.009 (manual_gold, 0.911 vs 0.902) as "loses the split that matters most" and "the first result in this benchmark where a non-RampNet model takes a split outright", 0.017 (morgantown) as a win, and the status line counts "3 of the 10 splits". The PR body promotes manual_gold to one of the two things reviewers must see before merging.

The direction is not fragile to RampNet's threshold — over the whole 0.05–0.95 grid RampNet's best on manual_gold is 0.904 @0.35 and on morgantown 0.851 @0.40, both still below the tiles arm — but the margin is inside the TTA delta on that split (finding 2): RampNet's committed manual_gold configuration (TTA, detections_meta.json) at 0.55 scores 0.9085 against the no-TTA 0.8990, so at 0.30 with TTA it would plausibly tie 0.911. A paired test would settle it and needs no GPU (both models' per-pano detections exist), but the tiles arm's detections are not committed (see the decision below), so it cannot be run from a clean clone.

Fix: qualify §2, the status line and the "3 of 10" count with the 0.02 rule and the TTA caveat ("within the noise rule; the manual_gold margin is the size of the TTA delta on that split; a paired test is blocked on publishing the tiles detections"). The PR body's reviewer note is Jon's to soften.

4. Medium — the scoreboard and the YOLO README now contradict this PR and do not point to it

• docs/model_scoreboard.md:362: "The YOLO tiles arms are absent — still training." The tiles arm is scored on all ten splits in this PR.
• scripts/model_comparison/yolo_baseline/README.md:88: "the tiles arms remain unevaluated."
• The scoreboard's headline table and its manual_gold column (RampNet 0.908 vs YOLO11x 0.851) are the published comparison; a reader of that page has no way to find that at matched operating points the residual is 0.039 and the best YOLO leg is within noise of RampNet on three splits. This PR touches neither file.

Fix: a short caveat block in model_scoreboard.md beside the headline table (or under "Choosing an operating point") linking operating_point_parity_51.md and yolo_geometry_51.md, stating that the YOLO rows are at the un-selected 0.25 default and what parity does to them; replace the two stale sentences. docs/model_comparison.md's YOLO section should get the same one-line pointer.

5. Medium — provenance is recorded, but four gaps are not stated beside the numbers

• env.txt: "repo dirty : 10 paths" at d964d5d. The doc says "at repo d964d5d" only. git status --porcelain | wc -l counts untracked outputs and modified code alike, so a reader cannot tell which; the driver should log the list, not the count.
• The control leg was not re-inferred. All ten *_pano.txt reports say [y11x_pano_h200] all N panos already cached; model load skipped. The control re-scores detections cached on makelab2 from the 2026-08-14 publishing run — which is the right test for a matcher change and is what "re-scored" means, but run_yolo_geometry_eval.sh:19 says "re-run", and the doc should say the control isolates the matcher and does not test inference reproducibility.
• Where the checkpoints live. env.txt records sha256s of copies under makelab2 yolo_ckpts/. Nothing in the PR says which durable path on klone (/gscratch/makelab/jonf/rampnet_yolo_baseline_51/<arm>/weights/best.pt per run_durable_snapshot.slurm) carries those hashes, or that the checkpoints are unpublished — so "reproducible by someone with cluster access" is true but undiscoverable, and "not reproducible from a clean clone" is unstated.
• The rescue run's outcome lives only in the PR body. rebuild_yolo_labels_from_cache.py and check_yolo_dataset_loads.py carry the expected counts (557,413 / 968,227 / 59,923 background) in docstrings; the job ids, dates, --verify PASS and the acceptance-test output are not in the repo.

Fix: a "Provenance and cost" block in yolo_geometry_51.md covering the four points; run_yolo_geometry_eval.sh logs git status --porcelain in full; commit the rebuild/snapshot job logs (or a dated section in yolo_baseline/README.md with job ids and the PASS lines).

6. Medium-low — cost is not recorded

driver.log gives the eval as 17:10:09 → 18:17:48 on one A40: 67 min 39 s wall, ≈ 1.13 GPU-h, $0 (makelab2). The klone rescue and snapshot jobs (CPU, ckpt-all) have no job ids anywhere in the repo. The time-and-money rule is a standing one even though #147 has not merged.

Fix: one line in each doc; a compute_log.jsonl row once #147 lands (state "pending #147" until then).

7. Low

• rebuild_yolo_labels_from_cache.py overclaims "byte-identical". Ultralytics verify_image_label (read in 8.4.104 here; 8.4.120 at eval) drops duplicate rows and, when it does, reorders the survivors (lb = lb[i] from np.unique(..., return_index=True)). A file that had a duplicated box is rebuilt deduplicated and sorted — training-equivalent, not byte-identical. write_text(body) also takes the platform newline; add newline="\n". The float32 arithmetic itself is right (≤ 6e-8 error against 5e-7 rounding), and the 1e-6 verify tolerance is correct.
• test_no_candidate_dev_split_is_ever_reported_over cannot fail. NON_POOLED is defined as the complement of US_SPLITS (:106), so isdisjoint is a tautology. The invariant is real — argparse choices and build() enforce it — but the doc cites this test as the guarantee. Assert it on build(dev) for every candidate instead (dev not in result["pool"] and the pool is exactly the pinned seven).
• PUBLISHED_RAMPNET_F1 = 0.827 and 0.55 are duplicated across the two scripts; the parity script already imports from yolo_geometry_51. Import them.
• The AP table's RampNet 0.849 is not in either artifact. It is the scoreboard's macro AP (model_scoreboard.md:32, scoreboard.json); say so in the caption or compute it from op_cache in the script.
• Slang in committed text: "load-bearing" (yolo_geometry_51.md:40, yolo_geometry_51.py:19, both test docstrings), "split-brain" (operating_point_parity_51.md:156, .py:64). Plain wording.
• scripts/analysis/README.md indexes the analysis scripts and does not list the two new ones.

---

Decisions for Jon, not fixes

• Register y11x_tiles and y11x_pano as roster legs and publish their detections. Neither is in rampnet/roster.py and benchmark/model_detections/ has no files for them, so the scoreboard cannot show the parity result and the paired test in finding 3 cannot be run from a clean clone. export_model_cache.py from the makelab2 cache is the path; it is the same "unpublished parity detections" gap Vistas at resolution parity: the handicap was real, and it was recall (#126) #137 carries for Vistas.
• Merge order with Add Laurens, IA — the first rural split (recall 0.390) and register it as the 8th US split #152/The Laurens deficit is the rig, not the town: add the GSV arm and score all 11 challengers on both (#151) #153. After finding 1 there is none. Before it, whichever merges second is red.
• Three seeds of y11x_tiles — the doc's own recommendation and a GPU spend.
• The GSV-split op_cache vs production-resample discrepancy (finding 2) is a pre-existing GT-completeness correction: spot-check the low-confidence incremental FPs from the operating-point curve #55/Assess location precision for the unassessed candidate cities — the gate every route to 500k depends on #96 matter, not this PR's; whether to regenerate those caches is separate.

What holds up

• Both artifacts regenerate byte-identically on Windows; LF-only, floats rounded to 4 dp; --check on both is a real guard (the JSON comparison is of the full payload, sensitivity included).
• The methodology is the fair one: a single uniform threshold per model, selected on a split the headline never reports over, with sensitivity over all three candidates; macro-mean matches the scoreboard's convention (scoreboard.py:33, 356–398); both floors really are 0.05 (meta.score_floor on all ten caches, YoloDetector.score_threshold = 0.05 at detectors.py:1290), so the AP comparison was always like-for-like as the correction says.
• RampNet's selected threshold on sao_paulo is 0.30 — Lower the deployment operating point (peak threshold 0.55 → ~0.25–0.35): a free, recall-first lever #54's recommendation, reached by an independent route.
• The control leg reproduces its published row to every digit and the test proves the check can fail.
• The decomposition arithmetic, the "6 of 7" claim (paterson a tie at 3 dp), the in-distribution reversal, and the recall-not-precision mechanism all check against the JSON.
• The retraction is kept beside the recommendation it retracts (yolo_geometry_51.md:152–167), and the "Superseded" block in §2 points forward — nothing was silently deleted.
• The rescue tooling has no destructive step: .tmp + rename after a hash match, a second source hash to catch a torn copy, refusal to overwrite a differing durable copy without FORCE=1, stale .cache files moved aside not deleted. Zero-byte background files are the correct Ultralytics semantics (nf, not nm), and the acceptance test drives YOLODataset itself.
• env.txt pins repo HEAD, ultralytics version, GPU and three sha256s, and a test asserts that stays true.
• No ToS or licensing text anywhere in the diff.

Fix list

• [F1] High — Pin the split population in both scripts as explicit tuples with a reason (seven pooled + the ten as run), derive NON_POOLED from them, and make the tests assert against the pinned tuples, not the live US_SPLITS/ALL_SPLITS — files: scripts/analysis/operating_point_parity_51.py, scripts/analysis/yolo_geometry_51.py, tests/test_operating_point_parity_51.py, tests/test_yolo_geometry_51.py
• [F2] High — Replace the "peak extraction / min_distance" explanation with the documented causes (flip-TTA on manual_gold; production-vs-native resample on the GSV splits; Mapillary splits identical), cite docs/operating_point.md's parity gate, add a per-split bundle-vs-op_cache Δ table to the artifact and the doc, and add a per-split control assertion (dev split and manual_gold included) beside the macro one — files: scripts/analysis/operating_point_parity_51.py, docs/operating_point_parity_51.md, tests/test_operating_point_parity_51.py, docs/data/operating_point_parity_51.json
• [F3] Medium — Qualify the manual_gold / morgantown / "3 of 10" reads with the doc's own 0.02 rule and the TTA caveat; state that a paired test is blocked on publishing the tiles detections — files: docs/operating_point_parity_51.md
• [F4] Medium — Add a caveat block to model_scoreboard.md linking both new docs and stating the YOLO rows are at the un-selected 0.25 default; replace the stale "tiles arms are absent / remain unevaluated" sentences; one-line pointer in docs/model_comparison.md's YOLO section — files: docs/model_scoreboard.md, scripts/model_comparison/yolo_baseline/README.md, docs/model_comparison.md
• [F5] Medium — Add a "Provenance and cost" block: the 10 dirty paths caveat, the control re-scored from cache not re-inferred, the durable checkpoint path + unpublished status, and the rescue job ids/dates/PASS output; make the driver log git status --porcelain in full — files: docs/yolo_geometry_51.md, scripts/model_comparison/yolo_baseline/run_yolo_geometry_eval.sh, scripts/model_comparison/yolo_baseline/README.md
• [F6] Medium-low — Record the eval cost (67 min 39 s on one A40 ≈ 1.13 GPU-h, $0) and the klone rescue jobs' ids in the docs — files: docs/yolo_geometry_51.md, docs/operating_point_parity_51.md
• [F7] Low — Docstring caveat on dedup/reorder (not byte-identical for files with duplicate boxes); write_text(body, newline="\n") — files: scripts/model_comparison/rebuild_yolo_labels_from_cache.py
• [F8] Low — Make the dev-split test assert on build(dev) output for every candidate instead of the tautological isdisjoint — files: tests/test_operating_point_parity_51.py
• [F9] Low — Import PUBLISHED_RAMPNET_F1 (and the 0.55 deployed threshold) from yolo_geometry_51 instead of redefining — files: scripts/analysis/operating_point_parity_51.py
• [F10] Low — State the source of the AP table's RampNet 0.849 (scoreboard macro AP) or compute it in the script — files: docs/operating_point_parity_51.md, scripts/analysis/operating_point_parity_51.py
• [F11] Low — Replace "load-bearing" and "split-brain" with plain wording — files: docs/yolo_geometry_51.md, docs/operating_point_parity_51.md, scripts/analysis/yolo_geometry_51.py, scripts/analysis/operating_point_parity_51.py, tests/test_operating_point_parity_51.py, tests/test_yolo_geometry_51.py
• [F12] Low — Add the two new scripts to the analysis index — files: scripts/analysis/README.md

🤖 Generated with Claude Code (claude-fable-5-1)