feat(sector): TNT-style in-sector drifting for large sectors (opt-in)#266
Merged
Conversation
Large sectorial sub-problems have more reach but get stuck in their own local optima under plain RAS+TBR. Add TNT `sectsch godrift`/`gocomb`: sectors with >= sectorGoDrift real tips are re-solved by tree-drifting; >= sectorGoComb by combined analysis (sectorCombStarts RAS+drift starts, keep-best). Reuses the existing drift_search; default-OFF (both thresholds 0) so every current search is byte-identical. Anchoring: an unmasked drift on the HTU-anchored reduced sector tree floats the HTU (re-roots the sector vs the rest of the tree) in ~80% of solves, which the root-structure check then reverts to a NON-TBR'd topology -> net-negative. Fixed by threading an optional sector_mask through drift_search -> drift_phase and its internal tbr_search calls (additive, default nullptr; top-level driven_search drift unaffected): search_sector pins the synthetic root, HTU, content root and its two children per RAS start, cutting the HTU-float revert rate ~80% -> ~12%. Residual reverts stay safe via the existing root_ok check. Validated (Hamilton matched-wall A/B, 4 EW sector-resolve datasets x 5 seeds, rasStarts=3 both arms, sectorMaxSize=80): pinned godrift wins-or- ties every dataset with zero regressions for 3-cycle arms (sectorGoDrift 25-40, sectorDriftCycles=3). Shipped OPT-IN only; the recipe-default flip is gated on a separate preset-level (rasStarts- matched) A/B. The gocomb fuse recombination sub-step is deferred (sectorFuseRounds reserved/unused). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lt-off) (#267) * feat(sector): implement the gocomb fuse recombination sub-step Completes TNT combined analysis for large sectors: after the sectorCombStarts RAS+drift starts, recombine them with tree_fuse (best start as recipient, the others as donors) and keep the fused tree only if it beats every start. Previously gocomb was RAS+drift keep-best; the fuse step was deferred because tree_fuse re-roots the reduced tree at tip 0, dropping the (HTU, sr_mapped) synthetic-root layout reinsert_sector needs. Re-anchoring: parsimony is rooting-invariant, so reanchor_fused_at_htu simply re-roots the fused tree at the HTU pseudo-tip. The HTU represents the rest of the tree at the sector boundary and stays adjacent to the content root, so its sibling after re-rooting is the whole non-HTU clade -- exactly what reinsert_sector grafts back, with the content clade still rooted at sr_mapped. The fused tree is accepted only through the SAME root_ok gate + reinsert path as the drift starts (no reduced-score shortcut); if fuse ever floated the HTU into the interior -- the move the drift path already reverts -- root_ok fails and the best RAS+drift start stands, so a fused tree is reinserted only when its id mapping is already valid. Default-OFF (needs sectorGoComb > 0); env TS_SECT_FUSE=0 disables only the fuse step (drift-only comb) to isolate its marginal contribution in an A/B. Reuses the existing tree_fuse and ts_tbr's reroot_at_tip; no new scoring code. Local correctness gate (Zanol2014 74t, Giles2015 78t -- the wps>=2 regime): fuse fires and wins, and every reinserted tree's C++ score equals an independent by-label TreeLength. An initial permutation-remap for the HTU-float case was implemented, found empirically unreachable (the content root always keeps id sr_mapped), and removed to avoid shipping untested pointer surgery. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(sector): matched-wall harness isolating the gocomb-fuse margin Adds dev/benchmarks/sector_fuse_ab.R: within each combStarts level the fuse and drift-only arms differ only by TS_SECT_FUSE, so any delta is fuse alone. Records the pre-committed decision rule (beat drift-only -> opt-in, else stay default-off). This is the harness behind the null marginal result. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
ms609
added a commit
that referenced
this pull request
Jul 8, 2026
…odrift) Flips the godrift mechanism from #266 into a `thorough` default after two preset-level matched-wall A/Bs (2026-07-08, arrays 17836031 then 17836637): rasStarts=3 + sectorGoDrift=25 + sectorDriftCycles=3 beats stock thorough on every dataset class with no regression -- mid-size sector-resolve -1.5, general-pool (68-88t) -0.4, large training matrices (131-205t) -2.7, with the rep-starved 205t project3763 -8 steps and the hard floor project4138 reaching its optimum. intensive/large inherit via alias (confirmed positive/neutral on large trees, never a regression). Details behind the numbers: - sectorGoDrift=25 is calibrated to thorough's sector geometry: xss/cssPartitions=6 make XSS/CSS sectors ~12-15 tips, so drift engages via RSS large-clade picks; the standalone-tuned 40 is near-inert under this preset. - rasStarts>=2 is required (drift's retention channel is inert at 1) and coupled: rasStarts=3 ALONE (no drift) regresses -- it triples every sector-solve for fewer replicates -- but the drift more than redeems the cost. rasStarts=2+drift ties 3+drift at matched wall, so 3 (marginally best, round-1-validated) is kept. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds TNT
sectsch godrift/gocombin-sector drifting for large sectorial sub-problems, behind two new size thresholds (sectorGoDrift,sectorGoComb), default-OFF — every current search is byte-identical.>= sectorGoDriftreal tips are re-solved by tree-drifting instead of plain RAS+TBR.>= sectorGoCombrunsectorCombStartsRAS+drift starts, keep-best (gocombtakes precedence when both fire).drift_search; no new scoring code.Why
Large sectors have more reach but get stuck in their own local optima under RAS+TBR (TNT's motivation for
godrift). This is a candidate reach lever from the TNT feature-gap audit.Anchoring fix (the interesting part)
An unmasked drift on the HTU-anchored reduced sector tree floats the HTU (re-roots the sector against the rest of the tree) in ~80% of solves; the existing root-structure check then reverts those to a non-TBR'd topology → net-negative. Fixed by threading an optional
sector_maskthroughdrift_search → drift_phaseand its internaltbr_searchcalls (additive, defaultnullptr; top-leveldriven_searchdrift is unaffected).search_sectorpins the synthetic root, HTU, content root and its two children per RAS start, cutting the HTU-float revert rate ~80% → ~12%. Residual reverts stay safe via the existingroot_okcheck.Validation
Hamilton matched-wall A/B (SLURM 17832946): 4 EW sector-resolve datasets × 5 seeds,
rasStarts=3both arms,sectorMaxSize=80. Pinned godrift wins-or-ties every dataset with zero regressions for 3-cycle arms; gains concentrate on hard data (Zanol2014 −2.4..−4.0, Zhu2013 −1.0..−1.6), easy data ties. Harness:dev/benchmarks/sector_drift_ab.R.Scope / follow-ups
rasStarts>=3, which no preset runs (thoroughisrasStarts=1), so enabling in a preset would ship an untested combination. That flip is gated on a separate preset-level (rasStarts-matched) A/B.gocombfuse recombination sub-step is deferred (sectorFuseRoundsreserved/unused) — coming as a follow-up PR.🤖 Generated with Claude Code