dist_gen.weighted_choice/zipf_choice rebuild their weight tables on every single call #147

Context

Raised by @tim-band during review of #137: "This does make propose even slower for multiple columns; perhaps we need to do some work to figure out how bad it is, because it might be that creating the proposers is 90% of it and so it's not a huge deal."

Profiled this directly (see the PR discussion for the methodology and full numbers). Short version: proposer construction is 1-7% of propose's cost, not 90% - the new evaluation/ranking layer dominates. But that's not because the new scoring math is itself expensive: on a 5000-row synthetic table, 8 ordinary proposers averaged ~10ms each to evaluate, while dist_gen.weighted_choice/dist_gen.zipf_choice (and their [sampled] variants) averaged ~70ms each - 7x more.

Root cause

WeightedChoiceProposer.generate_data(count) (proposers/choice.py):

def generate_data(self, count: int) -> list[Any]:
    """Generate ``count`` random data points for this column."""
    return [dist_gen.weighted_choice(self.values) for _ in range(count)]

calls weighted_choice(a) (providers.py) once per row:

def weighted_choice(self, a: list[dict[str, Any]]) -> Any:
    if not a:
        return None
    vs = []
    counts = []
    for vc in a:
        count = vc.get("count", 0)
        if count:
            counts.append(count)
            vs.append(vc.get("value", None))
    c = random.choices(vs, weights=counts)[0]
    return c

a (self.values) is the same list every call within one generate_data() loop (~500 distinct values in the profiled case), but vs/counts are rebuilt from scratch every single call, and random.choices() itself does O(n) setup (building cumulative weights) internally each time too. For a 4000-row preview sample against ~500 distinct values, that's ~2,000,000 list-rebuild operations per proposer, every time propose runs against a categorical-ish column. zipf_choice/zipf_choice_direct has the same shape of problem.

This was invisible before #137 because the old propose preview only ever requested limit=5 rows; the new evaluation layer needs sample_size=4000 for a statistically meaningful comparison, which is what exposes this.

Possible fix

Precompute the (vs, counts) pair (and/or a random.choices-ready cumulative-weight structure) once per generate_data(count) call instead of once per row - e.g. move the list-building loop out of weighted_choice() and into WeightedChoiceProposer.generate_data(), calling a cheaper per-draw primitive in the loop. Same idea for zipf_choice.

Worth checking separately

Not verified yet, but worth checking: if real bulk data generation (via config.yaml + the actual generation engine, not just propose's preview) also calls weighted_choice/zipf_choice once per generated row, this isn't just a propose-preview cost - it could already be slowing down real table population for any column using these generators, independent of #137, potentially much more significantly for a large destination table.