fix(assistantbench): distinguish thousands separators from decimal commas

[Andiii208]

Root cause

fix_number rewrites every comma to a period to support European decimal commas, which misreads US thousands separators: "3,080,000" becomes "3.080.000", which float() rejects, so the answer falls back to string comparison and scores 0; "1,000" becomes 1.0, a 1000x error that scores 1.0. The same logic is duplicated in evaluate_dicts.py, so the dict evaluation path is affected too (JSON value "1,000" against gold 1000 scores 0.5 instead of 1.0).

Approach

Disambiguate the comma before the float() attempt, in one shared helper (_fix_comma in evaluation/evaluate_utils/utils.py) used by both fix_number copies:

• multiple commas, or a single comma followed by exactly 3 digits → thousands separator, commas removed
• single comma followed by 1–2 digits → European decimal comma, comma turned into a period
• anything else → unchanged (non-numeric strings keep falling back to string comparison)

The multiple-comma rule also handles grouping styles that aren't strict groups of three (e.g. 12,34,567), and a single comma followed by 4+ digits is intentionally left as before. Note this evaluator is a verbatim copy of the official AssistantBench leaderboard evaluator, so the same fix likely applies there.

Verification

Before / after, calling question_scorer directly:

prediction	gold	before	after
3,080,000	3080000	0	1.0
1,010,000	1010000	0	1.0
1,000	1000	0	1.0
1,000	1	1.0	0
1,010	1.01	1.0	0
14,2	14.2	1.0	1.0
[{"price": "1,000"}]	{"price": 1000}	0.5	1.0

New regression tests in tests/assistantbench/test_evaluation.py cover these cases. All 33 pinned rows keep their exact scores, and the full suite passes: pytest -m 'not pricy' tests/assistantbench → 85 passed (79 before + 6 new). black --check . (24.2.0, same as CI) is clean.

Fixes #404