[fanny-riols]

Why

Native speakers reviewed EVA conversations in 9 non-English languages (de, nl, fr, fr-CA, es, it, hi, ko, ja). Many of the judge errors they found depend on the language: how letters are named in a spelled code, which politeness formulas are expected, which letters sound alike. The judge prompts had no place for that kind of guidance.

This PR adds that place. It does not add any guidance yet; that comes in the two PRs stacked on top of this one.

What changes

• {language_guidance} slot in 9 of the 10 judge templates in judge.yaml, just before the response format. In the two key-entities prompts it ends the Correctness Criteria instead, before the worked example, since the guidance refines those criteria. user_behavioral_fidelity has no slot: it scores the simulated user against its own instructions (persona, goal, language directive), not against a language's norms.
• src/eva/metrics/language_guidance.py (new) loads configs/prompts/judge_language_guidance/<metric>.yaml, a mapping of language code to guidance text, and renders it as a ### <Language>-specific guidance block after a blank line.
  • A regional variant gets its base language's text followed by its own entry: fr-CA gets the fr guidance, then the fr-CA additions.
  • English never gets guidance, so it renders an empty string.
  • A guidance file whose top level is not a mapping, or whose value is not a string, raises a clear error.
• get_judge_prompt(..., language=...) in base.py fills the slot. language is a required keyword, so a new caller can't silently skip the guidance. The guidance is included in the prompt hash, so a guidance edit shows up in the stored prompt hash.
• Drift guard (signatures.py): a metric's prompt hash now also covers its guidance file.
• Call sites pass context.language: the base judge classes, speech fidelity and key entities.

English is unchanged

The slot replaces an existing blank line. With empty guidance, every judge prompt renders byte-identical to main. A new test compares the English renders with a golden file taken from main.

The stored prompt_hash of English runs does change, because it hashes the template text, and the template now has the slot.

Testing

• tests/unit/metrics/test_language_guidance.py:
  • English and missing files render empty;
  • fr-CA gets the base text then its own;
  • the block header;
  • a malformed guidance file raises;
  • every guidance file matches a judge prompt that has the slot, and every key is a supported non-English language code; no braces;
  • every judge template except user_behavioral_fidelity has the slot;
  • get_judge_prompt injects the guidance and the prompt hash changes with it;
  • English prompts match the golden renders (tests/fixtures/judge_prompts_en.json).
• Every judge prompt rendered in all 10 languages with this branch and with main: all 100 are byte-identical.
• Full unit suite: 2168 passed, 52 skipped, 3 xfailed.

Stack

1. Judge prompts: per-language guidance slot #249 This PR: guidance slot and loader.
2. Judge prompts: shared rules for spelled codes, mixed-language speech, glosses and politeness #250 Shared judge rules (based on this PR).
3. Judge prompts: guidance for de, nl, fr, fr-CA, es, it, hi, ko, ja #251 Per-language guidance content (based on PR 2).

Metric versions are bumped once, in PR 3, so each metric is bumped once for the whole stack.