Skip to content

feat(i18n): deterministic integrity/typography checks and real per-language style guides - #630

Open
Timur Tukaev (tym83) wants to merge 4 commits into
feat/i18n-pipelinefrom
feat/i18n-quality-improvements
Open

feat(i18n): deterministic integrity/typography checks and real per-language style guides#630
Timur Tukaev (tym83) wants to merge 4 commits into
feat/i18n-pipelinefrom
feat/i18n-quality-improvements

Conversation

@tym83

Copy link
Copy Markdown
Contributor

Summary

Three quality improvements to the translation pipeline. No architectural change — the glossary + style-guide + back-translation + two-reviewer + digest design is kept and extended.

Stacked on feat/i18n-pipeline, since that is where the pipeline lives.

What

  • Link destinations are masked. Masking previously covered code, shortcodes and comments, so a URL sitting in ordinary prose ([text](https://…)) was defended by a prompt rule alone — the model could mutate or localize it and nothing would catch it. URLs are now placeholders like any other protected span, while link text stays exposed and still gets translated.

  • Two deterministic checks feed the existing revise loop.

    • integrity_findings() compares versions, bare CLI flags and do-not-translate terms between source and translation, catching a localized version separator (v1.5v1,5), a dropped --flag, or a transliterated brand.
    • check_typography() enforces the rules the style guides state: Russian guillemets, German „…“, Spanish inverted marks, Chinese full-width punctuation, European-Portuguese vocabulary leaking into pt-BR, Devanagari digits.

    Both look at prose only — markup, code, link targets and list markers are exempt, so the checks do not fire on ASCII punctuation that is correct inside an HTML attribute or a path.

  • The six per-language style guides were 3–6 lines each, yet the entire fluency and typography strategy rests on injecting them into the translate prompt and both reviewer prompts. They are now 80–100 lines each and work as an instruction set and a review rubric: register and address form, heading conventions, a decision rule for terms outside the glossary, typography, number/date formatting, the grammar traps of translating from English into that language, calque patterns with fixes, false friends, and a per-language checklist of MT failure modes. Every original decision is preserved and expanded around.

  • Fixed a live contradiction. Hard rule 5 in prompts/translate.md said to keep numbers unchanged, while several style guides require a decimal comma or a different date order in prose — the model was reading two incompatible instructions. The rule now separates a number's value (and any version or identifier: literal, reproduced character-for-character) from its formatting in prose (follows the language).

  • lint_translations.py applies the typography rules to already-published pages, where a hand edit is otherwise never re-checked. Advisory by default, --strict to gate.

Why

Only back-ticked code was structurally guaranteed; bare-prose commands, identifiers and link URLs were soft. On a docs site this dense with code, a silently mutated URL or flag is the most expensive failure mode available. The style guides were the second gap: the architecture already routes them into three prompts, but there was almost nothing in them to route.

Validation

  • 91 tests pass (74 existing + 17 new covering link masking, integrity findings and typography).
  • The typography lint reports 0 findings across the entire existing translated tree after false-positive tuning.

Deliberately left as follow-ups

Glossary growth loop (harvesting recurring terminology findings into glossary.yaml), a needs_human reviewer flag distinct from revise, and a severity-based soft hold on the auto-reviewed stamp.

Timur Tukaev (tym83) and others added 3 commits July 24, 2026 10:31
Masking previously covered code, shortcodes and comments, so a URL, a bare
CLI flag, a version number or a brand sitting in ordinary prose was defended
by a prompt rule alone. Link destinations are now masked like any other
protected span, and two deterministic checks feed the existing revise loop:

- integrity_findings() compares versions, bare flags and do-not-translate
  terms between source and translation, catching a localized version
  separator or a transliterated brand.
- check_typography() enforces the per-language rules the style guides state
  (Russian guillemets, German quotes, Spanish inverted marks, Chinese
  full-width punctuation, pt-PT vocabulary leaks, Devanagari digits).

Both look at prose only; markup, code and link targets are exempt so the
checks do not cry wolf on correct ASCII punctuation in an HTML attribute.
lint_translations.py applies the same typography rules to already-published
pages, where a hand edit is otherwise never re-checked.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Each guide was 3-6 lines, yet the whole fluency and typography strategy
rests on injecting them into the translate and both reviewer prompts. They
are now 80-100 lines each and work as an instruction set and a review
rubric: register and address form, heading conventions, a decision rule for
terms outside the glossary, typography, number/date formatting, the grammar
traps of translating from English into that language, calque patterns with
fixes, false friends, and a reviewer checklist of the MT failure modes
specific to the language.

Every original decision is preserved; the guides expand around them.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
…te prompt

Hard rule 5 said to keep numbers unchanged, while several style guides
require a decimal comma, a different thousands separator or a different
date order in prose. The model was reading two incompatible instructions.
The rule now separates the two ideas it was conflating: a number's VALUE and
any version or identifier are literal and must be reproduced exactly, while
formatting in ordinary prose follows the language's style guide.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: cc6005be-bbab-439f-98f6-a159381883f5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/i18n-quality-improvements

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Two gaps found in review by @lexfrei.

protect() masked fences, shortcodes, comments and inline code, but not raw
HTML tags — and .html pages are mostly markup (Goldmark also runs with
unsafe: true, so Markdown pages carry raw HTML). content/en/docs/v1.5/
roadmap.html produced zero protected spans: its links, target="_blank" and
<br /> all reached the model verbatim. Worse, the placeholder guard in
_split_payload_response counts §§…§§ tokens, so a page with no tokens had
nothing to fail closed on — a translated class= or a dropped </div> would
ship silently, since the gate does not inspect markup, check-i18n.sh only
compares digests, and Hugo renders broken HTML without complaint. Tags are
now masked; the text between them stays exposed for translation. Autolinks
are masked first so <https://…> is not swallowed by the tag pattern.

translate_page also stamped l10n: mt unconditionally, after
merge_target_only_keys — clobbering the very marker the README names as the
human triage signal. Eight pages carry l10n: transcreate, including the four
localized _index.html homepages, which are in translate_globs; one edit to
the English homepage would have replaced a hand-written transcreation with
machine output, and the banner is wired into the docs layout only, so the
homepage would have carried it with no disclosure. Such pages are now kept
out of the worklist, l10n is never downgraded, and pages whose English
source has drifted are reported in the weekly PR so a human can refresh them
deliberately.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: tym83 <6355522@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant