fix(dsls): keep CFG slot content from running past its end tag

[Ki-Seki]

Summary

Under output_type="cfg", slot contents could violate their regex (e.g. regex toxic .* filled with prosthetics), and some responses failed to parse with Tag ids should be in order. This PR makes the compiled CFG actually enforce each slot's regex, and translates Python regex escapes that llguidance reads differently.

Root cause

1. A slot could run past its own end tag. The GIM tags are plain-text tokens (<|/MASKED|> is < | / MASK ED | > for GIM-4B), so a slot terminal such as /toxic .*/ or /[^，]+/ can match <|/MASKED|> itself. With m_k[capture, suffix="<|/MASKED|>"], the slot ends at the end tag only if the regex is already satisfied. Otherwise llguidance keeps the end tag, and every tag after it, inside that slot. Generation then stops on the vLLM stop string <|/GIM_RESPONSE|> while the grammar is still in the slot, and the non-strict parser accepts the unconstrained text. Replaying such a response token by token through LLMatcher with the GIM-4B tokenizer shows that every token is allowed and the grammar does not end in an accepting state.
2. Regex dialect. An unescaped / closes the lark /.../ literal early. Python accepts escaped non-ASCII characters such as \（ or \。, but Rust regex rejects them, and it reads \< / \> as word boundaries. 20 of the 961 GIMBench-regex-match queries failed with Invalid CFG grammar.

Changes

• build_cfg now intersects every slot terminal with a NO_MAGIC terminal: T_k: /regex/ & NO_MAGIC. NO_MAGIC forbids the magic strings that MaskedTag already rejects in content (all of them except |>).
• New to_llguidance_regex() strips the backslash from escaped non-alphanumeric literals and always writes / as \/. Escaped ASCII letters and digits (\d, \w, 一, ...) and regex metacharacters are kept as they are.
• Tests: updated the expected grammars, and added regression tests that replay valid, escaping, and dialect-specific responses through LLMatcher with a byte-level tokenizer.

Verification

• pytest: 64 passed; main passes 61, and the 3 extra are the new tests. The 9 tests/models/test_vllm_offline.py tests fail the same way on main in an environment without vllm. ruff check, ruff format --check and mypy src pass.

• All 961 GIMBench-regex-match queries now compile, up from 941. On 898 real (regex, content) pairs from past result files, Python re.fullmatch and llguidance agree 100%.

• Mask computation cost is unchanged (about 0.4 ms/token with the GIM-4B vocabulary).

• GIMBench rerun: Sculpt-AI/GIM-4B, GIMBench-regex-match first 100 items, vllm-offline, cfg, vLLM 0.19.1 with the guidance backend. Both runs used gimkit 0.1.1; the "after" run only swaps in this dsls.py, which is otherwise identical between 0.1.1 and main.

  	before	after
  Match rate (GIMBench, error items excluded)	83.40% (216/259)	83.21% (223/268)
  Items with errors (out-of-order tags / grammar compile)	2 / 1	0 / 0
  Items that stopped while still inside a slot	9	0
  Items that hit max_tokens=8192	10	18
  Regex match within generations that completed the grammar	208/208	214/214

  After the fix, every remaining miss comes from a generation that hit the token budget. The model keeps trying to close a slot whose regex is not satisfied yet (the regex is not shown in the prompt), so it is no longer a structural violation.

Notes (not changed here)

• Truncated generations (finish_reason == "length") are still silently repaired by the non-strict infill. Callers that need the guarantee should treat them as failures.

🤖 Generated with Claude Code

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://SculptAI.github.io/GIMKit/pr-preview/pr-192/
Built to branch gh-pages at 2026-10-10 08:51 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!