Skip to content

BaseChatTemplateParser strips every TokenConfig string from message content to block control-token injection. For Qwen that set included tool_start_token ("\n<tool_call>\n") and tool_end_token ("\n</tool_call>"), which are ordinary text in tool-use system prompts and in assistant tool-call turns. Stripping them removed the <tool_call> wrapper from the function-calling format example that the Qwen3.5 chat template writes into the system prompt, so the model emitted bare <function=...> blocks and often failed to end its turn. In agentic RL on DeepSWE with Qwen3.5-35B-A3B, about a fifth of rollouts degenerated into repeated </function> until they hit the response-length cap, which more than doubled step time. - #2820

Merged
copybara-service[bot] merged 1 commit into
mainfrom
test_997511662
Oct 11, 2026

Conversation

@copybara-service

Copy link
Copy Markdown

BaseChatTemplateParser strips every TokenConfig string from message content to block control-token injection. For Qwen that set included tool_start_token ("\n<tool_call>\n") and tool_end_token ("\n</tool_call>"), which are ordinary text in tool-use system prompts and in assistant tool-call turns. Stripping them removed the <tool_call> wrapper from the function-calling format example that the Qwen3.5 chat template writes into the system prompt, so the model emitted bare <function=...> blocks and often failed to end its turn. In agentic RL on DeepSWE with Qwen3.5-35B-A3B, about a fifth of rollouts degenerated into repeated </function> until they hit the response-length cap, which more than doubled step time.

Exclude tool_start_token and tool_end_token from sanitization. The exclusion is in the base class, so Gemma4 now also keeps its <|tool_call> / <tool_call|> markup and Llama no longer strips its tool header from content; turn terminators such as <|im_end|> and <|eot_id|> are still stripped.

Also render runs of consecutive tool messages the way the Qwen chat template does, as a single user turn with one <tool_response> block per message, which changes the token layout for turns with several tool calls. QwenChatTemplateParser.preprocess_messages marks such runs so that tokenize_and_generate_masks, which renders one message at a time, produces the same text as parse.

…ge content to block control-token injection. For Qwen that set included `tool_start_token` (`"\n<tool_call>\n"`) and `tool_end_token` (`"\n</tool_call>"`), which are ordinary text in tool-use system prompts and in assistant tool-call turns. Stripping them removed the `<tool_call>` wrapper from the function-calling format example that the Qwen3.5 chat template writes into the system prompt, so the model emitted bare `<function=...>` blocks and often failed to end its turn. In agentic RL on DeepSWE with Qwen3.5-35B-A3B, about a fifth of rollouts degenerated into repeated `</function>` until they hit the response-length cap, which more than doubled step time.

Exclude `tool_start_token` and `tool_end_token` from sanitization. The exclusion is in the base class, so Gemma4 now also keeps its `<|tool_call>` / `<tool_call|>` markup and Llama no longer strips its tool header from content; turn terminators such as `<|im_end|>` and `<|eot_id|>` are still stripped.

Also render runs of consecutive tool messages the way the Qwen chat template does, as a single user turn with one `<tool_response>` block per message, which changes the token layout for turns with several tool calls. `QwenChatTemplateParser.preprocess_messages` marks such runs so that `tokenize_and_generate_masks`, which renders one message at a time, produces the same text as `parse`.

PiperOrigin-RevId: 997567041
@copybara-service
copybara-service Bot merged commit 8dcf606 into main Oct 11, 2026
4 checks passed
@copybara-service
copybara-service Bot deleted the test_997511662 branch October 11, 2026 22:33

This branch was successfully deployed

1 active deployment
testing — 8dcf6067 Deployed Oct 12, 2026 by copybara-service[bot] via tunix_tpu_unit_tests / run_dev #11282
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants