fix(llm): prevent orphaned tool_calls and missing type field in continue-gen path#908
fix(llm): prevent orphaned tool_calls and missing type field in continue-gen path#908lobstersyrup wants to merge 2 commits intomodelscope:mainfrom
Conversation
_call_llm_for_continue_gen was passing raw Tool dicts directly to
_call_llm without calling format_tools first. The raw Tool TypedDict
lacks the 'type': 'function' field required by OpenAI-compatible APIs.
This caused 400 errors ('tools[0]: missing field type') from providers
that strictly validate tool definitions (e.g., DeepSeek), particularly
when long responses triggered finish_reason=length and entered the
continue-generation path.
Fixed in both OpenAI._call_llm_for_continue_gen and
DeepSeek._call_llm_for_continue_gen to call self.format_tools(tools),
matching the existing pattern in generate().
There was a problem hiding this comment.
Code Review
This pull request updates the _call_llm_for_continue_gen method in both DeepSeekLLM and OpenAILLM to ensure tools are properly formatted before the LLM call. While the tool formatting fix is correct, the implementation in DeepSeekLLM contains several critical issues: a logic error where the stop sequences are lost because list.append() returns None, a potential AttributeError due to a naming mismatch in the message formatting call, and a missing update to the API call counter.
| stop = kwargs.pop('stop', []).append('```') | ||
| return self._call_llm( | ||
| messages=messages, tools=tools, stop=stop, **kwargs) | ||
| messages=messages, tools=self.format_tools(tools), stop=stop, **kwargs) |
There was a problem hiding this comment.
While adding self.format_tools(tools) correctly addresses the tool formatting issue, this method contains several other critical bugs that will likely prevent it from functioning as intended:
- AttributeError: Line 37 calls
self.format_input_message(messages), but the base class method is named_format_input_message(with a leading underscore). This will cause a runtime error. - Logic Error: Line 38 incorrectly sets the
stopvariable toNonebecauselist.append()operates in-place and returnsNone. Consequently, thestop=stopargument passed on line 40 will always beNone, effectively losing all intended stop sequences. - Redundancy: The
_call_llmmethod already handles message formatting internally (via_format_input_message), making the call on line 37 unnecessary even if the name were correct. - Missing State Update: Unlike the base class implementation in
OpenAI, this override fails to incrementmessages[-1].api_calls, which may affect tracking or logic dependent on the number of API attempts.
I recommend refactoring this method to address these issues alongside the tool formatting fix.
|
Thanks for the thorough review. All four findings are technically correct, but they identify pre-existing bugs in a class that's dead code: The Our one-line change (adding The real fix this PR delivers is in |
When finish_reason=length with tool_calls present, _continue_generate and _stream_continue_generate append the partial message (with tool_calls) to history then immediately call the LLM for continuation without executing the tools. The resulting messages list has assistant tool_calls with no corresponding tool responses, causing: openai.BadRequestError: insufficient tool messages following tool_calls message Fix: return early when tool_calls are present, letting the normal step() tool-execution loop handle the message. - _continue_generate: return new_message immediately if tool_calls - _stream_continue_generate: skip continuation if tool_calls present Fixes modelscope#909
Fix 1: Missing
typefield in continue-gen tool callsProblem:
_call_llm_for_continue_genpasses rawToolTypedDicts directly to_call_llmwithout formatting first. TheToolTypedDict has notypefield — it gets added byformat_tools(). When a response hitsfinish_reason: lengthand enters the continue-generation path, the unformatted tools cause:Fix: Added
self.format_tools(tools)in_call_llm_for_continue_genin bothopenai_llm.pyanddeepseek_llm.py.Fix 2: Orphaned tool_calls causing "insufficient tool messages" error (fixes #909)
Problem: When
finish_reason: lengthfires withtool_callspresent in the truncated response,_continue_generateand_stream_continue_generateappend the partial assistant message (containing tool_calls) to the message history, then immediately call the LLM for continuation without executing the tools. The resulting messages list has assistanttool_callsbut no corresponding tool responses, causing:This causes sub-agents (e.g., Reporter) to crash after 3 retries, leading to infinite re-spawn loops.
Fix: Return early when
tool_callsare present, letting the normalstep()tool-execution loop handle the truncated message:_continue_generate(line 526):if new_message.tool_calls: return new_message_stream_continue_generate(line 346): addnot message.tool_calls andguard before the finish_reason continuation checkTesting
Tested both fixes with
deepseek-v4-proon thedeep_research/v2pipeline. Without the fixes, the pipeline reliably hits both errors. With both fixes applied, the pipeline completes without errors.