feat(voice): delegate reasoning and tools to a second LLM - #6602
Draft
longcw wants to merge 1 commit into
Draft
Conversation
An agent can split into a conversation model that talks to the user and a delegation LLM that does the reasoning and tool work. The conversation model is offered a single builtin `lk_agents_delegate` tool; every other tool goes to the delegation LLM unless flagged NO_DELEGATE. Delegation is expressed as an ordinary tool call, so it reuses the async tool lifecycle rather than adding a parallel one: tracing, duplicate handling, cancellation, and the deferred reply that carries the answer back. Delegated tools run on a session-scoped executor and their progress is re-attributed to the delegate call, so each side only sees tool traffic for its own tools.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Splits an agent into a conversation model that owns the dialogue and a
delegation_llmthat does the reasoning and tool work and never speaks. Realtime and low-latency models are good at conversation and weak at tool calling; frontier text models are the reverse, and today an agent has to pick one and accept the other half.How it works
Delegation is an ordinary tool call β one builtin
lk_agents_delegateβ so it reuses the async tool lifecycle rather than adding a parallel one: tracing, duplicate handling, cancellation, and the deferred reply that carries the answer back.The call releases the turn immediately, and its dispatch note being answered is what acknowledges to the user; the answer arrives later as its own reply. Relying on the model to speak alongside the call does not work β realtime models routinely emit the call and no speech.
Each side only sees tool traffic for its own tools. The delegation's calls stay out of the conversation entirely, and the
delegatepairs stay out of the delegation's context, so neither model reads a tool it cannot call.API
AgentSession(delegation_llm=..., delegation_options=...), same onAgent, agent wins per keyAgent.delegation_node(task, ctx, chat_ctx, tools, model_settings)overrides the loop; returns the answer, orNoneto answer without speakingToolFlag.NO_DELEGATEkeeps a latency-critical tool on the conversation modelRunContext.update(..., silent=True)releases a tool without generating a replyBehaviour changes outside delegation
_deliver_replyno longer generates a reply when the model hasauto_tool_reply_generationβ it continues on its own. This affects async tools on google, aws, phonic and ultravox, and is worth a manual check on those providers.CANCELLABLEandNO_DELEGATE; the combination is contradictory and now raises at decoration time.