Skip to content

fix(flows): keep non-text static_instruction as a stable request prefix - #6653

Open
chelsealong wants to merge 2 commits into
google:mainfrom
chelsealong:fix-static-instruction-context-cache-prefix
Open

fix(flows): keep non-text static_instruction as a stable request prefix#6653
chelsealong wants to merge 2 commits into
google:mainfrom
chelsealong:fix-static-instruction-context-cache-prefix

Conversation

@chelsealong

Copy link
Copy Markdown

Fixes #6652

Bug

When LlmAgent.static_instruction contains non-text content (e.g. a PDF
provided through file_data), ADK extracts that content into a "user"
content entry (via LlmRequest.append_instructions) so it can be sent as
part of contents.

On the first turn this lands at the front of the request as expected. On
the second and later turns, _add_instructions_to_user_content in
contents.py inserts all instruction-related contents (the static
non-text content and the dynamic instruction) right before the last
continuous batch of user content — i.e. after the existing conversation
history rather than at the front of the request:

previous user message
previous model response
static PDF content        <-- moved behind history
dynamic instruction
current user message

This defeats the "stable prefix" that provider-side implicit context
caching relies on, so a large static PDF never gets a cache hit past the
first turn, as described in #6652.

Fix

LlmRequest.append_instructions now tracks the user contents it extracts
from a Content argument (currently only used for static_instruction)
in a new private list, _static_instruction_contents, in addition to
appending them to contents as before.

_add_instructions_to_user_content in contents.py checks this list: if
it's non-empty, the full set of instruction-related contents (static
content followed by the dynamic instruction, in the order they were
built) is inserted ahead of conversation history instead of before the
last user batch. This keeps the static content — and the dynamic
instruction that follows it — as a stable prefix ahead of conversation
history:

static PDF content
dynamic instruction
previous user message
previous model response
current user message

When static_instruction has no non-text parts (the common case, and the
existing behavior for pure dynamic-instruction agents), nothing changes:
instructions still get inserted right before the latest user turn.

_add_instructions_to_user_content actually has a second call site:
_finalize_dynamic_instructions in base_llm_flow.py, reached when the
experimental DYNAMIC_INSTRUCTION_ROUTING feature is on and a tool (e.g.
preload_memory_tool, load_artifacts_tool, load_mcp_resource_tool)
contributes a dynamic instruction via _append_dynamic_instructions. That
call happens later in the same turn's preprocessing, after the static
prefix has already been placed by the first call. An earlier version of
this fix inserted at a fixed index 0 on every call, which meant this
second call re-inserted at the front too, pushing the tool's dynamic
content in front of the static prefix the first call had just placed —
recreating the exact bug this PR fixes, one call later.

To fix that, LlmRequest now tracks
_static_instruction_prefix_end_index, the index right after the
inserted static-instruction prefix. Only the first call inserts at index
0 (and records where the prefix ends); any later call for the same
request inserts right after that tracked index instead, so tool-triggered
dynamic instructions land after the static prefix without displacing it.

Testing plan

Added test_static_instruction_file_precedes_multi_turn_history in
tests/unittests/flows/llm_flows/test_instructions.py, which reproduces
the multi-turn scenario from the issue (static file_data instruction +
dynamic instruction + 3 turns of history) and asserts the static content
and dynamic instruction both precede the history.

Added test_static_instruction_file_stays_prefix_after_tool_dynamic_instruction,
which reproduces the second-call-site regression: after the main content
processor places the static prefix, it simulates a tool-triggered dynamic
instruction the way _finalize_dynamic_instructions does, and asserts the
static content is still at index 0 afterward.

Verified both new tests fail without their respective fixes:

$ git checkout HEAD~1 -- src/google/adk/flows/llm_flows/contents.py src/google/adk/models/llm_request.py
$ python -m pytest tests/unittests/flows/llm_flows/test_instructions.py::test_static_instruction_file_precedes_multi_turn_history
AssertionError: assert 'First message' == 'Referenced f...: file_data_0'

$ git checkout HEAD~2 -- src/google/adk/flows/llm_flows/contents.py src/google/adk/models/llm_request.py
$ python -m pytest tests/unittests/flows/llm_flows/test_instructions.py::test_static_instruction_file_stays_prefix_after_tool_dynamic_instruction
AssertionError: assert 'Relevant mem...r likes pizza' == 'Referenced f...: file_data_0'

With the fix:

$ git checkout HEAD -- src/google/adk/flows/llm_flows/contents.py src/google/adk/models/llm_request.py
$ python -m pytest tests/unittests/flows/llm_flows/test_instructions.py
38 passed
$ python -m pytest tests/unittests/flows/llm_flows/ tests/unittests/models/test_llm_request.py
562 passed, 1 xfailed
$ python -m pytest tests/unittests/flows/
529 passed, 1 xfailed
$ python -m pytest tests/unittests/tools/ -k "preload_memory or load_artifacts or load_mcp_resource"
24 passed

Also verified formatting with pyink and import order with isort
(project's pinned versions) — no changes needed.

AI assistance disclosure

This change was authored with the help of an AI coding agent (Claude),
with the diff reviewed and the reproduction/test verified before being
pushed. An earlier version of this PR was caught by review as fixing the
bug in the main content-processor call path while leaving a second call
path (tool-triggered dynamic instructions under the experimental
DYNAMIC_INSTRUCTION_ROUTING feature) able to re-break the same
invariant; that gap has been closed and covered by a new regression test.

On the second and later turns, static_instruction content containing
non-text parts (e.g. a PDF via file_data) was inserted after existing
conversation history instead of staying at the front of the request.
This broke the stable prefix that provider-side implicit context
caching relies on.

Track the user contents extracted from a Content passed to
append_instructions (currently only static_instruction) separately, and
insert them at the very beginning of the request, ahead of both the
dynamic instruction and the conversation history.

Fixes google#6652
…nd regression

_add_instructions_to_user_content had a second call site
(_finalize_dynamic_instructions in base_llm_flow.py), reached when the
experimental DYNAMIC_INSTRUCTION_ROUTING feature is on and a tool (e.g.
preload_memory_tool) contributes a dynamic instruction. Because the
guard added in the previous commit checked a flag that is never reset,
that second call also inserted at index 0, pushing the tool's content
in front of the static prefix the first call had just placed there --
defeating the fix.

Track the index right after the inserted static-instruction prefix and
have subsequent calls insert there instead of at index 0, so later
tool-triggered instructions land after the prefix without displacing
it.
@adk-bot adk-bot added the core [Component] This issue is related to the core interface and implementation label Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core [Component] This issue is related to the core interface and implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

static_instruction and instruction are moved behind conversation history when static_instruction contains non-text content

3 participants