Skip to content

Bug: Systemic parts[0] indexing drops multimodal streaming data and bypasses content validation #6616

Description

@Harshitmishra001

🔴 Required Information

Describe the Bug:
The framework makes unsafe parts[0] indexing assumptions in the core Gemini Live API connection (gemini_llm_connection.py) and flow modules (base_llm_flow.py). This causes two critical systemic bugs:

  1. Validation Bypass: Mixed-content blocks (e.g., text followed by a tool response) bypass ValueError validation entirely and are incorrectly routed as normal client content.
  2. Dropped Multimodal Streaming: The Live API streaming receiver silently drops subsequent parts in multi-part streaming chunks (e.g., mid-chunk thought boundary transitions or inline data) because it only extracts data from content.parts[0].

Steps to Reproduce:
Please provide a numbered list of steps to reproduce the behavior:

  1. Initialize a GeminiLlmConnection.
  2. Construct a types.Content object with multiple parts, where the first part is text and the second part is a types.FunctionResponse.
  3. Call await gemini_connection.send_content(content).
  4. Observe that the content is routed to the standard send() pipeline instead of correctly raising a ValueError.

Expected Behavior:

  1. Sending a mixed-content block containing a function response should strictly trigger the validation ValueError("Function-response content cannot mix function and non-function parts.") regardless of index.
  2. The Live API receive() generator should iterate over all parts in content.parts to ensure no multimodal data or mid-chunk transitions are dropped.

Observed Behavior:

  1. The framework explicitly checks if content.parts[0].function_response, bypassing validation entirely if the text part comes first.
  2. The receive() loop hardcodes content.parts[0], permanently ignoring any subsequent parts sent in the same chunk.

Environment Details:

  • ADK Library Version: main branch
  • Desktop OS: N/A (Architecture bug)
  • Python Version: 3.11+

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: gemini-3.1-flash-live-preview

🟡 Optional Information

Regression:
N/A - This appears to be a systemic issue tied to early architectural assumptions that chunks and payloads would only ever contain a single part.

Logs:

N/A - Issue prevents proper error logs from being emitted due to silent data drops/misrouting.

Screenshots / Video:
N/A

Additional Context:
While it might be rare in current API responses for certain models to stream multiplexed chunks, the Gemini protocol does not prohibit it. The API could technically stream multiplexed chunks (e.g., an image with text, or multiple tool responses at once). By removing these fragile parts[0] assumptions, the framework becomes robust against future Live API updates.

Minimal Reproduction Code:

import pytest
from google.genai import types

# Example of the Mixed Content Validation Bypass
@pytest.mark.asyncio
async def test_mixed_content_bypass(gemini_connection):
    function_response = types.FunctionResponse(name='test_function', response={'result': 'success'})
    content = types.Content(
        role='user',
        parts=[
            types.Part.from_text(text='Hello'),
            types.Part(function_response=function_response),
        ],
    )

    # EXPECTED: ValueError("Function-response content cannot mix...")
    # OBSERVED: Silently misroutes to standard send() pipeline.
    await gemini_connection.send_content(content)

How often has this issue occurred?:

  • Always (100%) - Occurs reliably whenever the framework encounters a multi-part array where the target data is not at index 0.

Metadata

Metadata

Labels

core[Component] This issue is related to the core interface and implementationrequest clarification[Status] The maintainer need clarification or more information from the author

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions