feat: add OpenAI Realtime support to live runs

[GitMarco27]

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

• Closes: extend google ADK live capabilities with openai realtime api #2719
• Related: realtime api from litellm or openai #1045

Description of Change

Problem:

The experimental OpenAILlm integration supports regular Chat Completions requests but cannot be used with ADK's standard Runner.run_live() flow.

Applications that need bidirectional voice or text streaming with OpenAI Realtime models therefore require provider-specific connection code outside ADK's standard live architecture.

Solution:

Implement OpenAILlm.connect() using the official OpenAI Python SDK Realtime connection and add an ADK BaseLlmConnection adapter for the OpenAI Realtime API.

The adapter:

• uses the existing ADK Runner.run_live() and LiveRequestQueue;
• supports bidirectional audio and text streaming;
• maps input and output transcriptions to ADK live events;
• handles server VAD, activity events, interruptions, and NO_INTERRUPTION;
• translates function calls and parallel tool responses while preserving their ordering;
• propagates usage metadata, completion status, provider errors, and EOF;
• supports an injected AsyncOpenAI client or asynchronous API key provider;
• excludes credentials and injected clients from Pydantic serialization and representation;
• documents file-based PCM streaming, supported configurations, and current limitations.

This initial implementation targets the public OpenAI API only. Azure OpenAI Realtime and LiteLLM Realtime proxies are intentionally outside the scope of this PR.

Usage example

OpenAI Realtime uses the standard ADK live flow; no provider-specific runner is required. The existing App, Runner, and session setup remains unchanged.

import asyncio

from google.genai import types

from google.adk.agents import Agent
from google.adk.agents import LiveRequestQueue
from google.adk.agents import RunConfig
from google.adk.agents.run_config import StreamingMode
from google.adk.labs.openai import OpenAILlm

agent = Agent(
    name="voice_agent",
    model=OpenAILlm(model="gpt-realtime"),
    instruction="You are a concise voice assistant.",
)

run_config = RunConfig(
    streaming_mode=StreamingMode.BIDI,
    response_modalities=[types.Modality.AUDIO],
    output_audio_transcription=types.AudioTranscriptionConfig(),
)


async def send_microphone_audio(queue: LiveRequestQueue) -> None:
  async for pcm_chunk in microphone_stream():
    queue.send_realtime(
        types.Blob(
            data=pcm_chunk,
            mime_type="audio/pcm;rate=24000",
        )
    )

  queue.send_audio_stream_end()


async def run_voice_session(runner) -> None:
  queue = LiveRequestQueue()
  sender = asyncio.create_task(send_microphone_audio(queue))

  try:
    async for event in runner.run_live(
        user_id="user",
        session_id="session",
        live_request_queue=queue,
        run_config=run_config,
    ):
      if event.output_transcription and event.output_transcription.text:
        print(event.output_transcription.text, end="", flush=True)

      if event.turn_complete:
        break
  finally:
    queue.close()
    await sender

Here, microphone_stream() represents an application-provided asynchronous source of PCM16 mono 24 kHz audio.

A complete prerecorded-file streaming example, including App, Runner, and session setup, is available in the google.adk.labs.openai README.

Testing Plan

Unit Tests:

• I have added or updated unit tests for my change.
• All unit tests pass locally.

Test results:

• uv run pytest tests/unittests/labs/openai -q
  • 104 passed
• OpenAI SDK 2.20 compatibility run:
  • 53 passed
• Focused tox runs on Python 3.10, 3.11, 3.12, 3.13, and 3.14:
  • 53 passed on every Python version
• All pre-commit hooks pass for the changed files.
• Package build and clean-wheel import tests pass.

The full tests/unittests suite was also run across Python 3.10–3.14. All new and OpenAI-related tests pass.

Each environment reports the existing failure:

tests/unittests/cli/utils/test_cli_tools_click.py::test_telemetry_cli_commands

The same failure was reproduced on an unmodified checkout of the current upstream main, where the command exits with status 2 instead of the status 0 expected by the test. It is therefore unrelated to this change and is not
modified as part of this PR.

One unrelated test_load_web_page_blocks_file_scheme_urls failure occurred once during the parallel Python 3.13 run and passed immediately when rerun in isolation.

Manual End-to-End (E2E) Tests:

1. Set OPENAI_API_KEY.
2. Create a Runner with OpenAILlm(model="gpt-realtime").
3. Configure a bidirectional audio run with input/output transcription and automatic activity detection disabled for deterministic prerecorded-file streaming.
4. Stream a PCM16 mono 24 kHz recording in real-time-paced 20 ms chunks.
5. Send audio_stream_end and consume events until turn_complete.
6. Verify the input transcript, output transcript, returned audio, and absence of provider errors.

Observed result:

E2E_SUMMARY={
  "model": "gpt-realtime",
  "events": 42,
  "turn_complete": true,
  "input_transcript": "Please answer in Italian by saying: test voice completed successfully.",
  "output_transcript": "Test voce completato con successo.",
  "output_audio_bytes": 235200,
  "errors": []
}

The returned audio was verified as PCM16 little-endian, mono, 24 kHz, with a duration of 4.9 seconds.

Checklist

• I have read the CONTRIBUTING.md document.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have manually tested my changes end-to-end.
• Any dependent changes have been merged and published in downstream modules. (N/A: no dependent changes.)

Additional context

The implementation is intentionally contained in the existing experimental google.adk.labs.openai package and does not introduce a provider-specific runner.

This PR fully addresses #2719. It also addresses the public OpenAI portion of #1045; LiteLLM Realtime proxy support remains out of scope.

Current documented limitations:

• Azure OpenAI Realtime is not supported.
• LiteLLM Realtime proxies are not supported.
• ADK session resumption is not mapped to OpenAI Realtime sessions.
• Exact server-side audio truncation on interruption is unavailable because the integration does not know how much generated audio the client has already played.