[yang0228]

Problem and resulting behavior

Closes #7351. One Gemini Live generation should count as one inference, regardless of text/audio chunking, thought-to-answer transitions, or text flushed before a tool call. A tool call followed by its response and another model answer counts as two; an interrupted generation followed by a new generation also counts as two. The evaluation owner confirmed these semantics in the issue discussion.

The original reproducer gives 1 / 2 / 3 for identical answers split into 1 / 2 / 3 chunks on current main. This PR gives 1 / 1 / 1, while retaining separate genuine requests and token usage.

Implementation and upstream integration

• Reconstruct the count from raw session events before projection removes completion markers. Track each author/Live connection independently, using completion, interruption, tool response, and typed client-message boundaries.
• Preserve transcription connection identity and stamp consolidated final text with model_version, so histories that omit partial chunks still count.
• Preserve legacy event-based fallback for missing/None counts; respect an explicit zero.
• The prerequisite fix(eval): preserve live usage without extra inference calls #7323 has landed via Copybara as 8cd3b12.
• Refreshed with upstream main 1cb926d5 in ab9eb64f, preserving published history. The landed usage-matching algorithm is retained; counting tracks only the surviving source events. Tests now call the current Live receiver module. The remaining diff is seven files, with no newly added source modules.

All upstream and PR regression tests were retained. Coverage includes typed client messages, 1/2/3 text and audio chunks, thought/answer and text/tool transitions, interruption/barge-in, separate authors/connections, late or standalone usage, empty/non-text content, and saved-JSON fallback/round trips.

Saved eval JSON: owner decision still requested

The optional Invocation.inference_call_count field preserves a count that cannot be recovered reliably after raw event boundaries are removed. New readers accept older eval JSON with the field absent. Newly saved JSON containing this field is rejected by older strict readers (extra="forbid"). Please confirm whether this saved-schema compatibility boundary is acceptable; it has not been treated as approved.

Fresh validation on ab9eb64 — 2026-11-08

A new Python 3.12.13 environment was provisioned under current main's constraints-3.12.txt: google-genai==2.30.0, Agent Platform/AI Platform 2.3.0. Dependency compatibility checks passed.

• Source tree: 626 passed, 1 xfailed across the modules below, including the entire live/ test directory.
• Independently built and installed wheel: 626 passed, 1 xfailed for the same tests copied outside the checkout. Confirmed ADK imports came from the new environment's site-packages.
• Original issue reproducer: unchanged main 1cb926d5 gives 1/2/3 and fails; this PR gives 1/1/1 and passes.
• Configured pre-commit checks pass for all seven changed files. The unit-guide check uses its documented NO_UNIT_GUIDE reason for modules imported by the upstream merge; this PR adds no source modules. The reason is recorded in the merge commit.
• Mypy 2.3.0, targeting the tested Python 3.12 runtime: no issues in the five changed production modules, on both unchanged main and this PR. The repository's default Python 3.11 target is blocked on both by the installed NumPy stub's Python 3.12-only syntax.

PYTHONPATH="$PWD/src" python -m pytest -p no:cacheprovider -q \
  tests/unittests/evaluation/test_evaluation_generator.py \
  tests/unittests/evaluation/test_efficiency_evaluators.py \
  tests/unittests/evaluation/test_eval_case.py \
  tests/unittests/models/test_gemini_llm_connection.py \
  tests/unittests/live \
  tests/unittests/flows/llm_flows/test_base_llm_flow_realtime.py \
  tests/unittests/flows/llm_flows/test_base_llm_flow_partial_handling.py
# 626 passed, 1 xfailed

These are focused validation results using deterministic local transport with real ADK components. The full unit suite, other Python versions, and remote Gemini E2E/billing behavior were not revalidated in this refresh. Remote CI still needs maintainer approval.

Checklist

• Read CONTRIBUTING.md and link the issue.
• Preserve the landed usage fix and resolve upstream conflicts.
• Retain requested regressions and complete fresh source/wheel validation.
• fix(eval): preserve live usage without extra inference calls #7323 has landed.
• Evaluation owner accepts the optional saved-schema field.
• Maintainer approves workflow runs and completes review.