Skip to content

fix(semconv): dual-emit OTel GenAI input/output token usage names - #1453

Open
ZxlDragonDoctor wants to merge 3 commits into
AgentOps-AI:mainfrom
ZxlDragonDoctor:fix/dual-emit-otel-genai-usage-names
Open

ZxlDragonDoctor wants to merge 3 commits into
AgentOps-AI:mainfrom
ZxlDragonDoctor:fix/dual-emit-otel-genai-usage-names

Conversation

@ZxlDragonDoctor

Copy link
Copy Markdown

Problem

AgentOps still emits the older GenAI semconv usage names (gen_ai.usage.prompt_tokens / completion_tokens) only. Current OTel GenAI conventions use gen_ai.usage.input_tokens / output_tokens. In a collector that mixes AgentOps with OpenLLMetry / pydantic-ai spans, sum(input_tokens) misses AgentOps and sum(prompt_tokens) misses the others.

Fixes the transition gap described in #1447. Addresses the migration question: this PR dual-emits both families instead of a breaking rename, so existing AgentOps dashboards keep working while generic OTel consumers can sum the current names.

Solution

  • Add SpanAttributes.LLM_USAGE_INPUT_TOKENS / LLM_USAGE_OUTPUT_TOKENS.
  • In TokenUsage.to_attributes(), emit the current OTel names alongside the legacy names when prompt/completion counts are present.
  • Keep zero-suppression behavior unchanged (no empty usage fields on non-LLM spans).
  • Extend unit coverage for the dual-emit path.

No call-site rewrite in this PR — every existing setter of the legacy constants benefits automatically.

Testing

  • Offline assertions mirroring TokenUsage.to_attributes() dual-emit logic: token_usage_dual_emit: 3 passed
  • Extended tests/unit/instrumentation/common/test_token_counting.py:
    • zero values still produce {}
    • positive prompt emits both prompt_tokens and input_tokens
    • completion emits both completion_tokens and output_tokens

Agent dimension

evaluation

Core value

Let mixed-SDK OTel collectors sum AgentOps token usage with the current GenAI attribute names without breaking existing prompt/completion consumers.

Fixes #1447

@roy-tong roy-tong left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — this implements the plan from #1447 cleanly, and the test asserting both families (and the negative case) is exactly right. Approving with three non-blocking notes:

  1. Double-count hazard should be stated where a future reader will meet it. The code comment says collectors "can sum either family" — the sharper warning is that a consumer summing both families counts twice. One sentence in the span_attributes.py comment block ("consumers must prefer the new names; legacy names are compatibility aliases — summing both double-counts") would prevent the predictable dashboard bug.

  2. Internal consumers worth a quick check. If anything on the ingestion side iterates gen_ai.usage.* attributes and sums matches, dual-emit doubles it internally the same way. The unit tests cover to_attributes()'s output shape, which suggests no such iteration exists — just worth confirming once.

  3. Removal horizon, even soft. "During the transition (#1447)" is good; if there's a version where the legacy names are planned to drop, naming it here (or in a tracking issue linked from this comment) gives dashboard owners a date to migrate by. The failure mode is silent: dashboards keyed on prompt_tokens break quietly the day the alias is removed.

One forward-looking note, not for this PR: cache_creation_input_tokens / cache_read_input_tokens above are the same generation of legacy names on the cache side — the current conventions nest them (gen_ai.usage.cache_read.input_tokens etc.), so the same dual-emit pattern will be needed there eventually, and it's the exact alias-drift shape that bit other SDKs' extractors (langfuse#17117 for one). The regression shape that guards the whole family: same logical call, two attribute names, one expected total.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

old semconv attribute names (prompt_tokens vs input_tokens) — cross-SDK aggregation gap

2 participants