Skip to content

Tool costs are recorded with no currency — attach a unit to gen_ai.usage.total_cost #1462

Description

@dhyabi2

The per-trace cost has no unit of account

AgentOps records a monetary cost for LLM/tool operations on spans, e.g. when a
decorated tool sets a cost:

# agentops/sdk/decorators/factory.py:392 (and 411, 432, 464)
span.set_attribute(SpanAttributes.LLM_USAGE_TOOL_COST, cost)

with the attribute key:

# agentops/semconv/span_attributes.py:68
LLM_USAGE_TOOL_COST = "gen_ai.usage.total_cost"

The value is a float produced from the tokencost price tables
(app/opentelemetry-collector/builder/costs/__init__.py models cost as a Decimal
"for precise currency calculations", loading per-token USD prices from
model_prices.json). So today the number is always implied USD — but the span
carries no unit of account. Anyone who self-hosts AgentOps, ingest traces billed
in a non-USD currency, or sinks spans into a downstream cost dashboard or billing
reconciliation step cannot tell what currency the number is in, and cross-account
or cross-deployment cost comparison is impossible. It is observability data with
no currency attached to it.

The standard direction

The OpenTelemetry GenAI semantic-conventions working group is standardizing
exactly this gap (there is no OTel convention for GenAI cost today; each vendor
rolls its own shape). The open proposal
add per-operation cost conventions (gen_ai.usage.cost.*) #443
defines:

  • gen_ai.usage.cost.amount (double) — the monetary cost at recording time
  • gen_ai.usage.cost.currency (string) — the ISO 4217 code, conditionally required when amount is set
  • gen_ai.usage.cost.source (enum provider | pricing_table | estimate) — provenance

So the attribute a span should carry beside its cost amount is a currency
companion, and the natural place to add it is the semantic-conventions store where
LLM_USAGE_TOOL_COST already lives.

Proposed change (small, additive)

  1. In agentops/semconv/span_attributes.py next to LLM_USAGE_TOOL_COST (line 68):
    # currency companion, aligned with OTel GenAI semconv PR #443
    LLM_USAGE_TOOL_COST_CURRENCY = "gen_ai.usage.cost.currency"
  2. In agentops/sdk/decorators/factory.py, on the line after each of the four
    LLM_USAGE_TOOL_COST set sites (392, 411, 432, 464), set the companion where a
    cost is set:
    span.set_attribute(SpanAttributes.LLM_USAGE_TOOL_COST_CURRENCY, "USD")
    (USD because the tokencost price tables price in USD today.)
    Keep the cost value itself and the charged amount untouched.
  3. A focused test asserting that a tool span with a cost also carries the
    currency attribute equal to "USD"; the existing test suite stays green.

Why it helps

  • The cost number becomes interpretable: it names its unit of account.
  • Aligns the vendor extension with the emerging OTel GenAI standard, where
    currency is required whenever an amount is set.
  • Lets a self-hosted operator attribute cost in any ISO 4217 currency (USD today,
    others later) without changing the amount logic — a property that matters to any
    per-trace cost-accounting or billing workflow that consumes AgentOps spans.

Contained to the decorator tool path, additive, one attribute key, no change to
the cost value or existing behavior. Happy to open a PR with the test if this
scope looks right.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions