Skip to content

Add managed OTLP trace export for Codex - #616

Open
tt-le wants to merge 1 commit into
tien/managed-otel-tracesfrom
tien/codex-otel-tracing
Open

tt-le wants to merge 1 commit into
tien/managed-otel-tracesfrom
tien/codex-otel-tracing

Conversation

@tt-le

@tt-le tt-le commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #599 (base tien/managed-otel-traces) — this diff is just the one Codex commit.

What

Extend the OTel tracing rail to Codex. When the managed CodingAgentConfig enables tracing for the codex agent (AgentConfig.tracing_config.enabled), ug codex exports OTLP trace spans to the AI Gateway ingest endpoint.

How

  • managed_resolve: OTEL_TRACING_TOOLS = ("claude", "codex") — the per-agent flag now sets codex_otel_tracing in state (same rail as Claude).
  • codex: at launch, codex.py mints a fresh Databricks bearer (reusing the OAUTH_TOKEN mint) and injects the [otel.trace_exporter.otlp-http] block — endpoint (full /v1/traces path, used verbatim), protocol = "binary" (http/protobuf), and Authorization: Bearer — as -c/--config overrides. Nothing is persisted; the token is minted per launch.

Auth model (verified against openai/codex source)

Codex keeps inference and telemetry auth fully separate (model-provider-info/src/lib.rs vs otel/src/config.rs):

  • Inference: [model_providers.<n>.auth] command-backed bearer (refreshing).
  • OTel: static [otel...headers] map only — no headers-helper, no refresh, no reuse of the inference cred.
    Both use the same Databricks workspace bearer for our gateway; they're just configured in two places.

Known limitation (stopgap)

Because Codex has no otelHeadersHelper equivalent, the bearer is pinned statically per launch — it goes stale after ~1h, and (via -c) is visible in the process command line. This is an interim manual-testing path. The durable fix is a local refresh proxy (the OTLP analog of ucode mcp-proxy / gateway_proxy.py): point trace_exporter at a loopback URL and refresh the bearer server-side, keeping it fresh and out of argv. Planned as a follow-up.

Validate

  • uv run ruff check . + format clean; tests/test_agent_codex.py + tests/test_managed_resolve.py pass (199 focused). Added: codex in the opt-in list, and launch injects/omits the [otel] -c args by the codex_otel_tracing flag.
  • End-to-end: a per-agent tracing_config.enabled codex stub resolves to codex_otel_tracing and produces the correct --config otel={...} launch arg (endpoint + binary + Bearer).
  • Pre-existing unrelated failures (verified on the base branch): the test_user_agent_arrives_at_gateway e2e probes and one smart-routing pty test.

This pull request and its description were written by Isaac.

@tt-le
tt-le force-pushed the tien/codex-otel-tracing branch from 387f433 to 78ad7a7 Compare September 15, 2026 13:42
@tt-le
tt-le force-pushed the tien/managed-otel-traces branch from 082dd0e to e0324d4 Compare September 15, 2026 16:32
Extend the OTel tracing rail (stacked on the Claude PR) to Codex: add `codex` to
`OTEL_TRACING_TOOLS`, so a managed per-agent `tracing_config.enabled` for the codex agent
sets `codex_otel_tracing` in state. At launch, `codex.py` mints a fresh Databricks bearer and
injects the `[otel.trace_exporter.otlp-http]` block (endpoint + http/protobuf + Authorization)
as `-c` config overrides — the endpoint is the AI Gateway ingest path, minted per launch and
never persisted.

Codex has no OTLP headers-helper (unlike Claude's otelHeadersHelper), so the bearer is pinned
statically per launch and goes stale after ~1h; a refreshing local proxy is the intended
follow-up. Inference and OTel auth are independent in Codex (model_providers.auth vs
otel.headers) and both use the same workspace bearer.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant