Repository navigation
Recognize native Codex and Claude Code telemetry in Agents - #33
Merged
Merged
Conversation
Codex and Claude Code export OTLP traces with their own attribute names, so the Agents view either skipped their datasets (Codex) or showed runs without tool names or token usage (Claude Code). Detect both harnesses from the dataset schema and derive the GenAI span fields the view reads: the turn becomes the agent span, model calls and tool calls get their operation, model, tool name and usage. Claude Code input usage includes cache reads and writes. Recorded values are never replaced, and the details panel still shows the fields as recorded. Codex traces a turn with hundreds of internal spans, so the run query keeps only turn, model-call and tool spans from Codex services, and the mapping attaches model and tool calls to their turn. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Reviewed 0febb045707ee9be014ee8ab71cdf2f0850d1b51 with two GPT-6.1-Sol and two Claude Opus 5.5 reviewers, each covering a different area. Actionable findings are in the threads below.
What holds up: datasets with only generic OTel GenAI telemetry get byte-identical SQL and untouched rows. Codex usage lifting from handle_responses matches the turn totals, recorded values are never overwritten, "Recorded fields" and "Parent span ID" read the original rows, and the turn total isn't double-counted. agentHarnesses.test.ts passes. The edge cases were reproduced with scratch scripts against the PR's functions; Playwright and live ingestion were not rerun.
…afely Address review findings on native Codex and Claude Code telemetry: - Detect every harness in a dataset. Claude Code mapping always runs, and Codex mapping, dropping and reparenting apply only to rows from Codex services (or, without a service name, to a Codex-only dataset). - Keep Codex spans that already carry standard GenAI fields, such as tool spans whose tool_name a collector renamed to gen_ai.tool.name. - Attach a detached Codex call to a turn only when it started during exactly one loaded turn, so calls of an unexported turn stay detached and multi-turn traces attach each call to its own turn. - Rewrite whichever parent field a row uses (span_parent_span_id and/or parent_span_id). - Leave Claude Code input usage unknown unless input_tokens parses. - Show the "Mapped from native telemetry" note only on rows something was derived for, and drive the record-limit banner from the queried row count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Map Codex rows in place among all loaded rows so a turn whose parent belongs to another service keeps that parent. Keep Codex stream and response spans that carry GenAI agent fields themselves, as the run query already does, and lift usage only from the ones that are dropped. Treat rows without a service name as Codex unless they are Claude Code spans, so mixed datasets without service names still map Codex turns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Codex response span kept for its own GenAI fields lost its dropped stream parent and was attached to the turn beside its model call, so its usage was counted twice. Walk dropped parents up to the nearest kept ancestor first. In mixed datasets, take a row without a service name to be Codex only if it has a Codex span name or a Codex tool call, so generic spans keep their recorded parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VibeSWE
added a commit
that referenced
this pull request
Oct 13, 2026
…ness-conversation-logs Bring in the review fixes from #33. Harness detection now returns a list; conversation logs stay joined for a single detected harness until the page handles mixed datasets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Codex and Claude Code are the coding agents people most often point at Parseable, but both export OTLP traces with their own attribute names:
gen_ai.agent.*,gen_ai.operation.name,gen_ai.request.modelorgen_ai.tool.name, so discovery skipped it.gen_ai.request.model. Tool calls had no name, token usage showed as not recorded, and the run was named after the raw span (claude_code.interaction).This PR teaches the Agents view to recognize both harnesses and read their native fields.
What changes
lib/agentHarnesses.ts): a dataset is recognized from schema fields specific to each harness (codex.turn.token_usage.input_tokens/codex.turn.phase,interaction.sequence/llm_request.context), together withspan_name. One dataset can hold both harnesses; each row is mapped by its own harness (Codex rows are those whoseservice.namestarts withcodex). Logs datasets are still not listed.session_task.turnbecomes the agent span,try_run_sampling_requestthe model calls, and spans withtool_namethe tool calls. Per-request usage is lifted fromhandle_responsesto its model call.claude_code.interactionbecomes the agent span,claude_code.llm_requestthe model calls (input usage =input_tokens + cache_read_tokens + cache_creation_tokens, left unknown wheninput_tokensis missing or malformed), andclaude_code.toolthe tool calls (tool_name).span_parent_span_idorparent_span_id) is rewritten.Evidence
The fixtures are trimmed from real exports into a local Parseable: Codex CLI 0.162.1 (app server and
codex exec) and Claude Code 2.1.296 withCLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1. Running this branch's query and mapping against that data produced:codex-tracesexectool; tokens match Codex's own turn totals (e.g. 32,558 in / 38 out); 28 rows loaded instead of thousandsclaude-code-tracesBashtool; 42,554 in / 154 out including cache tokenscodex-logsTesting
npm run format:check,tsc --noEmit,vite buildnpm test: 2,350 passed, 14 skipped.agentHarnesses.test.tscovers detection (including mixed datasets), SQL (including that absent columns are never referenced and GenAI-field spans are kept), Codex usage lifting, time-range turn attachment across several turns, cross-service parents, both parent fields, Claude cache-inclusive and unknown usage, the mapping note, and recorded fields staying unchanged.agents.spec.ts+agent-setup.spec.tson a Blacksmith testbox: 26 passed. New specs: native Codex traces → discovered dataset, run row, tree, mapping note and recorded parent ID; record-limit banner when Codex mapping drops queried rows. The other Agents specs (agents-navigation,agents-review-*) also passed; one test failed once on cold-start Vite re-optimization and passed on rerun.Limitations
claude_code.llm_request.execcall inside the turn represents them.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.