Repository navigation
Join Codex and Claude Code logs into Agents run conversations - #34
Merged
Merged
Conversation
Both harnesses export prompts, responses and tool results as log records stamped with the trace and span they happened in. For a harness traces dataset, Agents now finds the matching logs dataset (preferring the one named like the traces dataset), queries only conversation events for the loaded runs' trace IDs, and attaches them to their spans in order: - Codex: the user prompt on its turn; tool arguments and output on the tool span, matched by call ID. - Claude Code: the user prompt, assistant responses on their model call (by request ID), tool input and approval decisions on the tool span (by tool use ID), and tool output from the tool span's trace event. Redacted content is labelled rather than hidden, and records for runs that are not loaded are ignored. A "Conversation logs" selector shows the paired dataset and can turn the join off; search also matches conversation text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Reviewed 64925c231ffdc0df6334cf653b57a960a11cb027 (the diff against #33's 0febb045) with two GPT-6.1-Sol and two Claude Opus 5.5 reviewers, each covering a different area. Actionable findings are in the threads below.
What holds up: generic GenAI datasets and demo mode send no extra requests, the SQL quoting is sound, stale responses can't overwrite newer ones, and a failed logs query leaves runs rendering. agentHarnessLogs.test.ts passes (8/8). The edge cases were reproduced with scratch scripts against the PR's functions; Playwright and live ingestion were not rerun.
…ness-conversation-logs Bring in the review fixes from #33. Harness detection now returns a list; conversation logs stay joined for a single detected harness until the page handles mixed datasets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address review findings on joining harness logs into run conversations: - Query logs newest first so the most recent runs keep their events when the record limit is reached, and say when the limit was reached. - Discover logs datasets once for all harnesses, keyed on a stable "needed" flag rather than the reloading run data, and reload it from Rescan datasets. Skip 403/404 schemas, report other failures with Retry, and only say no logs dataset exists when the scan succeeded. - Drop the logs choice when switching traces datasets, and ignore a logs param that is not a candidate for the current dataset's harnesses. Mixed traces datasets offer logs of every detected harness and join the selected one with its own harness. - Attach prompts to the turn whose time range contains them, scope tool call IDs to their trace and prefer the record's own tool span, and keep repeated log events while still collapsing flattened span events. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Match a log event to the harness's turn spans, so an enclosing agent span of another service no longer makes every prompt ambiguous. An event inside exactly one turn goes to that turn; a Codex prompt otherwise goes to the turn starting within 100 ms after it. An event after the only loaded turn is left unattached, since it may belong to a turn not exported yet, and untimed events fall back to the only turn rather than the trace root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex logs a prompt on the turn's input span, which the turn records as its parent. Match prompts through that parent first, unless several turns share it, and keep the timing rule only as a fallback. Only the harness's own turns are candidates, and turns that start together leave a prompt unattached instead of picking one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Other events logged on a span a turn records as its parent are placed by their time, as before, instead of being taken to belong to that turn. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VibeSWE
changed the base branch from
feat/agents-native-harness-fields
to
main
October 13, 2026 05:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #33. Review that first; this PR's diff is against its branch.
Summary
#33 builds Codex and Claude Code runs from their traces. The prompts, responses and tool results are exported separately, as log records, so the Conversation tab stayed empty. Both harnesses stamp each log record with the trace and span it happened in, so the join can be automatic.
What changes
Logs dataset pairing (
lib/agentHarnessLogs.ts): when the selected traces dataset holds Codex or Claude Code traces, Agents scans the workspace once for logs datasets of either harness (Rescan datasets repeats the scan; Refresh doesn't). Schemas that return 403/404 are skipped; other failures are shown with Retry instead of reporting that no logs dataset exists. It recognizes Codex logs byconversation.id+originator, and Claude Code logs byprompt.id+event.sequence, each withtrace_idandevent.name. It picks the one named like the traces dataset (codex-traces→codex-logs). A Conversation logs selector shows the choice and can turn the join off (logs=nonein the URL). The choice is cleared when switching traces datasets, and alogs=value that isn't a candidate for the current dataset is ignored. A traces dataset with both harnesses offers logs of either and joins the selected one with its own harness.Query: one extra query fetches only conversation events (
event.name IN …) for the loaded runs' trace IDs, within the same time bounds and record limit, newest first, so the most recent runs keep their events. A note says when the limit was reached.Attachment, in the order things happened:
codex.turn_inputspan the prompt is logged on, else the turn containing it in time; ambiguous prompts stay unattachedrequest_idargumentson the tool span, bycall_idwithin the tracetool_inputon the tool span, bytool_use_idwithin the traceoutputon the tool spantool.outputtrace event (shown even without a logs dataset)call_idtool_use_idRedacted content keeps its recorded value and gets a "(redacted)" label, so it's clear the exporter withheld it rather than the dashboard.
Unloaded runs: records for runs outside the current view are ignored, never guessed. Codex's code-mode child tool calls run in a separate trace with no turn, so they're skipped; the outer
execresult already includes their output.Conversation tab and search: the tab lists spans in time order, and run search now matches conversation text.
Setup guide: says to export logs to their own dataset as well.
Evidence
Captured with content flags on for a probe run:
codex exec -c otel.log_user_prompt=true, and Claude Code withOTEL_LOG_USER_PROMPTS,OTEL_LOG_ASSISTANT_RESPONSES,OTEL_LOG_TOOL_DETAILSandOTEL_LOG_TOOL_CONTENT. Running this branch's discovery, queries and attachment against the localcodex-traces/codex-logsandclaude-code-traces/claude-code-logsproduced:Testing
npm run format:check,tsc --noEmit,vite buildnpm test: 2,389 passed, 14 skipped. NewagentHarnessLogs.test.ts, with fixtures trimmed from the real logs, covers detection, discovery (skip, unchecked and rethrow paths), pairing, quoted newest-first SQL, Codex/Claude attachment and ordering, per-turn prompt attribution, trace-scoped tool calls, repeated events, redaction labels, trace-only Claude tool output, and ignoring unloaded runs.agents.spec.ts+agent-setup.spec.ts: 32 passed. New specs: the paired logs dataset (selector default, Conversation tab, log query, search, turning logs off), switching traces datasets withlogsset, Refresh without rediscovery and Rescan picking up a new logs dataset, the logs record-limit note, a failed logs scan with Retry, and a mixed traces dataset.LogsPagedynamic import failing while Vite re-optimizes dependencies.origin/mainfails the same way on this machine (2 failed / 42 passed, including a "Target crashed"), and the failing tests pass on rerun. I've treated this as local dev-server instability, not a regression; CI should confirm.Limitations
log_user_prompt = false.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.