feat(openai-agents)!: emit invoke_agent, chat and execute_tool spans - #33
Draft
apucacao wants to merge 3 commits into
Draft
feat(openai-agents)!: emit invoke_agent, chat and execute_tool spans#33apucacao wants to merge 3 commits into
apucacao wants to merge 3 commits into
Conversation
Author
|
bugbot run |
apucacao
force-pushed
the
ag/py-telemetry-openai-agents
branch
from
August 11, 2026 20:43
efe4f71 to
aec3684
Compare
Author
|
bugbot run |
apucacao
force-pushed
the
ag/py-telemetry-openai-agents
branch
from
August 11, 2026 21:01
aec3684 to
60c2410
Compare
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 60c2410. Configure here.
One flat span named openai.agent.run becomes the tree the TypeScript SDK emits:
an invoke_agent root, one `chat {model}` child per model turn, one
`execute_tool {name}` child per tool call. The per-turn data was already in
hand: this handler walked the Runner's raw responses to sum usage, and simply
never opened a span per turn.
BREAKING CHANGE: the span this handler emits is renamed from `openai.agent.run`
and `openai.agent.run.stream` to `invoke_agent`. Queries selecting on the old
names will not match. Prompt and completion content is no longer on spans
unless the caller passes capture_content=True.
Cached tokens are now reported, read from the cached-tokens detail and left out
of the input total, because OpenAI already counts them inside it. Cache creation
is always zero.
gen_ai.response.model stays the requested name here, on both the root and the
chat spans, which is deliberately different from openai-messages. The
TypeScript twin has never resolved the answering model in this handler and no
test pins it, so reporting one would invent behaviour rather than match it.
Finish reasons are derived from the Responses API's status rather than mapped
through the shared table, which does not apply: there is no finish_reason field
to map. A function call in the output takes precedence over status, because a
live capture put `completed` on every turn including the six that stopped to
call a tool.
Abandoning the stream needed more than ending our spans. The old path iterated
the Runner's event stream with no cleanup at all, and breaking out of that loop
only stops us reading: the Runner's own background task keeps calling the model
and spending tokens until told to stop. Teardown now cancels the streamed run
as well as closing the span tree, so an abandoned stream stops costing money.
Tests: 53 to 71.
Two sources describe the same spend and they overlap. The run hooks add each turn as it finishes, so by the time the run raises they already hold every completed turn, and the exception carries the SDK's own aggregate over those same turns. The error path added the aggregate to the accumulator, so any run that failed after paid turns reported roughly twice what it cost. MaxTurnsExceeded does that by definition, which makes this the common case rather than an edge one. A three-turn run reporting 70 input tokens reported 140. The aggregate is the authoritative figure, so it now replaces the accumulator rather than adding to it, matching what the TypeScript handler does. When the error carries no aggregate, which is what a tool handler's own error looks like, the accumulator is all there is and is used instead. Neither having anything still writes nothing, because all-zero attributes would assert the run cost nothing. Three tests, one per branch. The double-count one fails with 140 against 70 when the fix is reverted, which is how I checked it pins the bug rather than the behaviour. Found by Bugbot on #33, severity High.
The wrapper never passed capture_content to the factory, so it stayed in kwargs and reached config(), which takes no such argument. A caller asking for content on spans got a TypeError rather than content. Lifted out alongside variables, which was already handled the same way and for the same reason: one configures the handler, the other belongs to the invocation, and config() accepts neither. Two tests, one per branch, asserting the flag reaches the factory and does not reach config(). Found by Bugbot on #33. It flagged this handler; five of the six wrappers have it, and the other four are fixed in their own layers.
apucacao
force-pushed
the
ag/py-telemetry-openai-agents
branch
from
August 11, 2026 21:18
60c2410 to
169ab0c
Compare
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 169ab0c. Configure here.
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 169ab0c. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces one flat span per call with the tree the TypeScript SDK emits, for
openai-agents.The per-turn data was already in hand: this handler walked the Runner's raw responses to sum usage, and simply never opened a span per turn.
Abandoning the stream was costing money
The old path iterated the Runner's event stream with no cleanup at all. Breaking out of that loop only stops us reading: the Runner's own background task keeps calling the model and spending tokens until told to stop.
Teardown now cancels the streamed run as well as closing the span tree.
Two things specific to this handler
gen_ai.response.modelstays the requested name, on both the root and the chat spans, which is deliberately different fromopenai-messages(feat(openai-messages)!: emit invoke_agent, chat and execute_tool spans #32). The TypeScript twin has never resolved the answering model here and no test pins it, so reporting one would invent behaviour rather than match it.Finish reasons are derived from the Responses API's status rather than mapped through the shared table, which does not apply: there is no
finish_reasonfield to map. A function call in the output takes precedence over status, because a live capture putcompletedon every turn including the six that stopped to call a tool.Other changes
Cached tokens are now reported, read from the cached-tokens detail and left out of the input total, because OpenAI already counts them inside it. Cache creation is always zero.
Breaking change
The span is renamed from
openai.agent.runtoinvoke_agent. Queries selecting on the old name will not match. Prompt and completion content is no longer on spans unless the caller passescapture_content=True.Where this sits
Needs the usage layer (#28) and the content layer (#29). Independent of the other five handler PRs; the stack orders them only because
gh stackis linear.Tests: 781 to 799.
Note
Overview
Replaces the flat
openai.agent.runspan with the TypeScript-aligned tree: aninvoke_agentroot, onechat {model}child per model turn, and siblingexecute_tool {name}spans per tool call—driven viaagents.RunHooksrather than a Model wrapper.Breaking: span rename (
openai.agent.run→invoke_agent), and prompt/completion/tool content is off by default unlesscapture_content=True.Also cancels abandoned streamed runs (previously the Runner kept spending tokens after the consumer stopped), reports cached-token usage, and writes failed-run spend onto the root without double-counting the SDK aggregate.
Reviewed by Cursor Bugbot for commit 169ab0c. Bugbot is set up for automated code reviews on this repo. Configure here.