feat: Structured Output via `output` Tool Call

[Matheus-Garbelini]

Overview

Currently, when structured output is on, the agent cannot call other tools. While this is understandable due to how endpoints implement structured outputs, it limits and add costs to agentic workflows which the agent can both do work and respond in the correct format. Therefore, an alternative to solve this problem is to enforce structured output via standard tool calling which already offer the output contracts.

Motivation

Agents are becoming extremely optimal when working via iterative tool calling feedback (due to their reinforcement learning favouring such) as opposed to just doing text transformations. I believe it makes sense to allow tool calls in the same turn of the structured output. And it looks like this is being cached on by other frameworks.

Enforcing structured output directly in the API (e.g., OpenAI’s response_format: { type: "json_schema" } or constrained decoding) strictly limits the model's token generation to match a specific shape and degrade performance. Enforcing it via an agent tool (Function Calling) treats the schema as an action the model decides to take.

Use cases

• Explore codebase and save results in the same turn: faster and less cost.
• Iterate on the answer in the same turn: the agent can do work, call the tool, then verify and call the tool again with any corrections. The user will receive the structured output which matches his workflow without needing additional agents only for transformations.
• Handle multiple instances of the response. Defining one massive, deeply nested schema for an API response_format can confuse the model and dilute output quality. During codebase exploration or long-horizon tasks, the model is free to output multiple times as the turn continues. This gives the user a lot more simplicity in the design: No need to create multiple structured output agents for the same long horizon tasks.
• Processing messy, unstructured data. If the agent can call the output tool multiple times, you get an aggregation of the structured data already.

Proposed solution

When investigating GitHub Copilot SDK, they enable structured output by letting the agent simply call an output tool: The argument of the tool becomes the response, and the tool argument schema automatically validates the response and makes the model retry if it does not comply. This is something that docker-agent already allows, but it's not internally used as the means for structured output.

The implementation of this is quite direct:

• If the model does not call the output tool, a prompt nudging the agent asks it to use the output tool. Fall back and retry works the same, no additional logic needed here other than the nudge.
• If the model does call the tool correctly, the arguments of the tool call are saved internally as structured output. The turn eventually ends by the agent.
• If the agent decides to call the structured output multiple times, do not constrain it, the call is appended in the list of responses such that Output = [Output1, Output2, Output3, ....]

Reference implementation by copilot: roccoren/ghcp-pool-go#2
Codex already does that internally also.

Alternatives

The current workaround for this is to:

• Create an additional agent that does the transformation: this is more costly since the turn will start thinking again.
• The user creates the tool manually, which is less convenient than the above.
• Multiple instances of the same structured output needs multiple turn calls.

Related issues

No response

Additional context

No response