feat: Add typed structured outputs for all six SDKs

.github/workflows/dotnet-sdk-tests.yml

@@ -42,6 +42,14 @@ jobs:
       - name: Build SDK
         run: dotnet build --no-restore
 
+      - name: Test structured output with reflection enabled
+        env:
+          DOTNET_ROLL_FORWARD: Major
+        run: >-
+          dotnet test test/GitHub.Copilot.SDK.Test.csproj --no-restore --framework net8.0
+          -p:JsonSerializerIsReflectionEnabledByDefault=true
+          --filter "FullyQualifiedName~GitHub.Copilot.Test.Unit.ClientSessionLifetimeTests.StructuredOutput"
+
   test:
     name: ".NET SDK Tests (${{ matrix.os }}, ${{ matrix.transport }}, ${{ matrix.backend }}, ${{ matrix.shard }})"
     if: github.event.repository.fork == false

.github/workflows/rust-sdk-tests.yml

@@ -80,7 +80,7 @@ jobs:
         # embed it. Tests exec against the setup-copilot CLI via
         # COPILOT_CLI_PATH (the env override wins over the dev cache).
         # The dedicated `bundle` job below exercises the embed pipeline.
-        run: cargo test --no-default-features --features test-support -- --test-threads=4 --nocapture
+        run: cargo test --no-default-features --features test-support,derive -- --test-threads=4 --nocapture
 
   clippy:
     name: "Rust SDK Format and Clippy"
@@ -139,7 +139,7 @@ jobs:
       - name: cargo clippy
         env:
           BUNDLED_CLI_CACHE_DIR: ${{ github.workspace }}/rust/.bundled-cli-cache
-        run: cargo clippy --all-targets --features test-support,bundled-in-process -- --no-deps -D warnings -D clippy::unwrap_used -D clippy::disallowed_macros -D clippy::await_holding_invalid_type
+        run: cargo clippy --all-targets --features test-support,bundled-in-process,derive -- --no-deps -D warnings -D clippy::unwrap_used -D clippy::disallowed_macros -D clippy::await_holding_invalid_type
 
   doc:
     name: "Rust SDK Docs"
@@ -265,7 +265,7 @@ jobs:
         # The harness forces serial execution in-process (both the async semaphore and
         # libtest via --test-threads=1) because it mirrors each test's environment onto
         # the shared process environment, so RUST_E2E_CONCURRENCY is not set here.
-        run: cargo test --no-default-features --features test-support,bundled-in-process --test e2e -- --test-threads=1 --nocapture
+        run: cargo test --no-default-features --features test-support,bundled-in-process,derive --test e2e -- --test-threads=1 --nocapture
 
   # Validates the bundled-CLI build path on all three supported
   # platforms. While the regular `cargo test` job above also exercises
@@ -376,8 +376,8 @@ jobs:
             export CARGO_TARGET_DIR=/tmp/copilot-sdk-rust-target
             if [ "$COPILOT_SDK_TEST_TRANSPORT" = "inprocess" ]; then
               unset RUST_E2E_CONCURRENCY
-              cargo test --no-default-features --features test-support,bundled-in-process --test e2e -- --test-threads=1 --nocapture
+              cargo test --no-default-features --features test-support,bundled-in-process,derive --test e2e -- --test-threads=1 --nocapture
             else
               export RUST_E2E_CONCURRENCY=4
-              cargo test --no-default-features --features test-support -- --test-threads=4 --nocapture
+              cargo test --no-default-features --features test-support,derive -- --test-threads=4 --nocapture
             fi

CONTRIBUTING.md

@@ -58,6 +58,98 @@ Setup, build, and test instructions are maintained with each SDK:
 - [Rust](rust/README.md#development)
 - [Java](java/README.md#development-setup)
 
+### Testing an unreleased runtime API
+
+The runtime's Rust contracts under `src/native/sdk-contract` produce both
+`generated/api.schema.json` (RPC methods) and
+`generated/session-events.schema.json` (event payloads). In a local checkout of
+`github/copilot-agent-runtime`, build the runtime and emit these schemas:
+
+```bash
+pnpm run build
+pnpm bazel build //src/native/schema-codegen:schema-codegen
+bazel-bin/src/native/schema-codegen/schema-codegen emit \
+  --api "$PWD/generated/api.schema.json" \
+  --session-events "$PWD/generated/session-events.schema.json"
+```
+
+The SDK generators normally download schemas from the pinned CLI release. To
+use the local schemas instead, pass the event-schema path followed by the
+RPC-schema path. From this repository's `scripts/codegen` directory:
+
+```bash
+npm ci
+for language in typescript csharp python go rust; do
+  node --import tsx "$language.ts" \
+    "$RUNTIME_ROOT/generated/session-events.schema.json" \
+    "$RUNTIME_ROOT/generated/api.schema.json"
+done
+```
+
+Set `RUNTIME_ROOT` to the absolute path of the runtime checkout. Java's generator
+at `java/scripts/codegen/java.ts` reads these files from
+`java/scripts/codegen/target/schemas` instead of accepting positional arguments;
+stage the local schemas there before running it. Do not hand-edit generated
+wrappers. Regenerating against a newer runtime
+also includes any other contract changes since the SDK's pinned release.
+
+Set `COPILOT_CLI_PATH` to the built runtime's `dist-cli/index.js` to run SDK E2Es
+against that checkout rather than the packaged runtime. For example:
+
+```bash
+export COPILOT_CLI_PATH="$RUNTIME_ROOT/dist-cli/index.js"
+# Supply GITHUB_TOKEN with Copilot access when recording new provider responses.
+cd nodejs
+npm test -- test/e2e/structured_output.e2e.test.ts
+cd ../dotnet
+dotnet test test/GitHub.Copilot.SDK.Test.csproj \
+  --filter FullyQualifiedName~StructuredOutputE2ETests
+```
+
+The shared harness records real inference responses under `test/snapshots`.
+Record new captures with `GITHUB_TOKEN` set and `GITHUB_ACTIONS` unset;
+never author model responses by hand. Rerun with `GITHUB_ACTIONS=true` and real
+provider credentials removed to require replay instead of forwarding cache
+misses upstream. A draft targeting an unreleased runtime should document the
+required runtime revision; update the pinned release only after it ships.
+Pinned-schema CI can report drift in such a draft, and Java codegen may
+automatically update generated files to match the pinned release.
+
+For recording behind `HTTPS_PROXY`, Node versions that support environment
+proxies (including Node 24.20) need `NODE_USE_ENV_PROXY=1` in the test runner's
+environment. If the host proxy substitutes a protected credential, set
+`GITHUB_TOKEN="$GH_TOKEN"` using its issued placeholder; do not print or persist
+the credential. Keep localhost and loopback in `NO_PROXY`.
+
+Equivalent cross-language E2Es must share snapshot names and prompts, not
+language-specific copies. The structured-output suite in **all six SDKs** reuses
+the following captures in `test/snapshots/structured_output/`, recorded using
+real CAPI `gpt-4.1` calls through the shared harness:
+
+| Shared capture (without `.yaml`) | Flow |
+| --- | --- |
+| `infers_typed_result_after_custom_tool` | Inferred typed result after a tool call, streamed text, then an unformatted follow-up |
+| `sends_explicit_schema_for_message_and_batch` | Explicit-schema batch RPC followed by a schema-bearing single send |
+| `send_selects_correlated_response_after_idle` | Event-driven send, tool commentary, originating-message correlation, and an idle boundary held by a stop hook |
+| `typed_wait_returns_stop_hook_correction` | Typed wait returns the corrected answer, not the first assistant message |
+| `typed_wait_returns_stop_hook_correction_after_terminal_tool` | Output-only finalization after a terminal tool, followed by a stop-hook correction |
+| `typed_result_after_terminal_tool_and_steering` | Immediate steering during a terminal tool preserves the active schema |
+| `typed_wait_returns_late_steering_response` | Steering after the first final answer remains part of the original run |
+| `concurrent_typed_sends_return_their_own_results` | Concurrent queued runs use different inferred types and return their own results |
+
+Typed cases call the public idiomatic APIs: Node/Zod, C# generics, Python/Pydantic,
+Go generics, Java annotated records using the existing tool schema generator,
+and Rust generics with `derive`/schemars. The tool/follow-up case also checks the
+actual provider request's inferred schema, so a recorded JSON response alone
+cannot mask missing schema forwarding. Explicit-schema and event-stream cases
+exercise the corresponding raw public APIs instead.
+
+Every language additionally checks rejection before admission and zero provider
+calls for oversized schemas and typed immediate steering. These cases have no
+model responses and therefore need **no snapshot**. Do not create canned responses
+or empty model captures for them. Unit tests supplement, rather than replace,
+the shared runtime E2Es.
+
 ## Submitting a Pull Request
 
 1. Fork and clone the repository

dotnet/README.md

@@ -251,6 +251,7 @@ Send a message to the session.
 - `Attachments` - File attachments
 - `Mode` - Delivery mode ("enqueue" or "immediate")
 - `Source` - Optional message origin: `MessageSource.User`, `MessageSource.System`, or `MessageSource.Agent(id)`. Omitted by default, preserving the runtime's default user behavior.
+- `ResponseSchema` - Experimental provider-native JSON Schema (`JsonElement`) for this turn.
 
 Returns the message ID.
 
@@ -277,6 +278,132 @@ await session.SendAndWaitAsync(new MessageOptions
 Agent sources serialize as `agent-<id>`. Pass the agent ID without adding a
 prefix. The SDK preserves its case and whitespace and rejects null IDs.
 
+##### Structured outputs (experimental)
+
+Use `SendAndWaitAsync<TResult>` to infer a JSON Schema from a .NET type and
+deserialize the final response. Schema inference uses
+`Microsoft.Extensions.AI.AIJsonUtilities`, the same technology as custom tools.
+In a reflection-enabled application, `await session.SendAndWaitAsync<Inventory>(prompt)`
+needs no serialization configuration. The example below supplies source-generated
+metadata so it also works when reflection serialization is disabled.
+
+```csharp
+var result = await session.SendAndWaitAsync<Inventory>(
+    "How many red widgets are in stock?",
+    serializerOptions: InventoryJsonContext.Default.Options);
+Console.WriteLine($"{result.Count} {result.Color} widgets");
+
+public sealed class Inventory
+{
+    public required int Count { get; set; }
+    public required string Color { get; set; }
+}
+
+[System.Text.Json.Serialization.JsonSourceGenerationOptions(
+    PropertyNamingPolicy = System.Text.Json.Serialization.JsonKnownNamingPolicy.CamelCase)]
+[System.Text.Json.Serialization.JsonSerializable(typeof(Inventory))]
+internal partial class InventoryJsonContext : System.Text.Json.Serialization.JsonSerializerContext;
+```
+
+The same serialization options govern schema inference and deserialization,
+including naming policies, `[JsonPropertyName]`, converters, required members,
+and nullable annotations. Options default to `AIJsonUtilities.DefaultOptions`,
+as for custom tools. Supply a source-generated resolver (as above) for Native
+AOT or when reflection serialization is disabled. The typed helper requests
+strict output, marks all schema properties required, and disallows additional
+properties; nullable properties can still contain JSON null.
+
+The helper waits for non-autopilot session idle after the requested user message
+is consumed, selecting only root assistant messages with that originating message
+ID. This can wait for other queued work to drain, but other messages and subagent
+responses cannot replace the result. Session errors or an aborted idle after the
+requested run starts conservatively fail the wait, even if later queued work
+caused them. It throws `InvalidOperationException` when there is no final response,
+and `JsonException` for invalid JSON, an incompatible
+value, or a null result. Deserialization is not full JSON Schema validation:
+validate application-specific constraints yourself. Timeout defaults to 60
+seconds; timeout and cancellation stop waiting without aborting runtime work.
+The original `MessageOptions` is not modified, and an explicit `ResponseSchema`
+cannot be combined with this typed overload.
+
+For an explicit schema, set `MessageOptions.ResponseSchema`. Schemas are opaque
+`JsonElement` values, just like custom-tool schemas. The SDK forwards this schema
+unchanged with the name `response` and `strict: true`. The untyped
+`SendAndWaitAsync` still returns an assistant message event; it does not validate
+or deserialize the response. Schema-bearing waits use the same message
+correlation as typed waits; unformatted waits retain their existing behavior.
+
+With `SendAsync`, collect root `AssistantMessageEvent` events whose
+`Data.OriginatingMessageId` matches the returned message ID, then select the last
+one without tool requests when the session becomes idle. Subscribe before sending
+because events can precede the send acknowledgement, and handle `SessionErrorEvent` normally.
+There is no final-message flag: stop hooks can reject an initial answer and
+request a correction. Those corrections retain the original schema and
+originating message ID, so `SendAndWaitAsync` selects the corrected response at
+idle. Independent queued sends retain their own schemas and IDs.
+
+```csharp
+using var schema = System.Text.Json.JsonDocument.Parse("""
+    {"type":"object","properties":{"count":{"type":"integer"}},"required":["count"],"additionalProperties":false}
+    """);
+var message = await session.SendAndWaitAsync(new MessageOptions
+{
+    Prompt = "Count the widgets.",
+    ResponseSchema = schema.RootElement.Clone(),
+});
+```
+
+Use the generated `session.Rpc` APIs for advanced response-format options:
+
+```csharp
+using GitHub.Copilot.Rpc;
+using System.Text.Json;
+
+using var schema = JsonDocument.Parse("""
+    {"type":"object","properties":{"count":{"type":"integer"}},"required":["count"],"additionalProperties":false}
+    """);
+var format = new ResponseFormatJsonSchema
+{
+    JsonSchema = new JsonSchemaResponseFormat
+    {
+        Name = "inventory",
+        Schema = schema.RootElement.Clone(),
+        Strict = true,
+        Description = "The inventory count",
+    },
+};
+await session.Rpc.SendAsync("Count the widgets.", responseFormat: format);
+// A batch shares one output contract:
+await session.Rpc.SendMessagesAsync(
+    [new() { Prompt = "There are 42 widgets." }, new() { Prompt = "Report the count." }],
+    responseFormat: format);
+```
+
+Raw schemas and outputs are passed through without validation or rewriting.
+Provider support and schema restrictions apply. The format persists through
+tool continuations in that run, not independent subsequent runs. An ordinary
+`Mode = "immediate"` steering message inherits the active format and originating
+message ID, even if it arrives after the final model request and is promoted
+into a follow-up run. Specifying a new format on an immediate message is rejected,
+even while idle.
+Each batch starts one run: the final returned message ID is its origin, preceding
+messages are context, and an empty batch has no origin. An immediate batch
+steers the active run instead and retains its origin.
+The schema is not a persisted session default: autonomous resume-pending work
+after a restart does not restore it. A terminal tool that clears context ends
+the old run; its fresh seed does not inherit the schema or origin. Such a run
+can finish without a structured result, in which case the typed wait throws.
+After a successful terminal tool, the runtime disables tools while the model
+produces the structured result. Stop-hook corrections remain supported.
+Remote sessions and known HydraFusion routes reject response formats before
+admission. Schemas larger than 32 MiB when JSON-encoded are also rejected before
+admission, using the runtime's existing request-size ceiling. This does not
+guarantee the schema plus conversation and tools fits the provider's budget.
+Use a provider route that enforces JSON Schema: an API-compatible gateway can
+ignore unsupported format fields, and the Claude Chat-completions compatibility
+route is not equivalent to Anthropic's native Messages endpoint. The SDK's
+pinned CLI release includes the required runtime support.
+
 ##### `On(Action<SessionEvent> handler): IDisposable`
 
 Subscribe to session events. Returns a disposable to unsubscribe.