[SDKv2] Various fixes and cleanups from converting the samples to use the V2 API#754
[SDKv2] Various fixes and cleanups from converting the samples to use the V2 API#754skottmckay wants to merge 1 commit into
Conversation
- Fix handling of tool definitions in Responses API when a cached session is found - Fix delete response to evict session from cache - Make support for `file://` more consistent for image and audio input - Fix C# setup for ItemQueue - it should inherit from Item for consistency - Fix C# setup for adding Item to Request - make ownership transfer optional for consistency - Fix SDKs to enable getting the overall Response when streaming - Make getting data from a TensorItem more user friendly
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR is a broad set of fixes and cleanups driven by converting the V2 API samples. It reshapes the streaming surface across C#/Python so callers can retrieve the terminal Response (with finish_reason/usage/aggregated items) after iteration completes, makes ItemQueue an Item for C# consistency, makes ownership transfer in Request.AddItem optional, fixes Responses-API tool definitions on cached sessions, evicts cached sessions on DELETE /v1/responses/{id}, normalizes file:// URI handling for image/audio inputs, and adds typed span/array accessors on TensorItem. Several C++ logging tweaks, a field-audit static_assert, and a Smoke.NetFx project removal round it out.
Changes:
- New
StreamingResponsewrapper (C#/Python) decouples per-item iteration from terminal Response retrieval, with explicit cancel/dispose/once-only enumeration semantics. - Responses handler extracts and reapplies tool definitions per request (mirroring chat-completions), and
DeleteResponseHandlerevicts cached sessions via the newSessionManager::EvictCached. file://parsing centralized inutil/file_uri.h(with percent-decoding) and applied to both image and audio inputs;ItemQueuenow inherits fromItem;TensorItemexposesAsSpan<T>()/ToArray<T>()/ElementCount;Request.AddItemgains atakeOwnershipflag.
Reviewed changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| sdk_v2/cpp/src/inferencing/generative/openresponses/response_converter.{h,cc} | Adds ExtractResponsesToolDefinitions (with tool_choice / allowed_tools handling) and field-audit static_asserts. |
| sdk_v2/cpp/src/service/responses_handler.cc | Applies extracted tools per request and evicts cached session on DELETE. |
| sdk_v2/cpp/src/inferencing/session/session.h | Adds ClearToolDefinitions(). |
| sdk_v2/cpp/src/inferencing/session/session_manager.{h,cc} | Adds EvictCached(key). |
| sdk_v2/cpp/src/util/file_uri.h | New centralized file:// parser with percent-decode. |
| sdk_v2/cpp/src/items/image_item.cc | Uses PathFromFileUri. |
| sdk_v2/cpp/src/inferencing/generative/audio/audio_session.cc | Uses PathFromFileUri for AudioItem.uri. |
| sdk_v2/cpp/src/ep_detection/ep_detector.cc | Adds debug logging of detected EP devices. |
| sdk_v2/cpp/test/* (session_manager_test, response_converter_test, file_uri_test, CMakeLists.txt) | Unit-test coverage for new behaviors. |
| sdk_v2/cs/src/StreamingResponse.cs | New IAsyncEnumerable+IAsyncDisposable wrapper with FinalResponse task. |
| sdk_v2/cs/src/Session.cs | ProcessStreamingRequestAsync returns StreamingResponse; producer snapshots cancellation before completing channel. |
| sdk_v2/cs/src/Request.cs | Adds takeOwnership parameter and null guard. |
| sdk_v2/cs/src/Items/ItemQueue.cs | Refactored to inherit from Item; resolves inner flItemQueue* once. |
| sdk_v2/cs/src/Items/Item.cs | FromNative dispatches ItemType.Queue to ItemQueue. |
| sdk_v2/cs/src/Items/TensorItem.cs | Adds AsSpan<T>, ToArray<T>, ElementCount with type-mapping. |
| sdk_v2/cs/test/FoundryLocal.Tests/{ChatSessionTests,AudioSessionTests}.cs | Streaming + final-response test coverage. |
| sdk_v2/cs/test/Smoke.NetFx/* | Project removed. |
| sdk_v2/python/src/foundry_local_sdk/session.py | New StreamingResponse class and revised process_streaming_request. |
| sdk_v2/python/src/foundry_local_sdk/openai/live_audio_session.py | Consumes new streaming shape; emits aggregated final transcript. |
| sdk_v2/python/src/foundry_local_sdk/items.py | MessageItem.is_simple_text()/get_simple_text(). |
| sdk_v2/python/src/foundry_local_sdk/{imodel,foundry_local_manager}.py | Switch ffi.callback signatures to typedef'd names. |
| sdk_v2/python/src/foundry_local_sdk/init.py | Exports ItemQueue, StreamingResponse. |
| sdk_v2/python/test/* | Unit/integration test coverage for new APIs. |
| .github/instructions/cpp-build.instructions.md, memories/repo/*.md | Documentation updates. |
file://more consistent for image and audio input