Skip to content

[SDKv2] Various fixes and cleanups from converting the samples to use the V2 API#754

Open
skottmckay wants to merge 1 commit into
mainfrom
skottmckay/SDKv2_MiscFixesAndImprovements
Open

[SDKv2] Various fixes and cleanups from converting the samples to use the V2 API#754
skottmckay wants to merge 1 commit into
mainfrom
skottmckay/SDKv2_MiscFixesAndImprovements

Conversation

@skottmckay
Copy link
Copy Markdown
Collaborator

  • Fix handling of tool definitions in Responses API when a cached session is found
  • Fix delete response to evict session from cache
  • Make support for file:// more consistent for image and audio input
  • Fix C# setup for ItemQueue
    • it should inherit from Item for consistency
  • Fix C# setup for adding Item to Request
    • make ownership transfer optional for consistency
  • Fix SDKs to enable getting the overall Response when streaming
  • Make getting data from a TensorItem more user friendly

- Fix handling of tool definitions in Responses API when a cached session is found
- Fix delete response to evict session from cache
- Make support for `file://` more consistent for image and audio input
- Fix C# setup for ItemQueue
  - it should inherit from Item for consistency
- Fix C# setup for adding Item to Request
  - make ownership transfer optional for consistency
- Fix SDKs to enable getting the overall Response when streaming
- Make getting data from a TensorItem more user friendly
Copilot AI review requested due to automatic review settings June 2, 2026 04:51
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 2, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
foundry-local Ready Ready Preview, Comment Jun 2, 2026 4:51am

Request Review

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR is a broad set of fixes and cleanups driven by converting the V2 API samples. It reshapes the streaming surface across C#/Python so callers can retrieve the terminal Response (with finish_reason/usage/aggregated items) after iteration completes, makes ItemQueue an Item for C# consistency, makes ownership transfer in Request.AddItem optional, fixes Responses-API tool definitions on cached sessions, evicts cached sessions on DELETE /v1/responses/{id}, normalizes file:// URI handling for image/audio inputs, and adds typed span/array accessors on TensorItem. Several C++ logging tweaks, a field-audit static_assert, and a Smoke.NetFx project removal round it out.

Changes:

  • New StreamingResponse wrapper (C#/Python) decouples per-item iteration from terminal Response retrieval, with explicit cancel/dispose/once-only enumeration semantics.
  • Responses handler extracts and reapplies tool definitions per request (mirroring chat-completions), and DeleteResponseHandler evicts cached sessions via the new SessionManager::EvictCached.
  • file:// parsing centralized in util/file_uri.h (with percent-decoding) and applied to both image and audio inputs; ItemQueue now inherits from Item; TensorItem exposes AsSpan<T>()/ToArray<T>()/ElementCount; Request.AddItem gains a takeOwnership flag.

Reviewed changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated no comments.

Show a summary per file
File Description
sdk_v2/cpp/src/inferencing/generative/openresponses/response_converter.{h,cc} Adds ExtractResponsesToolDefinitions (with tool_choice / allowed_tools handling) and field-audit static_asserts.
sdk_v2/cpp/src/service/responses_handler.cc Applies extracted tools per request and evicts cached session on DELETE.
sdk_v2/cpp/src/inferencing/session/session.h Adds ClearToolDefinitions().
sdk_v2/cpp/src/inferencing/session/session_manager.{h,cc} Adds EvictCached(key).
sdk_v2/cpp/src/util/file_uri.h New centralized file:// parser with percent-decode.
sdk_v2/cpp/src/items/image_item.cc Uses PathFromFileUri.
sdk_v2/cpp/src/inferencing/generative/audio/audio_session.cc Uses PathFromFileUri for AudioItem.uri.
sdk_v2/cpp/src/ep_detection/ep_detector.cc Adds debug logging of detected EP devices.
sdk_v2/cpp/test/* (session_manager_test, response_converter_test, file_uri_test, CMakeLists.txt) Unit-test coverage for new behaviors.
sdk_v2/cs/src/StreamingResponse.cs New IAsyncEnumerable+IAsyncDisposable wrapper with FinalResponse task.
sdk_v2/cs/src/Session.cs ProcessStreamingRequestAsync returns StreamingResponse; producer snapshots cancellation before completing channel.
sdk_v2/cs/src/Request.cs Adds takeOwnership parameter and null guard.
sdk_v2/cs/src/Items/ItemQueue.cs Refactored to inherit from Item; resolves inner flItemQueue* once.
sdk_v2/cs/src/Items/Item.cs FromNative dispatches ItemType.Queue to ItemQueue.
sdk_v2/cs/src/Items/TensorItem.cs Adds AsSpan<T>, ToArray<T>, ElementCount with type-mapping.
sdk_v2/cs/test/FoundryLocal.Tests/{ChatSessionTests,AudioSessionTests}.cs Streaming + final-response test coverage.
sdk_v2/cs/test/Smoke.NetFx/* Project removed.
sdk_v2/python/src/foundry_local_sdk/session.py New StreamingResponse class and revised process_streaming_request.
sdk_v2/python/src/foundry_local_sdk/openai/live_audio_session.py Consumes new streaming shape; emits aggregated final transcript.
sdk_v2/python/src/foundry_local_sdk/items.py MessageItem.is_simple_text()/get_simple_text().
sdk_v2/python/src/foundry_local_sdk/{imodel,foundry_local_manager}.py Switch ffi.callback signatures to typedef'd names.
sdk_v2/python/src/foundry_local_sdk/init.py Exports ItemQueue, StreamingResponse.
sdk_v2/python/test/* Unit/integration test coverage for new APIs.
.github/instructions/cpp-build.instructions.md, memories/repo/*.md Documentation updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants