Conversation
Design note 1: why I modelled this as a per-request snapshotThe invariant I started from was: the app should be able to change the instructions and tools for the next request without changing the identity of the I considered mutating That led me to treat a session as two different kinds of state:
This is also why |
Design note 2: the tool snapshot is a causality rule, not just an implementation detailThe subtle case for me was a tool continuation. Suppose request A is sent with Tool A available, and the model returns a call to Tool A. Before the continuation request, application state changes and the dynamic body now resolves to Tool B. I think two different moments need two different rules:
In other words, the request context provides causal consistency for one provider turn; it is not a cache for the whole response loop. The behavioral tests exercise this for both streaming and non-streaming paths. They assert that the model sees A and then B, while the executed tool is still A. The failure and cancellation cases also check that a completed dynamic-tool side effect is not replayed on a later retry. I included those cases because a design that works only for the happy-path request/continuation sequence would not be safe enough for session orchestration. |
Process note: why I proved the full path first, but do not expect this to land as one large PRI implemented this downstream-first because the main uncertainty was semantic rather than syntactic: would the same-session model still behave correctly through real orchestration, tool execution, persistence, and restoration? I first exercised the design across AnyLanguageModel, AIReasoningCore, and SwiftChat, including the Context A / Tool A → Context B / Tool B transition. That gave me evidence for the ownership boundary before asking upstream to commit to the public API. The resulting draft touches many built-in adapters because each adapter currently chooses where to read My current decomposition is only a proposal:
There are reasons to reorder that series, especially if the public API should lead the implementation, so I would rather get maintainer guidance before manufacturing a stack of PRs. #264 also changes reasoning/transcript construction in several of the same adapters; waiting for its direction avoids repeatedly rebasing mechanical code and obscuring its semantic changes. Two surface questions remain intentionally open in this draft:
I have intentionally left broader state-ownership concepts such as |
@mattt — opening this as a draft early to align on the API shape, landing order, and how you would prefer it split before I invest in polishing it for merge.
Intent
This adds a Foundation Models 27-shaped
DynamicInstructionsAPI for request-scoped instructions and tools. The same design is already exercised end to end in a downstream stack, but this PR is deliberately not ready to merge yet.The proposed behavior is:
DynamicInstructionssupports builder composition, conditionals, collections, tools, and type erasure.LanguageModelSession.init(model:dynamicInstructions:history:)reevaluates the body immediately before every model request, including tool-continuation requests.DynamicInstructionsitself.Base and current overlap
This draft is rebased on current
mainat6da8838(0.14.1). In particular, it preserves the Anthropic blank-text filtering merged in #271.#264 still overlaps several provider adapter files. I am intentionally opening this before choosing a final merge/rebase strategy so we can avoid optimizing the series around assumptions that may change when #264 lands.
If the overall direction looks useful, a possible mini-PR sequence is:
DynamicInstructionsbuilder/value surface and API compatibility coverage;I am happy to reorder or reshape that split based on your preference.
Downstream evidence
The earlier downstream version shipped through:
qoli/AnyLanguageModel@fde42a4qoli/AIReasoningCore@68c012cqoli/SwiftChat@56cef0bThat integration covered a same-session transition from Context A / Tool A to Context B / Tool B while preserving the prior transcript, including the tool-continuation path.
Known design questions / non-goals
model; I would like to settle whether parity or the existing AnyLanguageModel convention should win before changing that surface.SessionProperty/DynamicProfile-style state ownership is intentionally not included here.Local verification
swift format lint --strict --recursive .git diff --checkCI=1 swift test— 608 tests across 61 suites, 0 failuresbuild-for-testingNo live provider calls were made. The full upstream toolchain / Linux / traits matrix is left to GitHub Actions.