fix(ai): bound Toolkit preliminary result buffering

[danieljvdm]

Toolkit.handle puts preliminary results in a private, unbounded queue. A handler can keep filling it while its consumer is paused, even when it yields every preliminary call. Downstream buffer limits cannot constrain that queue.

Bound it to 16 entries so producers wait for consumers. Use Stream.ensuring instead of onEnd to interrupt handlers when consumers stop early or cancel, including producers waiting on a full queue.

sequenceDiagram
  participant Handler
  participant Queue as Native Toolkit queue
  participant Consumer as Returned Stream
  Handler->>Queue: preliminary(result)
  Note over Handler,Queue: At most 16 queued results<br/>Further offers suspend
  Consumer->>Queue: consume result
  Queue-->>Handler: resume pending offer
  Consumer->>Handler: interrupt on stream close

Loading

With consumption paused, the regression accepts 64 updates before the fix and 16 after. Tests also verify ordered delivery, success/failure settlement, and cleanup while the parent remains alive. All 51 Tool tests, type checks, and lint pass. The docs check reports six existing diagnostics in Multipart.ts and ChildProcess.ts.

[changeset-bot]

🦋 Changeset detected

Latest commit: 132d312

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages

Name	Type
effect	Patch
@effect/opentelemetry	Patch
@effect/vitest	Patch
@effect/ai-anthropic	Patch
@effect/ai-openai-compat	Patch
@effect/ai-openai	Patch
@effect/ai-openrouter	Patch
@effect/atom-react	Patch
@effect/atom-solid	Patch
@effect/atom-vue	Patch
@effect/platform-browser	Patch
@effect/platform-bun	Patch
@effect/platform-deno	Patch
@effect/platform-node-shared	Patch
@effect/platform-node	Patch
@effect/sql-clickhouse	Patch
@effect/sql-d1	Patch
@effect/sql-libsql	Patch
@effect/sql-mssql	Patch
@effect/sql-mysql2	Patch
@effect/sql-pg	Patch
@effect/sql-pglite	Patch
@effect/sql-sqlite-bun	Patch
@effect/sql-sqlite-do	Patch
@effect/sql-sqlite-node	Patch
@effect/sql-sqlite-react-native	Patch
@effect/sql-sqlite-wasm	Patch
@effect/docgen	Patch
@effect/doctest	Patch
@effect/openapi-generator	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name	Current Size	Previous Size	Difference
arbitrary-combinators.ts	34.05 KB	34.05 KB	0.00 KB (0.00%)
basic.ts	7.15 KB	7.15 KB	0.00 KB (0.00%)
batching.ts	10.38 KB	10.38 KB	0.00 KB (0.00%)
brand.ts	6.63 KB	6.63 KB	0.00 KB (0.00%)
cache.ts	11.03 KB	11.03 KB	0.00 KB (0.00%)
config.ts	21.76 KB	21.76 KB	0.00 KB (0.00%)
differ.ts	20.55 KB	20.55 KB	0.00 KB (0.00%)
http-client.ts	22.20 KB	22.20 KB	0.00 KB (0.00%)
http-router.ts	32.91 KB	32.91 KB	0.00 KB (0.00%)
logger.ts	11.12 KB	11.12 KB	0.00 KB (0.00%)
metric.ts	9.28 KB	9.28 KB	0.00 KB (0.00%)
optic.ts	6.87 KB	6.87 KB	0.00 KB (0.00%)
pubsub.ts	15.49 KB	15.49 KB	0.00 KB (0.00%)
queue.ts	12.09 KB	12.09 KB	0.00 KB (0.00%)
schedule.ts	11.20 KB	11.20 KB	0.00 KB (0.00%)
schema-binary.ts	39.71 KB	39.71 KB	0.00 KB (0.00%)
schema-class.ts	20.32 KB	20.32 KB	0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts	30.48 KB	30.48 KB	0.00 KB (0.00%)
schema-representation-roundtrip.ts	26.42 KB	26.42 KB	0.00 KB (0.00%)
schema-string-transformation.ts	13.89 KB	13.89 KB	0.00 KB (0.00%)
schema-string.ts	11.38 KB	11.38 KB	0.00 KB (0.00%)
schema-template-literal.ts	15.76 KB	15.76 KB	0.00 KB (0.00%)
schema-toArbitrary.ts	33.60 KB	33.60 KB	0.00 KB (0.00%)
schema-toCodeDocument.ts	24.76 KB	24.76 KB	0.00 KB (0.00%)
schema-toCodecJson.ts	19.52 KB	19.52 KB	0.00 KB (0.00%)
schema-toEquivalence.ts	19.65 KB	19.65 KB	0.00 KB (0.00%)
schema-toFormatter.ts	19.77 KB	19.77 KB	0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts	23.85 KB	23.85 KB	0.00 KB (0.00%)
schema-toRepresentation.ts	19.79 KB	19.79 KB	0.00 KB (0.00%)
schema.ts	19.53 KB	19.53 KB	0.00 KB (0.00%)
stm.ts	13.03 KB	13.03 KB	0.00 KB (0.00%)
stream.ts	10.06 KB	10.06 KB	0.00 KB (0.00%)

[kvnloo]

Exact-head comment on #8065 head 132d312.

KEEP bound Toolkit preliminary Queue to capacity 16 (backpressure while full) and Stream.ensuring(Fiber.interrupt(fiber)) so handlers stop when the result stream closes.

CHECK new Tool.test.ts backpressure case pins accepted==16 while latch open, then drains to 64 for both success and failure settlement.

Author danieljvdm not kvnloo. No competing PR from me.