fix(engine): cache response codecs and bound tool execution

[danieljvdm]

Reuse each live Toolkit's response codec across streamed chunks, bound tool-batch fibers by the resolved concurrency, and return broker results from the exact JSON bytes admitted by the broker. Published tool progress now uses owned snapshots with an 8 MiB cumulative payload allowance per run, including detached replay.

The native Channel merge keeps a finite number of call fibers even at concurrency one. Using Stream.mergeAll directly selects its sequential fast path at one, which broke existing early-close telemetry and failure-observer tests. The explicit tool semaphore and sequential barriers remain in place.

sequenceDiagram
  participant Handler
  participant Broker
  participant Redactor
  participant Caller
  Handler->>Broker: encoded result
  opt redactor configured
    Broker->>Redactor: validated JSON
    Redactor-->>Broker: replacement result
  end
  Broker->>Broker: serialize once, admit bytes, decode owned snapshot
  Broker-->>Caller: ProgrammaticCallSuccess with admitted snapshot

Loading

Callers can tighten the progress allowance:

AgentRuntime.start(agent, input, {
  bufferLimits: { maxToolProgressBytes: 1024 * 1024 },
})

Exceeding that cumulative progress allowance fails with ModelProtocolError; progress is not truncated. Application progress must be plain JSON; accessors and custom serialization are rejected. Terminal tool results retain their separate toolResultBounds policy. This PR bounds published payloads. The native Toolkit producer queue needs the separate upstream backpressure fix, which is not yet part of the pinned Effect release.

A local Node 24.20.0 ARM64 benchmark measured complete AgentRuntime.run calls, each streaming 1,005 parts for a 16,000-character ASCII string answer without invoking tools. One warm-up preceded five measured samples per case in separate baseline and candidate processes:

Declared tools	Baseline median ms	Baseline range ms	Candidate median ms	Candidate range ms
0	144.07	143.55 to 147.93	131.98	128.06 to 137.83
10	1249.26	1123.58 to 1270.81	154.95	126.90 to 233.50
50	5277.70	5195.32 to 5429.43	133.93	130.50 to 136.29

These are local full-run measurements. Hosted CPU and provider latency were not measured. A separate blocked-handler diagnostic at concurrency two measured 11 suspended fibers for both 4-call and 64-call batches after the fix, versus 11 and 71 before.

Validation passed with 311 engine tests, engine type checks, and vp run ready. Regressions cover mutable/getter-backed broker results, detached progress ownership and byte limits, provider progress accounting, declaration order, and cleanup at concurrency one and two. The new progress cohort failed five cases before the fix and passed all six afterward.

The first full gate hit an intermittent existing Cloudflare alarm assertion at alarm.test.ts:274. An isolated Cloudflare rerun passed 419 tests with two skips, then the full gate passed at the unchanged commit. Independent source review found no further correctness issues.

[effect-agent]

Effect Agent review

Tip

No actionable findings.

Scope	Files	New findings
Full diff	11 reviewed	✅ None

Summary

No concrete defects found in the supplied change.

_{3 model calls · 61,918 input (675 uncached · 11,214 cached · 50,029 cache write; 18.1% cache reads) / 1,873 output tokens · ≈ $0.2948 at GPT-5.6 Sol rates · $0.999999 spending ceiling · inspected at c49e015 · 4 automatic reviews remain}

[github-actions]

Bundle size

Fixture	Part	Base gzip	PR gzip	Change	PR minified
agent-root	initial	81.07 kB	81.07 kB	0.00 kB / 0.00%	263.91 kB
agent-module	initial	80.77 kB	80.77 kB	0.00 kB / 0.00%	263.05 kB
runtime-root	initial	165.53 kB	167.69 kB	+2.16 kB / +1.30%	553.12 kB
runtime-module	initial	164.83 kB	166.87 kB	+2.04 kB / +1.24%	551.13 kB
lazy-root	initial	166.51 kB	168.63 kB	+2.12 kB / +1.27%	555.63 kB
lazy-root	deferred	0.08 kB	0.08 kB	0.00 kB / 0.00%	0.07 kB
lazy-root	total	166.60 kB	168.72 kB	+2.12 kB / +1.27%	555.70 kB
lazy-module	initial	134.73 kB	136.83 kB	+2.10 kB / +1.56%	438.67 kB
lazy-module	deferred	31.62 kB	31.94 kB	+0.32 kB / +1.01%	113.90 kB
lazy-module	total	166.35 kB	168.77 kB	+2.42 kB / +1.45%	552.57 kB

Minified ESM for es2022, browser target, including Effect and other dependencies. Gzip is measured per chunk. Initial includes statically imported shared chunks; deferred is the remaining output. New exports have no prior baseline.

Chunks, module analysis, and exact bytes for c49e01566825.