Skip to content

feat: expose tool failure modes and handling diagnostics - #423

Merged
danieljvdm merged 2 commits into
mainfrom
dan/effect-tool-error-diagnostics
Sep 10, 2026
Merged

danieljvdm merged 2 commits into
mainfrom
dan/effect-tool-error-diagnostics

Conversation

@danieljvdm

Copy link
Copy Markdown
Owner

Effect's default failureMode: "error" can end an agent run even when a tool declares a failure schema. Tool failure events and telemetry previously did not distinguish these propagated failures from failures returned as results.

Add Agent.inspectTools for inspecting the native configuration of a Definition or Binding without acquiring handlers or a model:

import { Agent } from "effect-agent";
import { Schema } from "effect";
import { Tool, Toolkit } from "effect/unstable/ai";

const agent = Agent.make("lookup", {
  input: Schema.String,
  output: Schema.String,
  instructions: "Look up the answer.",
  toolkit: Toolkit.make(Tool.make("lookup", {
    success: Schema.String,
    failure: Schema.String,
    failureMode: "return",
  })),
});

Agent.inspectTools(agent);
// [{ name: "lookup", failureMode: "return", requiresHandler: true }]

ToolCallFailed gains optional failureMode and failureHandling fields. Application tool spans and terminal logs expose the same distinction through effect_agent.tool.failure_mode and effect_agent.tool.failure_handling: propagated, returned-to-model, or returned-to-caller for broker outcomes. Older events remain decodable with an unknown route.

The configured mode is not a fatal/recoverable boolean: a defect can propagate under "return", and the broker can contain a typed failure under "error". returned-to-model identifies the result path; delivery still requires the complete batch to commit and another model call. Preserve native failure behavior and error/service inference, and document the observer's scope and strict preflight validation.

Validation: vp run ready passed, including the full test suite, package checks, documentation examples, and builds. Focused coverage also passed all 196 tests across inspection, schema compatibility, runtime diagnostics, broker outcomes, and failure observers; explicit core/engine type checks passed.

Related upstream documentation PR: Effect-TS/effect#8156.

@effect-agent effect-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effect Agent review

Caution

This attempt is incomplete because the pull request moved to a newer commit.

No findings were published from this attempt.

This notice records the attempt on the inspected commit. The newer commit still needs its own review.

@effect-agent effect-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effect Agent review

Tip

No actionable findings.

Scope Files New findings
Full diff 13 reviewed ✅ None

Summary

No concrete defects found in the supplied change.

8 model calls · 140,391 input (1,832 uncached · 112,988 cached · 25,571 cache write; 80.5% cache reads) / 730 output tokens · ≈ $0.9749 at GPT-6 Astra rates · $20.396890 spending ceiling · inspected at 2b4bcbc · 3 automatic reviews remain

@github-actions

Copy link
Copy Markdown
Contributor

Bundle size

Fixture Part Base gzip PR gzip Change PR minified
agent-root initial 81.25 kB 82.73 kB +1.48 kB / +1.82% 268.98 kB
agent-module initial 80.95 kB 80.96 kB +0.02 kB / +0.02% 263.75 kB
runtime-root initial 180.26 kB 180.46 kB +0.20 kB / +0.11% 597.56 kB
runtime-module initial 180.59 kB 180.93 kB +0.33 kB / +0.18% 595.43 kB
lazy-root initial 181.97 kB 182.43 kB +0.46 kB / +0.25% 599.43 kB
lazy-root deferred 0.08 kB 0.08 kB 0.00 kB / 0.00% 0.07 kB
lazy-root total 182.06 kB 182.51 kB +0.46 kB / +0.25% 599.50 kB
lazy-module initial 137.86 kB 137.89 kB +0.03 kB / +0.02% 441.73 kB
lazy-module deferred 42.85 kB 43.20 kB +0.35 kB / +0.82% 153.84 kB
lazy-module total 180.71 kB 181.09 kB +0.38 kB / +0.21% 595.56 kB

Minified ESM for es2022, browser target, including Effect and other dependencies. Gzip is measured per chunk. Initial includes statically imported shared chunks; deferred is the remaining output. New exports have no prior baseline.

Chunks, module analysis, and exact bytes for 2b4bcbc84022.

@github-actions

Copy link
Copy Markdown
Contributor

Runtime performance

Informational medians in milliseconds, nine samples per workload and revision. Same fixture bytes and Node runtime; separate lockfiles and public builds.

Workload Base Head Head/base
small-run 2.81 2.71 -3.4%
small-stream 3.08 2.60 -15.7%
stream-64k-1 4.11 3.65 -11.2%
stream-64k-64 7.42 7.05 -5.0%
stream-64k-1024 59.90 59.47 -0.7%
stream-64k-4096 224.28 216.00 -3.7%
history-65536 4.09 4.36 6.4%
history-1048576 19.38 19.19 -1.0%
parallel-tools-8 16.24 16.54 1.8%
tool-rounds-4 59.34 61.92 4.3%
durable-fresh-0 42.71 36.13 -15.4%
checkpoint-recovery-0 29.80 33.46 12.3%
settled-ledger-0 47.43 37.82 -20.3%
durable-fresh-256 77.74 82.37 6.0%
checkpoint-recovery-256 29.46 30.08 2.1%
settled-ledger-256 42.73 38.11 -10.8%
durable-fresh-2048 389.92 380.45 -2.4%
checkpoint-recovery-2048 30.95 32.23 4.1%
settled-ledger-2048 37.65 37.98 0.9%

Raw samples, spread, cold process totals, failures, environment, and exact revisions for 2b4bcbc84022. Timing deltas are not a merge threshold.

@danieljvdm
danieljvdm merged commit 84d6684 into main Sep 10, 2026
12 checks passed
@danieljvdm
danieljvdm deleted the dan/effect-tool-error-diagnostics branch September 10, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant