Skip to content

refactor: share provider-failure classification between the Action and the gateway - #1869

Merged
marcusrbrown merged 9 commits into
mainfrom
feat/shared-provider-failure
Oct 19, 2026
Merged

marcusrbrown merged 9 commits into
mainfrom
feat/shared-provider-failure

Conversation

@marcusrbrown

Copy link
Copy Markdown
Collaborator

Until now the Action and the gateway each had their own copy of provider-failure classification, and the copies disagreed. A quota retry stopped an Action run, while the gateway waited out OpenCode's whole retry budget. A model retry did the reverse. The two surfaces also read error payloads differently, and the Action's SSE and poll paths labeled the same error differently. Fixes kept landing on one side only (#1283 on the Action, #1854 on the gateway).

This is the first step in consolidating the Action and gateway (see docs/brainstorms/2026-10-18-action-gateway-consolidation-requirements.md). One runtime function, classifyProviderFailure, now reads a provider signal and returns the failure kind, the strength of the evidence, whether the run stops now, and the label. Both surfaces call it and act on its stop decision, and their own copies are gone.

The rule

  • A run stops on a structured signal (auth, quota, model not found) or on two established text patterns: the usage-limit wording and Model not found:.
  • Looser wording, such as auth phrasing or "model is not supported", labels the failure but never stops the run. It never turns a retryable error into a non-retryable one.
  • On a retry status, the run stops for auth_unavailable, account_rate_limit, model_not_found and model_not_supported. Any other retry keeps waiting.
  • A single typed extractor, terminalSignalFromPayload, reads OpenCode error payloads for both surfaces. A parity test runs the same fixtures through both and requires them to agree.
  • Some behavior stays per surface on purpose: abort detection, overflow recovery (the Action restarts in a fresh session, the gateway waits for compaction), descendant-session errors, and how each event loop treats errors that aren't provider failures.

Behavior changes

Surface Signal Before After
Gateway account_rate_limit retry waits out the retry budget stops with provider-quota-exceeded
Gateway session.error with 402, insufficient_quota, or usage-limit text stops with unclassified stops with provider-quota-exceeded
Gateway assistant message error with 402, insufficient_quota, or usage-limit text fails as provider-rejected-request or unclassified fails with provider-quota-exceeded
Action model_not_found / model_not_supported retry waits stops with a model-not-found error
Action poll assistant error with usage-limit text generic error, exit 0 when a comment target exists quota error, exit 1
Action terminal payload with auth wording or "model is not supported" generic agent error, exit 0 when a comment target exists provider-auth or model-not-found label, exit 1; timing unchanged
Action model retry on a descendant session ignored marks that ledger entry unknown; root unaffected
Both error payload whose top-level and data fields disagree each surface read its own field first both read data.* first (name top level first), skipping wrong-typed values

Operator contract 1.13.0

  • Adds the provider failure reason provider-quota-exceeded. This is a MINOR bump.
  • The dashboard pins the contract exactly, so the gateway and the dashboard must be deployed together. Merging this PR on its own is safe.

Also

  • The per-kind classifiers are no longer exported from the runtime package root, so nothing can bypass the stop rule.
  • A guard test keeps @actions/* imports out of packages/runtime.
  • Usage-limit text matching is linear, so a long repeated phrase no longer stalls the event loop.
  • Docs: the consolidation requirements doc and two solution docs are updated.

Verification

  • Every behavior change has a test that fails when the change is reverted. The parity test covers 17 payload shapes.
  • check-types, lint, test:scripts (688), the full test run, and build all pass. Test counts: runtime 1104, action 2594, workspace-agent 773, harness 439, gateway 5330, evals 173.
  • dist/ is rebuilt.

@fro-bot fro-bot left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CONDITIONAL

Blocking issues

Severity Location Issue and required fix
P2 provider-failure.ts:142–143 Hint labels revoke existing retries. labelHint preserves retryability only for explicit isRetryable: true, bypassing the generic ladder’s retryable HTTP 429 and fetch-error cases. For example, {name: 'APIError', data: {statusCode: 429, message: 'no auth available'}} and the string 'fetch failed: no auth available' now return retryable: false despite stop: false. The Action’s attempt-outcome.ts:126 consequently selects a terminal outcome instead of retrying. Derive hint retryability from the generic classification, retaining the hint label and stop: false.

Non-blocking concerns

None

Missing tests

  • Add auth/model hint cases combined with HTTP 429 and an absent/false isRetryable, plus plain-string fetch failures. Assert preserved retryable: true and Action retry admission through the SSE/poll failure paths. Current hint tests cover explicit retryability but miss these implicit retry sources.

Risk assessment (MED): likelihood of regression, security exposure, or blast radius

The regression prevents recovery from transient provider failures in the Action. The operator-contract change also requires the documented paired dashboard deployment.


Run Summary
Field Value
Event pull_request
Repository fro-bot/agent
Run ID 39209648659
Cache hit
Session ses_ead2871c1ffekFNuUZOJVNllhe
Verification Source/test inspection and one focused probe; no build, lint, type-check, or test-suite commands
Coverage Seven reviewer returns structurally admitted; adversarial return rejected for schema errors. Its surface was independently covered by correctness/reliability review and the probe. Aggregation did not complete.
Delivery Response-file review; this protocol does not support inline annotations, so the finding links to the exact source lines.

The following probe tested whether adding hint wording removes otherwise established retryability. Both the 429 and fetch-error controls returned retryable: true; adding auth wording returned retryable: false. All four returned stop: false.

bun -e 'import {classifyProviderFailure, terminalSignalFromPayload} from "./packages/runtime/src/agent/error-format/provider-failure.ts"; for (const payload of [{name:"APIError",data:{statusCode:429,message:"temporary outage"}},{name:"APIError",data:{statusCode:429,message:"no auth available"}},"fetch failed", "fetch failed: no auth available"]) {const result=classifyProviderFailure(terminalSignalFromPayload(payload,{detail:typeof payload==="string"?payload:"Session error: APIError (status 429)"})); console.log(JSON.stringify({payload,kind:result.kind,stop:result.stop,retryable:result.errorInfo.retryable}));}'

@fro-bot fro-bot left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: PASS

The previous blocker is resolved in a9c5336: hint labels now preserve generic retryability. Added tests cover HTTP 429 with absent/false retry flags, fetch-error hints, and Action retry admission through SSE and polling.

Blocking issues

None

Non-blocking concerns

None

Missing tests

None

Risk assessment (LOW): likelihood of regression, security exposure, or blast radius

Low incremental risk from this focused fix. The documented paired gateway/dashboard deployment requirement for contract 1.13.0 remains applicable.


Run Summary
Field Value
Event pull_request
Repository fro-bot/agent
Run ID 39213165426
Cache hit
Session ses_ead2871c1ffekFNuUZOJVNllhe
Scope Follow-up commit a9c5336 against previously reviewed 0e28e76; prior blocker and regression coverage
Review Correctness, testing, project standards, reliability, prior feedback, and TypeScript safety; all six returns admitted with no findings
Verification Diff/source/test inspection only; no build, lint, type-check, test suite, or empirical probe run

@marcusrbrown
marcusrbrown merged commit 1c4d5f5 into main Oct 19, 2026
15 checks passed
@marcusrbrown
marcusrbrown deleted the feat/shared-provider-failure branch October 19, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants