Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,9 @@ if (result.status === "aborted") {

</TypeScriptExample>

`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, and the signal is in memory only. If the Durable Object hibernates mid-turn and chat recovery is enabled, the recovered turn usually continues without the original signal; for pre-stream interruptions, recovery can instead retry the latest unanswered user message automatically. An abort fired after restart has no effect on the recovered turn.
`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, and the signal is in memory only. If the Durable Object hibernates mid-turn, durable recovery usually continues without the original signal. For pre-stream interruptions, recovery can retry the latest unanswered user message. An abort fired after restart has no effect on the recovered turn.

Persist cancellation intent when cancellation must survive a restart. Read that state in `onChatRecovery()` and return `{ continue: false }` to prevent another model call.

Use `cancelSubmission(submissionId)` for durable cancellation when work was accepted with `submitMessages()` or when cancellation must cross Worker and Durable Object RPC boundaries.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -397,7 +397,7 @@ if (result.status === "aborted") {

</TypeScriptExample>

`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, so construct the controller inside the Durable Object that calls `saveMessages()` or `continueLastTurn()`. The signal is in memory only; if the Durable Object hibernates mid-turn and `chatRecovery` is enabled, the recovered turn runs without the original signal.
`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, so construct the controller inside the Durable Object that calls `saveMessages()` or `continueLastTurn()`. The signal is in memory only. If the Durable Object hibernates mid-turn, the recovered turn runs without the original signal. Persist cancellation intent when cancellation must survive a restart.

### `onChatResponse`

Expand Down Expand Up @@ -575,25 +575,13 @@ Use `abortRequest()` when you know the request ID. Use `abortAllRequests()` for

### Stream recovery

Automatic stream resumption (the `resume` option on `useAgentChat`) is **client reconnect recovery** — it resumes an active stream when a client disconnects and reconnects. It does not cover Durable Object eviction: if the Worker process or Durable Object is evicted while the model call is in flight, the stream itself is gone. `chatRecovery` handles that case.
Automatic stream resumption (the `resume` option on `useAgentChat`) is **client reconnect recovery** — it resumes an active stream when a client disconnects and reconnects. It does not cover Durable Object eviction. If the Worker process or Durable Object is evicted while the model call is in flight, the stream itself is gone. Durable chat recovery handles that case.

When a Durable Object is evicted mid-stream (code update, inactivity timeout, resource limit), the LLM connection is severed permanently and the in-memory streaming state is lost. `chatRecovery` wraps each chat turn in a [`runFiber()`](/agents/runtime/execution/durable-execution/), providing automatic `keepAlive` during streaming and a recovery hook on restart.
When a Durable Object is evicted mid-stream, the LLM connection is severed permanently. Durable recovery wraps every `AIChatAgent` and [`Think`](/agents/harnesses/think/) chat turn in a [`runFiber()`](/agents/runtime/execution/durable-execution/). The fiber provides automatic `keepAlive` during streaming and a recovery hook on restart.

<TypeScriptExample>

```ts
export class ChatAgent extends AIChatAgent {
override chatRecovery = true;
}
```

</TypeScriptExample>
The fiber row survives in SQLite after an eviction. On the next activation, the framework detects the interrupted fiber. It reconstructs the partial response from buffered stream chunks and calls `onChatRecovery`.

`AIChatAgent` defaults `chatRecovery` to `false`, so existing chat agents only get client reconnect and resumable-stream behavior unless they opt in. [`Think`](/agents/harnesses/think/) defaults it to `true`.

When enabled, every `onChatMessage` call runs inside a fiber. If the agent is evicted mid-stream, the fiber row survives in SQLite. On the next activation, the framework detects the interrupted fiber, reconstructs the partial response from buffered stream chunks, and calls `onChatRecovery`.

`chatRecovery` can also be set to a configuration object to bound recovery and customize the terminal experience when recovery cannot succeed:
Durable recovery is always on. Use `chatRecovery` only to tune recovery budgets and terminal behavior:

<TypeScriptExample>

Expand All @@ -606,8 +594,8 @@ export class ChatAgent extends AIChatAgent {
// Primary stuck-turn bound. Resets on every progress-bearing attempt, so a
// turn that keeps producing content survives unbounded interruption.
noProgressTimeoutMs: 5 * 60 * 1000,
// Runaway-loop guard. Defaults to Infinity (no cap). Set a finite value to
// seal a turn that keeps emitting content but never converges.
// Runaway-loop guard. Defaults to 1,000. Set a higher value for a long
// agentic turn, or Infinity to remove the cap.
maxRecoveryWork: 200,
// Caller policy consulted from the second recovery attempt onward. Return
// false to stop recovery. This is where you enforce a token/cost budget.
Expand All @@ -634,7 +622,8 @@ The `chatRecovery` object accepts the following configuration options:
| `stableTimeoutMs` | `10_000` | How long a recovery attempt waits for the isolate to reach stable state before rescheduling. |
| `terminalMessage` | generic message | The message shown to the user when recovery is given up on. |
| `noProgressTimeoutMs` | `300_000` (5 min) | Primary stuck-turn bound: how long an incident may go without forward progress before it is sealed (`no_progress_timeout`). **Resets on every progress-bearing attempt**, so a turn that keeps producing content survives unbounded interruption. |
| `maxRecoveryWork` | `Infinity` | Runaway-loop guard. Maximum produced content/tool units since the incident began before a still-progressing turn is sealed. Defaults to no cap. |
| `maxRecoveryWork` | `1,000` | Runaway-loop guard. Maximum produced content/tool units since the incident began before a still-progressing turn is sealed. Set a higher value or `Infinity` for a long agentic turn. |
| `maxOomRetries` | `3` | Retry budget for Durable Object memory-limit resets. Set `0` to stop after the first memory-limit reset. |
| `shouldKeepRecovering` | — | Caller policy consulted from the second recovery attempt onward. Return `false` to stop recovery. Use it to enforce a token or cost budget. `ctx.work` is a coarse segment count, not tokens, so track real spend yourself. |
| `onExhausted` | — | Called once when recovery is given up on, before the terminal message is delivered. Inspect `ctx.reason` for why. |

Expand All @@ -651,16 +640,17 @@ The `chatRecovery` object accepts the following configuration options:
| `work` | `number` | Coarse, monotonic count of content/tool segments produced since the incident opened (not tokens). |
| `ageMs` | `number` | Wall-clock ms since the incident's first interruption. |

A progressing turn is never terminated by the framework on its own — it survives unbounded interruption (for example a dense deploy window) as long as it keeps making forward progress. Recovery is sealed only by one of these `ctx.reason` values:
A progressing turn survives repeated interruptions as long as it stays within the `maxRecoveryWork` limit. Recovery is sealed by one of these `ctx.reason` values:

- `no_progress_timeout` — no forward progress within the no-progress window (a stuck turn).
- `max_attempts_exceeded` — the attempt cap was spent on a tight no-progress alarm loop.
- `work_budget_exceeded` — the turn kept producing content but exceeded `maxRecoveryWork` (a runaway loop).
- `recovery_aborted` — your `shouldKeepRecovering` hook returned `false`.
- `out_of_memory` — recovery exceeded the memory-limit retry budget.
- `stable_timeout` — recovery attempts kept timing out waiting for stable state until the budget drained (extreme churn).

:::tip
A finite `maxRecoveryWork` can seal a legitimately long turn. Set a cap well above what a healthy turn produces, or use `shouldKeepRecovering` with real token or cost accounting for a precise budget.
The `maxRecoveryWork` default prevents a progressing turn from running forever. Increase it for long agentic turns. Use `shouldKeepRecovering` with durable token or cost data for a precise budget.
:::

#### Turns waiting on a human are not sealed
Expand Down Expand Up @@ -700,8 +690,6 @@ import type {
} from "@cloudflare/ai-chat";

export class ChatAgent extends AIChatAgent {
override chatRecovery = true;

override async onChatRecovery(
ctx: ChatRecoveryContext,
): Promise<ChatRecoveryOptions> {
Expand Down Expand Up @@ -763,6 +751,15 @@ override async onChatRecovery(
}
```

#### Control automatic continuation

Durable bookkeeping remains active when automatic continuation is not appropriate.

- Return `{ continue: false }` when another model call is unsafe.
- Persist cancellation intent and read it in `onChatRecovery()`.
- Record idempotency keys before external side effects.
- Use recovery budgets with durable spend data to limit cost.

#### `continueLastTurn`

Appends to the last assistant message by re-calling `onChatMessage` with the saved request body. The response is streamed as a continuation — appended to the existing assistant message, not a new one. No synthetic user message is created.
Expand All @@ -784,8 +781,6 @@ Use `this.stash()` inside `onChatMessage` to persist provider-specific data for

```ts
export class ChatAgent extends AIChatAgent {
override chatRecovery = true;

async onChatMessage(_onFinish, options) {
const result = streamText({
model: openai("gpt-5.4"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -540,7 +540,7 @@ For the full `subAgent()` API — typed RPC stubs, client routing, access contro

The patterns above handle the project manager's coordination work — scheduling, delegating, polling. But the project manager also uses an LLM directly: generating plans, summarizing progress, drafting status emails. Those LLM calls stream tokens over a connection that cannot be resumed if the agent is evicted mid-response.

For chat-oriented agents built on `AIChatAgent`, this is an even sharper problem — the user is watching the response stream in real time and sees it stop mid-sentence. `chatRecovery` wraps each chat turn in a `runFiber`, providing automatic `keepAlive` during streaming and a recovery hook when the agent restarts:
For chat-oriented agents built on `AIChatAgent` or `Think`, this is an even sharper problem — the user watches the response stream in real time and sees it stop mid-sentence. Durable recovery wraps every chat turn in a `runFiber`. This provides automatic `keepAlive` during streaming and a recovery hook when the agent restarts:

```ts
import { AIChatAgent } from "@cloudflare/ai-chat";
Expand All @@ -550,8 +550,6 @@ import type {
} from "@cloudflare/ai-chat";

class ProjectChat extends AIChatAgent<Env> {
override chatRecovery = true;

override async onChatRecovery(
ctx: ChatRecoveryContext,
): Promise<ChatRecoveryOptions> {
Expand All @@ -575,7 +573,7 @@ The right recovery strategy depends on the LLM provider:

Use `ctx.createdAt` to suppress stale recoveries. For example, if a recovered chat turn is older than a few minutes, you may persist the partial answer but skip automatic continuation to avoid surprising the user with an old response.

[`Think`](/agents/harnesses/think/) enables `chatRecovery` by default. The default path persists partial output and auto-continues or retries the turn when safe, so many apps do not need a custom hook. Override `onChatRecovery` when a provider has a better recovery strategy, or configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal user experience.
`AIChatAgent` and [`Think`](/agents/harnesses/think/) always use durable recovery. The default path persists partial output and continues or retries the turn when safe. Override `onChatRecovery` when a provider has a better recovery strategy. Configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal experience.

If the agent is interrupted before any assistant stream chunks are written, there is no partial assistant message to continue. When the latest persisted message is still the unanswered user message from that turn, chat recovery retries the turn automatically unless `onChatRecovery` returns `{ continue: false }`.

Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/agents/harnesses/think/client-tools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ When a turn produces several client tool calls at once, Think waits for **all**

## Survive restarts while waiting for a human

A Durable Object can be evicted at any time, including while a turn is paused on an approval prompt or a client-side tool call. Because `Think` enables [`chatRecovery`](/agents/harnesses/think/recovery/) by default, the SDK treats such a turn as waiting on the human, not stuck. It parks the turn instead of failing it, and the user's eventual approval or tool result resumes the conversation.
A Durable Object can be evicted at any time, including while a turn is paused on an approval prompt or a client-side tool call. [`Think` durable recovery](/agents/harnesses/think/recovery/) is always on. The SDK treats such a turn as waiting on the human, not stuck. It parks the turn instead of failing it. The user's eventual approval or tool result resumes the conversation.

For which interactions are exempt from recovery budgets, refer to [Turns waiting on a human are not sealed](/agents/communication-channels/chat/chat-agents/#turns-waiting-on-a-human-are-not-sealed).

Expand Down
4 changes: 2 additions & 2 deletions src/content/docs/agents/harnesses/think/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ Think is configured by overriding methods and properties on your `Think` subclas
| `messageConcurrency` | `"queue"` | How overlapping submits behave — refer to [Client tools](/agents/harnesses/think/client-tools/#message-concurrency) |
| `includeMcpTools` | `true` | Convert connected MCP tools to AI SDK tools and add them to model turns. Refer to [MCP tools](/agents/harnesses/think/tools/#mcp-tools) |
| `waitForMcpConnections` | `false` | Wait for MCP servers before inference |
| `chatRecovery` | `true` | Wrap WebSocket, sub-agent, programmatic, and continuation turns in `runFiber` for durable execution. Set to a configuration object with `maxAttempts`, `stableTimeoutMs`, `terminalMessage`, and `onExhausted` to tune bounded recovery |
| `chatStreamStallTimeoutMs` | `0` (off) | Opt-in inactivity watchdog: abort a turn whose model stream produces no chunk for this long (measures the gap between chunks, including tool execution). With `chatRecovery` on, a stall routes into bounded recovery |
| `chatRecovery` | Always on | Durable recovery configuration. Refer to [Durable recovery](/agents/harnesses/think/recovery/) for all options and defaults |
| `chatStreamStallTimeoutMs` | `0` (off) | Opt-in inactivity watchdog: abort a turn whose model stream produces no chunk for this long (measures the gap between chunks, including tool execution). A stall routes into bounded recovery |
| `contextOverflow` | `undefined` | Opt-in mid-turn context-overflow handling with `reactive`, `maxRetries`, and `proactive` options. Requires `classifyChatError` plus a session compaction function — refer to [Context-window overflow recovery](/agents/harnesses/think/recovery/#context-window-overflow-recovery) |

For `chatRecovery` and `chatStreamStallTimeoutMs` behavior, refer to [Durable recovery](/agents/harnesses/think/recovery/).
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/agents/harnesses/think/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ Key behaviors:

- **Blocking modes cannot nest.** Calling `wait`/`stream`/`continuation` (or the equivalent shortcut) from _inside_ an active turn — for example, from a tool's `execute` — throws, because it would deadlock the turn queue. From inside a turn, use `runTurn({ mode: "submit" })` (durable, runs after the current turn frees the queue) or [`addMessages()`](#add-messages-without-a-turn) (transcript only, no inference).
- **`submit` is idempotent.** Pass `submissionId` and/or `idempotencyKey`; re-submitting a known key returns the existing record with `accepted: false` instead of starting a second turn. See [Programmatic submissions](/agents/harnesses/think/programmatic-submissions/).
- **Recovery-safe.** When `chatRecovery` is enabled, the `wait`, `stream`, and drained `submit` paths all run inference inside a recovery fiber, so an interrupted turn resumes after eviction.
- **Recovery-safe.** The `wait`, `stream`, and drained `submit` paths run inference inside a recovery fiber, so an interrupted turn resumes after eviction.

`runTurn` is exported alongside its option and result types: `RunTurnOptions`, `RunTurnWait`, `RunTurnSubmit`, `RunTurnStream`, `TurnInputMessages`, and `TurnResult`.

Expand Down
Loading