From 9e59903a423a2b83fdad8a75212b8a29b805ec9d Mon Sep 17 00:00:00 2001 From: Ben Reitz Date: Fri, 7 Aug 2026 16:41:30 +0100 Subject: [PATCH] docs: document unconditional chat recovery --- .../chat/autonomous-responses.mdx | 4 +- .../chat/chat-agents.mdx | 47 +++++++++---------- .../agentic-patterns/long-running-agents.mdx | 6 +-- .../agents/harnesses/think/client-tools.mdx | 2 +- .../agents/harnesses/think/configuration.mdx | 4 +- .../docs/agents/harnesses/think/index.mdx | 2 +- .../docs/agents/harnesses/think/recovery.mdx | 19 ++++---- .../runtime/execution/durable-execution.mdx | 2 +- .../observability/diagnostics-channels.mdx | 2 +- 9 files changed, 43 insertions(+), 45 deletions(-) diff --git a/src/content/docs/agents/communication-channels/chat/autonomous-responses.mdx b/src/content/docs/agents/communication-channels/chat/autonomous-responses.mdx index 6b7d04f96b0..69eef060c82 100644 --- a/src/content/docs/agents/communication-channels/chat/autonomous-responses.mdx +++ b/src/content/docs/agents/communication-channels/chat/autonomous-responses.mdx @@ -504,7 +504,9 @@ if (result.status === "aborted") { -`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, and the signal is in memory only. If the Durable Object hibernates mid-turn and chat recovery is enabled, the recovered turn usually continues without the original signal; for pre-stream interruptions, recovery can instead retry the latest unanswered user message automatically. An abort fired after restart has no effect on the recovered turn. +`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, and the signal is in memory only. If the Durable Object hibernates mid-turn, durable recovery usually continues without the original signal. For pre-stream interruptions, recovery can retry the latest unanswered user message. An abort fired after restart has no effect on the recovered turn. + +Persist cancellation intent when cancellation must survive a restart. Read that state in `onChatRecovery()` and return `{ continue: false }` to prevent another model call. Use `cancelSubmission(submissionId)` for durable cancellation when work was accepted with `submitMessages()` or when cancellation must cross Worker and Durable Object RPC boundaries. diff --git a/src/content/docs/agents/communication-channels/chat/chat-agents.mdx b/src/content/docs/agents/communication-channels/chat/chat-agents.mdx index b7b7be84537..28a70c3970c 100644 --- a/src/content/docs/agents/communication-channels/chat/chat-agents.mdx +++ b/src/content/docs/agents/communication-channels/chat/chat-agents.mdx @@ -397,7 +397,7 @@ if (result.status === "aborted") { -`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, so construct the controller inside the Durable Object that calls `saveMessages()` or `continueLastTurn()`. The signal is in memory only; if the Durable Object hibernates mid-turn and `chatRecovery` is enabled, the recovered turn runs without the original signal. +`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, so construct the controller inside the Durable Object that calls `saveMessages()` or `continueLastTurn()`. The signal is in memory only. If the Durable Object hibernates mid-turn, the recovered turn runs without the original signal. Persist cancellation intent when cancellation must survive a restart. ### `onChatResponse` @@ -575,25 +575,13 @@ Use `abortRequest()` when you know the request ID. Use `abortAllRequests()` for ### Stream recovery -Automatic stream resumption (the `resume` option on `useAgentChat`) is **client reconnect recovery** — it resumes an active stream when a client disconnects and reconnects. It does not cover Durable Object eviction: if the Worker process or Durable Object is evicted while the model call is in flight, the stream itself is gone. `chatRecovery` handles that case. +Automatic stream resumption (the `resume` option on `useAgentChat`) is **client reconnect recovery** — it resumes an active stream when a client disconnects and reconnects. It does not cover Durable Object eviction. If the Worker process or Durable Object is evicted while the model call is in flight, the stream itself is gone. Durable chat recovery handles that case. -When a Durable Object is evicted mid-stream (code update, inactivity timeout, resource limit), the LLM connection is severed permanently and the in-memory streaming state is lost. `chatRecovery` wraps each chat turn in a [`runFiber()`](/agents/runtime/execution/durable-execution/), providing automatic `keepAlive` during streaming and a recovery hook on restart. +When a Durable Object is evicted mid-stream, the LLM connection is severed permanently. Durable recovery wraps every `AIChatAgent` and [`Think`](/agents/harnesses/think/) chat turn in a [`runFiber()`](/agents/runtime/execution/durable-execution/). The fiber provides automatic `keepAlive` during streaming and a recovery hook on restart. - - -```ts -export class ChatAgent extends AIChatAgent { - override chatRecovery = true; -} -``` - - +The fiber row survives in SQLite after an eviction. On the next activation, the framework detects the interrupted fiber. It reconstructs the partial response from buffered stream chunks and calls `onChatRecovery`. -`AIChatAgent` defaults `chatRecovery` to `false`, so existing chat agents only get client reconnect and resumable-stream behavior unless they opt in. [`Think`](/agents/harnesses/think/) defaults it to `true`. - -When enabled, every `onChatMessage` call runs inside a fiber. If the agent is evicted mid-stream, the fiber row survives in SQLite. On the next activation, the framework detects the interrupted fiber, reconstructs the partial response from buffered stream chunks, and calls `onChatRecovery`. - -`chatRecovery` can also be set to a configuration object to bound recovery and customize the terminal experience when recovery cannot succeed: +Durable recovery is always on. Use `chatRecovery` only to tune recovery budgets and terminal behavior: @@ -606,8 +594,8 @@ export class ChatAgent extends AIChatAgent { // Primary stuck-turn bound. Resets on every progress-bearing attempt, so a // turn that keeps producing content survives unbounded interruption. noProgressTimeoutMs: 5 * 60 * 1000, - // Runaway-loop guard. Defaults to Infinity (no cap). Set a finite value to - // seal a turn that keeps emitting content but never converges. + // Runaway-loop guard. Defaults to 1,000. Set a higher value for a long + // agentic turn, or Infinity to remove the cap. maxRecoveryWork: 200, // Caller policy consulted from the second recovery attempt onward. Return // false to stop recovery. This is where you enforce a token/cost budget. @@ -634,7 +622,8 @@ The `chatRecovery` object accepts the following configuration options: | `stableTimeoutMs` | `10_000` | How long a recovery attempt waits for the isolate to reach stable state before rescheduling. | | `terminalMessage` | generic message | The message shown to the user when recovery is given up on. | | `noProgressTimeoutMs` | `300_000` (5 min) | Primary stuck-turn bound: how long an incident may go without forward progress before it is sealed (`no_progress_timeout`). **Resets on every progress-bearing attempt**, so a turn that keeps producing content survives unbounded interruption. | -| `maxRecoveryWork` | `Infinity` | Runaway-loop guard. Maximum produced content/tool units since the incident began before a still-progressing turn is sealed. Defaults to no cap. | +| `maxRecoveryWork` | `1,000` | Runaway-loop guard. Maximum produced content/tool units since the incident began before a still-progressing turn is sealed. Set a higher value or `Infinity` for a long agentic turn. | +| `maxOomRetries` | `3` | Retry budget for Durable Object memory-limit resets. Set `0` to stop after the first memory-limit reset. | | `shouldKeepRecovering` | — | Caller policy consulted from the second recovery attempt onward. Return `false` to stop recovery. Use it to enforce a token or cost budget. `ctx.work` is a coarse segment count, not tokens, so track real spend yourself. | | `onExhausted` | — | Called once when recovery is given up on, before the terminal message is delivered. Inspect `ctx.reason` for why. | @@ -651,16 +640,17 @@ The `chatRecovery` object accepts the following configuration options: | `work` | `number` | Coarse, monotonic count of content/tool segments produced since the incident opened (not tokens). | | `ageMs` | `number` | Wall-clock ms since the incident's first interruption. | -A progressing turn is never terminated by the framework on its own — it survives unbounded interruption (for example a dense deploy window) as long as it keeps making forward progress. Recovery is sealed only by one of these `ctx.reason` values: +A progressing turn survives repeated interruptions as long as it stays within the `maxRecoveryWork` limit. Recovery is sealed by one of these `ctx.reason` values: - `no_progress_timeout` — no forward progress within the no-progress window (a stuck turn). - `max_attempts_exceeded` — the attempt cap was spent on a tight no-progress alarm loop. - `work_budget_exceeded` — the turn kept producing content but exceeded `maxRecoveryWork` (a runaway loop). - `recovery_aborted` — your `shouldKeepRecovering` hook returned `false`. +- `out_of_memory` — recovery exceeded the memory-limit retry budget. - `stable_timeout` — recovery attempts kept timing out waiting for stable state until the budget drained (extreme churn). :::tip -A finite `maxRecoveryWork` can seal a legitimately long turn. Set a cap well above what a healthy turn produces, or use `shouldKeepRecovering` with real token or cost accounting for a precise budget. +The `maxRecoveryWork` default prevents a progressing turn from running forever. Increase it for long agentic turns. Use `shouldKeepRecovering` with durable token or cost data for a precise budget. ::: #### Turns waiting on a human are not sealed @@ -700,8 +690,6 @@ import type { } from "@cloudflare/ai-chat"; export class ChatAgent extends AIChatAgent { - override chatRecovery = true; - override async onChatRecovery( ctx: ChatRecoveryContext, ): Promise { @@ -763,6 +751,15 @@ override async onChatRecovery( } ``` +#### Control automatic continuation + +Durable bookkeeping remains active when automatic continuation is not appropriate. + +- Return `{ continue: false }` when another model call is unsafe. +- Persist cancellation intent and read it in `onChatRecovery()`. +- Record idempotency keys before external side effects. +- Use recovery budgets with durable spend data to limit cost. + #### `continueLastTurn` Appends to the last assistant message by re-calling `onChatMessage` with the saved request body. The response is streamed as a continuation — appended to the existing assistant message, not a new one. No synthetic user message is created. @@ -784,8 +781,6 @@ Use `this.stash()` inside `onChatMessage` to persist provider-specific data for ```ts export class ChatAgent extends AIChatAgent { - override chatRecovery = true; - async onChatMessage(_onFinish, options) { const result = streamText({ model: openai("gpt-5.4"), diff --git a/src/content/docs/agents/concepts/agentic-patterns/long-running-agents.mdx b/src/content/docs/agents/concepts/agentic-patterns/long-running-agents.mdx index 98cb4f0ca36..d52bb8f45c8 100644 --- a/src/content/docs/agents/concepts/agentic-patterns/long-running-agents.mdx +++ b/src/content/docs/agents/concepts/agentic-patterns/long-running-agents.mdx @@ -540,7 +540,7 @@ For the full `subAgent()` API — typed RPC stubs, client routing, access contro The patterns above handle the project manager's coordination work — scheduling, delegating, polling. But the project manager also uses an LLM directly: generating plans, summarizing progress, drafting status emails. Those LLM calls stream tokens over a connection that cannot be resumed if the agent is evicted mid-response. -For chat-oriented agents built on `AIChatAgent`, this is an even sharper problem — the user is watching the response stream in real time and sees it stop mid-sentence. `chatRecovery` wraps each chat turn in a `runFiber`, providing automatic `keepAlive` during streaming and a recovery hook when the agent restarts: +For chat-oriented agents built on `AIChatAgent` or `Think`, this is an even sharper problem — the user watches the response stream in real time and sees it stop mid-sentence. Durable recovery wraps every chat turn in a `runFiber`. This provides automatic `keepAlive` during streaming and a recovery hook when the agent restarts: ```ts import { AIChatAgent } from "@cloudflare/ai-chat"; @@ -550,8 +550,6 @@ import type { } from "@cloudflare/ai-chat"; class ProjectChat extends AIChatAgent { - override chatRecovery = true; - override async onChatRecovery( ctx: ChatRecoveryContext, ): Promise { @@ -575,7 +573,7 @@ The right recovery strategy depends on the LLM provider: Use `ctx.createdAt` to suppress stale recoveries. For example, if a recovered chat turn is older than a few minutes, you may persist the partial answer but skip automatic continuation to avoid surprising the user with an old response. -[`Think`](/agents/harnesses/think/) enables `chatRecovery` by default. The default path persists partial output and auto-continues or retries the turn when safe, so many apps do not need a custom hook. Override `onChatRecovery` when a provider has a better recovery strategy, or configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal user experience. +`AIChatAgent` and [`Think`](/agents/harnesses/think/) always use durable recovery. The default path persists partial output and continues or retries the turn when safe. Override `onChatRecovery` when a provider has a better recovery strategy. Configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal experience. If the agent is interrupted before any assistant stream chunks are written, there is no partial assistant message to continue. When the latest persisted message is still the unanswered user message from that turn, chat recovery retries the turn automatically unless `onChatRecovery` returns `{ continue: false }`. diff --git a/src/content/docs/agents/harnesses/think/client-tools.mdx b/src/content/docs/agents/harnesses/think/client-tools.mdx index 405d84e1a46..4c3433368a1 100644 --- a/src/content/docs/agents/harnesses/think/client-tools.mdx +++ b/src/content/docs/agents/harnesses/think/client-tools.mdx @@ -116,7 +116,7 @@ When a turn produces several client tool calls at once, Think waits for **all** ## Survive restarts while waiting for a human -A Durable Object can be evicted at any time, including while a turn is paused on an approval prompt or a client-side tool call. Because `Think` enables [`chatRecovery`](/agents/harnesses/think/recovery/) by default, the SDK treats such a turn as waiting on the human, not stuck. It parks the turn instead of failing it, and the user's eventual approval or tool result resumes the conversation. +A Durable Object can be evicted at any time, including while a turn is paused on an approval prompt or a client-side tool call. [`Think` durable recovery](/agents/harnesses/think/recovery/) is always on. The SDK treats such a turn as waiting on the human, not stuck. It parks the turn instead of failing it. The user's eventual approval or tool result resumes the conversation. For which interactions are exempt from recovery budgets, refer to [Turns waiting on a human are not sealed](/agents/communication-channels/chat/chat-agents/#turns-waiting-on-a-human-are-not-sealed). diff --git a/src/content/docs/agents/harnesses/think/configuration.mdx b/src/content/docs/agents/harnesses/think/configuration.mdx index 633fbce15af..70ffc0fe9f6 100644 --- a/src/content/docs/agents/harnesses/think/configuration.mdx +++ b/src/content/docs/agents/harnesses/think/configuration.mdx @@ -31,8 +31,8 @@ Think is configured by overriding methods and properties on your `Think` subclas | `messageConcurrency` | `"queue"` | How overlapping submits behave — refer to [Client tools](/agents/harnesses/think/client-tools/#message-concurrency) | | `includeMcpTools` | `true` | Convert connected MCP tools to AI SDK tools and add them to model turns. Refer to [MCP tools](/agents/harnesses/think/tools/#mcp-tools) | | `waitForMcpConnections` | `false` | Wait for MCP servers before inference | -| `chatRecovery` | `true` | Wrap WebSocket, sub-agent, programmatic, and continuation turns in `runFiber` for durable execution. Set to a configuration object with `maxAttempts`, `stableTimeoutMs`, `terminalMessage`, and `onExhausted` to tune bounded recovery | -| `chatStreamStallTimeoutMs` | `0` (off) | Opt-in inactivity watchdog: abort a turn whose model stream produces no chunk for this long (measures the gap between chunks, including tool execution). With `chatRecovery` on, a stall routes into bounded recovery | +| `chatRecovery` | Always on | Durable recovery configuration. Refer to [Durable recovery](/agents/harnesses/think/recovery/) for all options and defaults | +| `chatStreamStallTimeoutMs` | `0` (off) | Opt-in inactivity watchdog: abort a turn whose model stream produces no chunk for this long (measures the gap between chunks, including tool execution). A stall routes into bounded recovery | | `contextOverflow` | `undefined` | Opt-in mid-turn context-overflow handling with `reactive`, `maxRetries`, and `proactive` options. Requires `classifyChatError` plus a session compaction function — refer to [Context-window overflow recovery](/agents/harnesses/think/recovery/#context-window-overflow-recovery) | For `chatRecovery` and `chatStreamStallTimeoutMs` behavior, refer to [Durable recovery](/agents/harnesses/think/recovery/). diff --git a/src/content/docs/agents/harnesses/think/index.mdx b/src/content/docs/agents/harnesses/think/index.mdx index c31799827fa..429d443b4bf 100644 --- a/src/content/docs/agents/harnesses/think/index.mdx +++ b/src/content/docs/agents/harnesses/think/index.mdx @@ -247,7 +247,7 @@ Key behaviors: - **Blocking modes cannot nest.** Calling `wait`/`stream`/`continuation` (or the equivalent shortcut) from _inside_ an active turn — for example, from a tool's `execute` — throws, because it would deadlock the turn queue. From inside a turn, use `runTurn({ mode: "submit" })` (durable, runs after the current turn frees the queue) or [`addMessages()`](#add-messages-without-a-turn) (transcript only, no inference). - **`submit` is idempotent.** Pass `submissionId` and/or `idempotencyKey`; re-submitting a known key returns the existing record with `accepted: false` instead of starting a second turn. See [Programmatic submissions](/agents/harnesses/think/programmatic-submissions/). -- **Recovery-safe.** When `chatRecovery` is enabled, the `wait`, `stream`, and drained `submit` paths all run inference inside a recovery fiber, so an interrupted turn resumes after eviction. +- **Recovery-safe.** The `wait`, `stream`, and drained `submit` paths run inference inside a recovery fiber, so an interrupted turn resumes after eviction. `runTurn` is exported alongside its option and result types: `RunTurnOptions`, `RunTurnWait`, `RunTurnSubmit`, `RunTurnStream`, `TurnInputMessages`, and `TurnResult`. diff --git a/src/content/docs/agents/harnesses/think/recovery.mdx b/src/content/docs/agents/harnesses/think/recovery.mdx index 5fe26f0ece8..b823581088b 100644 --- a/src/content/docs/agents/harnesses/think/recovery.mdx +++ b/src/content/docs/agents/harnesses/think/recovery.mdx @@ -10,17 +10,17 @@ products: import { TypeScriptExample } from "~/components"; -Think wraps chat turns in recoverable [fibers](/agents/runtime/execution/durable-execution/) by default (`chatRecovery = true`). If the Durable Object is evicted mid-stream, Think reconstructs any buffered chunks, persists partial output, and schedules either a continuation of the assistant turn or a retry of the unanswered user turn. +Think always wraps chat turns in recoverable [fibers](/agents/runtime/execution/durable-execution/). If the Durable Object is evicted mid-stream, Think reconstructs any buffered chunks. It persists partial output and schedules a continuation or retry. :::note -This is on by default and works without configuration — most apps never touch this page. Read it when you want provider-specific recovery, a stall watchdog, or to tune the terminal experience after recovery gives up. +Durable recovery works without configuration. Most apps never need to configure it. Use this page for provider-specific recovery, a stall watchdog, or terminal behavior. ::: -When `chatRecovery` is `true`, WebSocket turns, sub-agent `chat()` turns, durable `submitMessages()` executions, auto-continuations, `saveMessages()`, and `continueLastTurn()` are wrapped in `runFiber`. +WebSocket turns, sub-agent `chat()` turns, durable `submitMessages()` executions, automatic continuations, `saveMessages()`, and `continueLastTurn()` are wrapped in `runFiber`. ## Bounded recovery -A stream-stall watchdog abort (`chatStreamStallTimeoutMs`) is treated as just another interruption: when `chatRecovery` is on, a stall routes into this same bounded path — the settled partial is preserved and a continuation is scheduled — so a transient hang recovers automatically. A persistently hanging provider exhausts the budget and terminalizes through the **same** exhaustion handling as a deploy or eviction interruption: `onExhausted` fires, the `chat:recovery:exhausted` event is emitted, and the configured `terminalMessage` is shown (not a raw stall error). +A stream-stall watchdog abort (`chatStreamStallTimeoutMs`) uses the same bounded recovery path. The SDK preserves the settled partial and schedules a continuation. A transient hang recovers automatically. A persistently hanging provider exhausts the budget through the same path as a deploy or eviction. The SDK calls `onExhausted`, emits `chat:recovery:exhausted`, and shows the configured `terminalMessage`. Configure bounded recovery by setting `chatRecovery` to an object: @@ -116,9 +116,11 @@ onChatRecovery(ctx: ChatRecoveryContext): ChatRecoveryOptions { Use `ctx.createdAt` to skip stale recoveries. For example, if the interrupted turn is older than a few minutes, return `{ continue: false }` so the partial response is preserved without starting an old continuation. +Durable bookkeeping remains active when automatic continuation is not appropriate. Return `{ continue: false }` to prevent another model call. For cancellation, side-effect, and cost controls, refer to [Control automatic continuation](/agents/communication-channels/chat/chat-agents/#control-automatic-continuation). + ### Recovery budgets and limits -Instead of `chatRecovery = true`, assign an object to tune how long recovery is allowed to run and when it is given up on. A turn that keeps making forward progress is never terminated by the framework on its own — duration is not a bound. Recovery is only sealed by one of the limits in the following table. +Assign a `chatRecovery` object to tune recovery limits and terminal behavior. A progressing turn survives repeated interruptions while it stays within the `maxRecoveryWork` limit. The following options control when recovery stops: @@ -127,7 +129,7 @@ export class MyAgent extends Think { override chatRecovery = { maxAttempts: 10, noProgressTimeoutMs: 5 * 60 * 1000, - maxRecoveryWork: Infinity, + maxRecoveryWork: 1_000, terminalMessage: "The assistant was interrupted and could not recover.", // Consulted from the second recovery attempt onward. Return false to stop. // Called as `config.shouldKeepRecovering(ctx)`, so it is NOT bound to the @@ -150,12 +152,13 @@ export class MyAgent extends Think { | `maxAttempts` | `10` | Attempt cap. Resets on forward progress, so it catches a tight no-progress alarm loop, not a healthy long turn. | | `stableTimeoutMs` | `10_000` | How long an attempt waits for the isolate to reach stable state before rescheduling. | | `noProgressTimeoutMs` | `300_000` (5 min) | Primary stuck-turn bound: max time without forward progress before sealing. **Resets on every progress-bearing attempt.** | -| `maxRecoveryWork` | `Infinity` | Runaway-loop guard: max produced content/tool units since the incident opened before a still-progressing turn is sealed. No cap by default. | +| `maxRecoveryWork` | `1,000` | Runaway-loop guard: maximum produced content/tool units before a still-progressing turn is sealed. Set a higher value or `Infinity` for a long agentic turn. | +| `maxOomRetries` | `3` | Retry budget for Durable Object memory-limit resets. Set `0` to stop after the first memory-limit reset. | | `shouldKeepRecovering` | — | Caller policy consulted from the second attempt onward. Return `false` to stop recovery. The hook point for a token/cost budget (`ctx.work` is a coarse segment count, not tokens). | | `terminalMessage` | generic message | Message shown to the user when recovery is given up on. | | `onExhausted` | — | Called once when recovery is given up on. Inspect `ctx.reason`. | -`ctx.reason` on the exhausted hook is one of: `no_progress_timeout` (stuck), `max_attempts_exceeded` (no-progress alarm loop), `work_budget_exceeded` (runaway), `recovery_aborted` (your `shouldKeepRecovering` returned `false`), or `stable_timeout` (extreme churn). Refer to [Stream recovery](/agents/communication-channels/chat/chat-agents/#stream-recovery) for the full shared reference — Think and `@cloudflare/ai-chat` use the same recovery configuration. +`ctx.reason` on the exhausted hook is one of: `no_progress_timeout` (stuck), `max_attempts_exceeded` (no-progress alarm loop), `work_budget_exceeded` (runaway), `recovery_aborted` (your `shouldKeepRecovering` returned `false`), `out_of_memory` (memory-limit retry budget), or `stable_timeout` (extreme churn). Refer to [Stream recovery](/agents/communication-channels/chat/chat-agents/#stream-recovery) for the full shared reference. Think and `@cloudflare/ai-chat` use the same recovery configuration. ## Repairing interrupted tool calls diff --git a/src/content/docs/agents/runtime/execution/durable-execution.mdx b/src/content/docs/agents/runtime/execution/durable-execution.mdx index fe84f5974fa..ee2ca4213a0 100644 --- a/src/content/docs/agents/runtime/execution/durable-execution.mdx +++ b/src/content/docs/agents/runtime/execution/durable-execution.mdx @@ -401,7 +401,7 @@ Key points: ### Chat recovery -`AIChatAgent` builds on fibers for LLM streaming recovery. When `chatRecovery` is enabled, each chat turn is wrapped in a fiber automatically. The framework handles the internal recovery path and exposes `onChatRecovery` for provider-specific strategies. Refer to [Long-running agents: Recovering interrupted LLM streams](/agents/concepts/agentic-patterns/long-running-agents/#recovering-interrupted-llm-streams) for details. +`AIChatAgent` and `Think` build on fibers for LLM streaming recovery. Every chat turn is wrapped in a fiber automatically. The framework handles the internal recovery path and exposes `onChatRecovery` for provider-specific strategies. Refer to [Long-running agents: Recovering interrupted LLM streams](/agents/concepts/agentic-patterns/long-running-agents/#recovering-interrupted-llm-streams) for details. ## Concurrent fibers diff --git a/src/content/docs/agents/runtime/operations/observability/diagnostics-channels.mdx b/src/content/docs/agents/runtime/operations/observability/diagnostics-channels.mdx index 3a16bc44b8e..514d08fc8f2 100644 --- a/src/content/docs/agents/runtime/operations/observability/diagnostics-channels.mdx +++ b/src/content/docs/agents/runtime/operations/observability/diagnostics-channels.mdx @@ -203,7 +203,7 @@ These events track chat message lifecycle, client-side tool interactions, and Th | `chat:recovery:skipped` | `{ incidentId, requestId, attempt, maxAttempts, recoveryKind, reason? }` | Recovery was skipped because the conversation changed or was no longer recoverable | | `chat:recovery:failed` | `{ incidentId, requestId, attempt, maxAttempts, recoveryKind, reason? }` | Recovery ran but failed | | `chat:recovery:exhausted` | `{ incidentId, requestId, attempt, maxAttempts, recoveryKind, reason }` | Recovery exceeded its configured attempt budget | -| `chat:stream:stalled` | `{ requestId, timeoutMs }` | The inactivity watchdog fired — no stream chunk arrived within `chatStreamStallTimeoutMs`. With `chatRecovery` on, the turn routes into recovery | +| `chat:stream:stalled` | `{ requestId, timeoutMs }` | The inactivity watchdog fired because no stream chunk arrived within `chatStreamStallTimeoutMs`. The turn routes into durable recovery | `recoveryKind` is `"retry"` when recovery replays an unanswered user turn and `"continue"` when it continues a partial assistant turn.