Repository navigation
Heartbeats only inside ctx.working; a shutdown is the default - #322
Conversation
An attempt heartbeated on its own while nothing of Restate's was outstanding, or while an annotation was open. That kept a handler stuck on a promise of its own alive forever, and an annotation around a tool that waits for a signal held its process for the whole wait. - `ctx.working()`: an async context manager. While any scope is open the attempt heartbeats, whatever it waits for; outside, Restate shuts it down at its inactivity timeout and it suspends on its calls (or starts again at once, waiting for none). - The driver counts the scopes. A scope's first heartbeat comes at once only if the attempt wrote nothing for an interval, so the database's scope per call costs no message. - The database's own calls (open, statements, commits) run in a scope. - Annotations are traces only: they keep nothing alive. - `tool(keepalive=True)` runs a tool of your own in `ctx.working()`; by default a tool is not kept alive. - Tests: the driver's and the server's heartbeat tests follow the new rule; the scripted frameworks of the long-work and dead-attempt tests run their model in `ctx.working()`; the test `Host` counts open scopes.
…tools run kept alive - OpenAI: each model request is kept alive from the end of `on_llm_start` to the top of `on_llm_end`, so the steering and commits around it are not. The SDK's own tools (shell, local shell, apply_patch, computer) are kept alive per call; a function tool is the user's, and is not. The summarizer's request too. Whatever a run leaves open is closed as it ends. - Pydantic AI: the model the run gets is wrapped, so exactly each request (streamed or not) is kept alive; a compaction's too. The harness's file system tools (by their kind) and its shell (by the names it gives its tools) are kept alive per call; `code_mode` is not, since it runs the user's tools. Deferred and approval waits stay outside. Known gap: the model requests of an OpenAI agent used as a tool of another (`Agent.as_tool`) are not kept alive: the SDK gives its nested run no hooks.
…s built-in local tools - ADK: the plugin keeps each model request alive from the end of its `before_model_callback` (after the steering) to the whole response, an error, or the event that ends the step if a callback answered instead. The shell, the skills' scripts and the environment's tools (known by their package, as ADK keeps their classes private) are kept alive per call. The summarizer's request too. A run closes what is left as it ends. Not covered: an agent's code executor, which runs as a processor of the flow. - LangGraph: the built-in agent's request and its summarizer's run in `ctx.working()`. A graph of your own gets a callback handler, inherited by its nodes' runs, that keeps each chat model request alive from its start (awaited before it is sent) to its end or error, and releases what is left before any wait for an interrupt. Known gap: the shell of LangChain's `ShellToolMiddleware` is a plain tool with nothing that tells it apart, so it is not kept alive.
There was a problem hiding this comment.
Caution
In a user LangGraph graph, a sync node that calls model.invoke now hangs forever before its request goes out. I reproduced it.
Approach: sound. Making keepalive opt-in with a scope counter in the driver fixes the cause rather than tuning the old heuristic. Wiring each adapter through its own framework hooks (no patching) is the right place for it. The one exception is LangChain's sync callback bridge, which the _KeepAlive handler doesn't account for.
Reviewed changes: I read the full diff of all 4 commits: the sessions driver and database, the four agent adapters, the docs, and the tests.
ctx.working()and the driver's scope count: heartbeats now run only while a scope is open. The first beat comes at once only if the attempt has been silent for a full interval.- Annotations are traces only:
open_annotation/close_annotationno longer affect heartbeats, and_store_waits/_awaits_restateare gone. - Database scopes:
sql_open, each store request andcommitrun in the host'sdatabase_working. tool(keepalive=True): the tool body runs inctx.working(). Plain tools are no longer kept alive.- OpenAI adapter: model requests are scoped from
on_llm_starttoon_llm_end, and the SDK's non-FunctionTooltools are scoped per call.ended()is now async and closes leftover scopes. - Pydantic AI adapter: a
_AliveWrapperModelwraps the run model and the compaction model. The harness's file-system kinds and shell names are kept alive. - ADK adapter: plugin hooks scope each request per (invocation, branch, agent), plus the built-in local tools.
release_allruns in the run'sfinally. - LangGraph adapter: the built-in agent and the summarizer are wrapped directly. User graphs get an inherited
_KeepAlivecallback handler. - Tests: each adapter checks open scopes through the test
Host. The driver and server tests follow the new rule.
claude-opus-5-5 | 𝕏
…rtbeat comes within an interval - LangChain runs the callbacks of a sync request (`invoke`, in a sync node) on a loop of its own, in a thread, and waits there for every task they start. The handler entered `ctx.working()` there, whose heartbeat task never ends: the request never went out. It now opens and closes its scopes on the attempt's loop, handed over without waiting; one that arrives after its stream ended is dropped. A test with a sync node, whose `working()` starts a task as the driver's does, hung before. - `ctx.working()` refuses to be entered on another loop than the attempt's. - A scope's first heartbeat waited a whole interval after any recent write, so the silence could reach almost two intervals; it now comes once the silence reaches one.
There was a problem hiding this comment.
ℹ️ Both earlier findings are fixed. One minor wording suggestion is inline.
Reviewed changes: Since e5f94ac, one commit (23cb6bb) fixes the sync LangGraph request hang and shortens the first heartbeat's delay.
_KeepAliveloop handoff: the handler captures the attempt's loop. It runs_open/_closeinline on that loop, and from any other loop it hands them over withrun_coroutine_threadsafewithout waiting. Because neither suspends before it touches the driver's scope, they stay in order. A_streamcounter drops an open that arrives afterrelease_all.- Driver guard:
working()raises if it is entered off the attempt's loop. Every in-repo caller (the adapters,tool(keepalive=True), the database boundary) is async on that loop. - First heartbeat: the first sleep is now only the time left in the interval, so the silence never exceeds one interval and the
turns.mdstatement holds. - Test: a sync node whose test
working()spawns a task the way the driver does. It passes locally along with the driver's heartbeat tests.
claude-opus-5-5 | 𝕏
There was a problem hiding this comment.
✅ No new issues found. Every earlier finding is addressed.
Reviewed changes: One commit since 23cb6bb.
- Off-loop error wording: the
RuntimeErrorthatctx.working()raises off the attempt's event loop now states the requirement ("must be entered on the attempt's own event loop").
claude-opus-5-5 | 𝕏

A session object's attempt heartbeated on its own while nothing of Restate's was outstanding, or while an annotation was open. That had three problems:
Heartbeats are now opt-in. Suspension is the default. This is the same change the TypeScript SDK made.
sessions
ctx.working(), an async context manager.SessionContextprotocol gainsworking(), so a test's own fake context needs it too (contextlib.nullcontext()will do).agents
tool(keepalive=True)runs a tool of your own inctx.working(). By default a tool is not kept alive, since it may wait on Restate.on_llm_startto the top ofon_llm_endWrapperModelaround the run's model, streamed or not; a compaction's tooinvokein a sync node)Known gaps
Agent.as_tool). The SDK gives its nested run no hooks.ShellToolMiddleware. It is a plain tool with nothing that tells it apart.Testing
ctx.working()keeps its attempt, and that work outside it is cut off and restarted.Host: model request, plain tool,keepalivetool, built-in local tool, approval or interrupt wait, after the turn, and failed or stopped runs;working()that starts a task as the driver's does;ctx.working().