-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
fix(chat): keep steering, action and injected messages in the conversation #4816
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
+3,529
−136
Open
Changes from all commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
c6c35d5
test: cover the head-start accumulator seed without hydrateMessages
ericallam 5fef13f
fix(chat): put injected steering messages into the accumulator
ericallam 4b0f52a
fix(chat): persist history an action rolled back
ericallam e399b60
fix(chat): make a response streamed from onAction part of the convers…
ericallam 52bd98d
fix(chat): route a system-role injection to the instructions lane
ericallam 47f2f8f
fix(chat): address review on the accumulator, instructions and action…
ericallam 52772b5
fix(chat): report a failed action stream instead of committing it as …
ericallam c6b0dbd
docs(chat): note the instructions delivery path and one-shot injectio…
ericallam e3318cf
docs(ai-chat): say what an action persists under each persistence model
ericallam 02e0a70
docs(ai-chat): delete the replaced answer in the regenerate example
ericallam e6618cf
docs(ai-chat): style-guide pass on the injection and action sections
ericallam 1f7fb5e
docs(ai-chat): drop em dashes from the docs and changesets
ericallam 3246d12
docs(ai-chat): drop the remaining em dashes from the actions and inje…
ericallam 86d67fa
docs(chat): warn about the two upgrade hazards in the changesets
ericallam e2737f6
fix(chat): consume injected instructions per turn, not per options build
ericallam 692c060
fix(chat): keep an injection made during the turn that consumed the lane
ericallam 900418d
fix(chat): rebuild the model messages after a steering injection
ericallam 83fe5ef
test(chat): cover the model-lane rebuild on a turn that captures no r…
ericallam 6683e4b
fix(chat): keep a steer in the lanes on the createSession surface
ericallam bf82221
fix(chat): append a steer to the model lane instead of rebuilding it
ericallam dd06745
fix(chat): report a drained steer from a turn that fails
ericallam a0a07bb
test(chat): pin a one-shot instruction across an action
ericallam c9fe897
test(chat): use the spread form in the action-instruction test
ericallam 4f535ac
fix(chat): keep a steer's prepared form for later turns
ericallam 39ea541
fix(chat): reconcile a steer into the model lane when the turn fails
ericallam 920c76d
fix(chat): keep a failed action from counting as a turn
ericallam e348d61
fix(chat): move the snapshot cursor on a failed turn
ericallam 0ef1caa
docs(chat): cover the second review round in the changesets
ericallam ae72c70
fix(chat): report a steer in the turn delta, and once after a history…
ericallam 62fd77c
fix(chat): keep a prepared steer through a history edit and a failed …
ericallam 6482650
fix(chat): deliver plain onAction replies, and keep compaction throug…
ericallam 472eaf4
fix(chat): keep an instruction injected after an action for the next …
ericallam 023432d
feat(chat): send actions through useChat so their turns render
ericallam d332a2a
feat(chat): let an action become a turn with chat.turn()
ericallam File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions can now become turns. `onAction` edits history with `chat.history`; to answer after the edit, return `chat.turn()` and a turn runs on the edited history with everything a turn has: the agent's system prompt and tools, steering, compaction, injected instructions, `onTurnStart` and `onTurnComplete`, and persistence. A regenerate is `chat.history.slice(0, -1); return chat.turn();`. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action }) => { | ||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| return chat.turn(); | ||
| } | ||
| if (action.type === "undo") chat.history.slice(0, -2); // edit only | ||
| }, | ||
| ``` | ||
|
|
||
| Returning a `StreamTextResult`, `string` or `UIMessage` from `onAction` is no longer supported and now fails with an error pointing to `chat.turn()`. A response produced that way skipped every turn guarantee, and its delivery to the browser was unreliable: the frontend never read the stream `transport.sendAction` returned, so a regenerate that appeared to work on the server did not render. The `onAction` event no longer carries `streamText` or `tools`, since the handler no longer calls the model. | ||
|
|
||
| History edits made by an action are still persisted as before: platform-managed snapshots are written after the edit, and apps with their own store mirror the edit themselves. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages are now kept in the conversation when you drive turns yourself with `chat.createSession()` or `chat.MessageAccumulator`. Previously a message that arrived mid-answer shaped that answer and then existed nowhere: it was missing from `turn.uiMessages`, so an app persisting from there never stored it, missing from `turn.messages`, so every later turn answered as though it had never been sent, and it was not queued as its own turn either. It now lands in both, the same way it does on `chat.agent`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Injected system context is merged into a single instruction block, so it works on every supported AI SDK version. Note that a cached system prompt gives up its cache entry for as long as an injection is live, since the cached prefix has changed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| `chat.inject()` with `role: "system"` now works. It previously put the system message into the conversation, which AI SDK 7 rejects for every provider: the next turn died with a generic "An error occurred." and persisted an empty assistant message, so the agent looked like it had stopped answering. System-role context is now appended to the model's instructions, which is also the only way to inject context the agent treats as trusted. | ||
|
|
||
| Two things to know. Instructions are delivered by `chat.toStreamTextOptions()`, so a `run()` that calls `streamText` without spreading it does not receive a system-role injection. The conversational lane has no such requirement. And an injection applies to the next turn only, rather than repeating on every turn that follows it. Every inference call in that turn sees it, so a `run()` that builds options more than once gets the same instructions each time. An instruction injected after an action has run, and before the next message, reaches that next turn rather than the one after it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Undo, edit and regenerate now survive a run ending. History rolled back from `onAction` was only kept in the running worker's memory, so the rollback held while that worker stayed warm and then reverted on the next continuation. The undone messages came back, minutes later, with no error. This also holds when the turn before the action failed: the rollback used to be written against the cursor from before that turn, so a continuation could replay output the failed turn had already superseded. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages injected mid-answer are now part of the conversation, both for your hooks and for the model on later turns. Previously they reached the model for the answer they steered and reached the browser, but nothing else: `onTurnComplete` never saw them, so an app storing its own transcript lost the instruction the answer was shaped by, and it vanished from the conversation on reload. The model also forgot the instruction from the next turn onwards, answering as though the message had never been sent, while the chat UI still showed it. This holds when the steered turn fails part-way, and when `pendingMessages.prepare` reshapes the message: later turns now see the same form the steered turn did, not the original message. | ||
|
|
||
| Approving a tool call no longer undoes compaction. A tool-approval continuation used to rebuild the model's context from the full conversation, so a chat that had been summarised to fit the context window was sent the whole transcript again on the next call, and could go over the limit it had just been compacted to avoid. The same applied to a regenerated answer that replaced an existing one. | ||
|
|
||
| If you worked around this by saving steering messages as they arrive, in `pendingMessages.onReceived` for example, that write now duplicates the one you get from `newUIMessages`. Drop it, or skip messages you have already stored. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions are sent through `useChat` so a turn that follows one renders like any turn. `TriggerChatTransport` recognises `body.action` on a `useChat` request and sends it as an action, so `sendMessage(undefined, { body: { action } })` or `regenerate({ body: { action } })` sends the action and `useChat` owns the response: it streams into the message list, `status` and `error` behave as for a message, and `stop` works. `useChatActions({ sendMessage })` in `@trigger.dev/sdk/chat/react` is a two-line convenience over that. | ||
|
|
||
| ```tsx | ||
| const { sendMessage } = useChat({ id: chatId, transport }); | ||
| const { sendAction } = useChatActions({ sendMessage }); | ||
| sendAction({ type: "regenerate" }); | ||
| ``` | ||
|
|
||
| Previously the frontend docs said `useChat` consumed the stream `transport.sendAction` returns; it never did, so an action's answer was never rendered by an app following them. `transport.sendAction` is unchanged for callers outside `useChat` and still returns a stream the caller must read. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,7 @@ | ||
| --- | ||
| title: "Actions" | ||
| sidebarTitle: "Actions" | ||
| description: "Custom commands sent from the frontend that mutate chat state without consuming a turn — undo, rollback, edit, regenerate." | ||
| description: "Custom commands sent from the frontend that mutate chat state without consuming a turn: undo, rollback, edit, regenerate." | ||
| --- | ||
|
|
||
| ## Overview | ||
|
|
@@ -54,7 +54,7 @@ export const myChat = chat.agent({ | |
|
|
||
| ## Returning a model response from an action | ||
|
|
||
| `onAction` can return a `StreamTextResult`, `string`, or `UIMessage` to produce a response. The returned stream is auto-piped to the frontend just like a normal turn, but the rest of the turn machinery (`onTurnStart`, `onTurnComplete`, etc.) still does not fire. | ||
| `onAction` can return a `StreamTextResult`, `string`, or `UIMessage` to produce a response. All three are sent to the frontend and added to the conversation just like a normal turn's answer, but the rest of the turn machinery (`onTurnStart`, `onTurnComplete`, etc.) still does not fire. A returned `UIMessage` must have `role: "assistant"`; its text and `data-*` parts are delivered, and other part types are dropped. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action, messages }) => { | ||
|
|
@@ -70,7 +70,37 @@ onAction: async ({ action, messages }) => { | |
| } | ||
| ``` | ||
|
|
||
| This is useful for actions that both mutate state and want a fresh model response (regenerate-from-here, retry-with-different-style). Persistence is your responsibility inside `onAction` itself; you have access to the streamed response object. | ||
| This is useful for actions that both mutate state and want a fresh model response (regenerate-from-here, retry-with-different-style). | ||
|
Comment on lines
70
to
+73
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔍 Actions guide documents removed returns The guide still recommends returning streams, strings, and UI messages. The final API rejects them and requires (Refers to this code) Was this helpful? React with 👍 or 👎 to provide feedback. |
||
|
|
||
| ### Actions and persistence | ||
|
|
||
| An action is not a turn, so `onTurnComplete` never fires, and that is where an app that owns its own transcript normally writes. What that means depends on which persistence model you use. | ||
|
|
||
| **Platform-managed** (no `hydrateMessages`): nothing to do. After an action that changed the conversation (a `chat.history` mutation, a response returned from `onAction`, or both), the runtime writes the snapshot, so the change survives the run ending. | ||
|
|
||
| **Your own store** (`hydrateMessages` registered): the runtime deliberately does not write, because your store is the source of truth. A history mutation and a returned response both live only in the running worker until you persist them, and a continuation rehydrates from your store, not from what the worker had in memory. `chat.pipeAndCapture` hands you the same assistant message the runtime would have captured: | ||
|
|
||
| ```ts | ||
| onAction: async ({ action, messages }) => { | ||
| if (action.type === "undo") { | ||
| chat.history.slice(0, -2); | ||
| await db.deleteLastExchange(chatId); // the rollback is yours to persist | ||
| } | ||
|
|
||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| await db.deleteLastAssistant(chatId); // drop the answer being replaced | ||
| const { message } = await chat.pipeAndCapture( | ||
| streamText({ model: anthropic("claude-sonnet-4-5"), messages }) | ||
| ); | ||
| if (message) await db.saveMessage(message); // then store the new one | ||
| } | ||
| }, | ||
| ``` | ||
|
|
||
| Mirror each mutation in your store, not only the additions. A `chat.history` mutation is invisible to your database, so a regenerate is a delete *and* an insert. Saving the new answer without removing the old one leaves both in the canonical transcript, and the next hydration returns the two of them. (An append-only or branching store is the exception: there you write a new version and resolve the head on read.) | ||
|
|
||
| Returning the stream instead of piping it yourself still works and still reaches the browser, but you have no message to store, so the next run does not know about it. | ||
|
|
||
| ## Gating actions on HITL state | ||
|
|
||
|
|
@@ -89,10 +119,10 @@ onAction: async ({ action, messages, signal }) => { | |
| ## Sending actions from the frontend | ||
|
|
||
| ```ts | ||
| // Browser — TriggerChatTransport | ||
| // Browser: TriggerChatTransport | ||
| const stream = await transport.sendAction(chatId, { type: "undo" }); | ||
|
|
||
| // Server — AgentChat | ||
| // Server: AgentChat | ||
| const stream = await agentChat.sendAction({ type: "rollback", targetMessageId: "msg-3" }); | ||
| ``` | ||
|
|
||
|
|
@@ -104,8 +134,8 @@ The action payload is validated against `actionSchema` on the backend; invalid a | |
|
|
||
| ## See also | ||
|
|
||
| - [`chat.history`](/ai-chat/backend#chat-history) — the imperative API actions use to mutate state | ||
| - [Sending actions from the frontend](/ai-chat/frontend#sending-actions) — `transport.sendAction` ergonomics | ||
| - [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages) — fires before `onAction` when set | ||
| - [Branching conversations](/ai-chat/patterns/branching-conversations) — pairs action handlers with backend-controlled history | ||
| - [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop) — gating fresh actions while a tool is waiting | ||
| - [`chat.history`](/ai-chat/backend#chat-history): the imperative API actions use to mutate state | ||
| - [Sending actions from the frontend](/ai-chat/frontend#sending-actions): `transport.sendAction` ergonomics | ||
| - [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages): fires before `onAction` when set | ||
| - [Branching conversations](/ai-chat/patterns/branching-conversations): pairs action handlers with backend-controlled history | ||
| - [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop): gating fresh actions while a tool is waiting | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔍 Release notes contradict action migration
Several changesets promise captured direct action replies. The final changeset removes those replies, so one release publishes conflicting guidance.
Was this helpful? React with 👍 or 👎 to provide feedback.