Repository navigation
feat(compaction): Summarize with the agent model and cached request prefix - #2602
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
5 screenshot changes — 5 changed · 0 added · 0 removed Review screenshots in Frameshift
|
…refix Capacity compaction asks the active agent model for the summary. The request is the agent request for the same messages plus one user message with the summary instruction, so the provider reads the cached prompt prefix. The output must be a complete <summary> block; one nudge, then the full history stays. The fast model and the 6,000 character cut are gone for this path. Slack no longer compacts before the agent starts. The agent compacts before its first model request and keeps recent user messages, the summary, then the new instruction. Handoff compaction is unchanged.
sentry-junior
Bot
force-pushed
the
compaction-summary-main-model
branch
from
November 3, 2026 12:13
ed2dc6b to
e71a23c
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e71a23c. Configure here.
The Turn commits its context before its instruction, and the selection checked one message at a time. A context message then looked like a user ask, took the retained budget, and repeated after the summary.
This was referenced Nov 3, 2026
thomasjiangcy
approved these changes
Nov 3, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Capacity compaction now gets its summary from the active agent model. Before, it sent the whole history to
openai/gpt-6-lunawithcacheRetention: "none". Big compactions (~170k input tokens) took ~12s p50 and read nothing from the cache.Request
buildAgentModelRequest(newchat/agent/context-summary.ts) builds the request that the Pi agent loop would send for the same messages: the agent'stransformContext→convertToLlm,state.model,systemPrompt,tools,thinkingLevel,sessionId,thinkingBudgets, transport, and API key. The summary call adds one user message with the instruction. Pi does not export its request builder, so this code repeatsstreamAssistantResponseandcreateLoopConfig. A unit test runs a real PiAgentand checks that the built request matches what the loop passed tostreamFn. If Pi changes these steps, the test fails.tool_choice, thinking, and effort stay the same, because Anthropic drops cached messages when they change. The prompt forbids tool calls instead.max_tokenschanges (8k). For budget models, Pi adds the thinking budget on top, so the budget does not change. For adaptive models (Opus 5.5), thinking counts against the 8k.gen_ai.prompt.name: junior.context_compaction. Itsgen_ai.usage.*attributes include cache read and cache creation.Prompt
The prompt says: do not continue the task, do not call tools, and return only
<summary>…</summary>. It asks for fixed sections: Goal / Constraints & preferences / Progress (Done, In progress, Blocked) / Key decisions / Open asks (in the user's exact words) / References (≤20) / Next steps. Each item is a one-line bullet, and the whole summary stays under 1,000 words. It leaves out content that is added again anyway. In update mode, it merges an earlier summary, and newer history wins.Validation
These count as failures: a tool call, a
lengthstop, empty output, or a missing tag. The first failure gets one nudge (instruction + nudge, so the cached prefix still applies). A second failure throws, and the history stays untouched. As before, it throws only if the history is already at the input limit. This path has noslice(0, 6000)anymore, so Junior never stores a truncated summary.Turn start
Slack's pre-agent
maybeCompactis removed. It had no agent request to reuse. The agent already compacts before its first model request on every surface (web already worked this way). A new Turn's compaction keeps the old replacement shape: up to 20k tokens of recent user messages from committed history, with their original provenance and source events, then the summary, then the new Turn's context and instruction. The new instruction is left out of the selection if the Turn committed it before compaction. Mid-turn replacement is unchanged: the instruction, then the summary.Handoff compaction (fast model, old prompt) and thresholds are unchanged.
Telemetry
Rebased on current
main(#2599, #2600, #2601). The agent's compaction before the first model request (applyActiveContextCompaction→compactActiveContextIfNeededwithmode: "blocking") now carries #2599's instrumentation:agent.context_compactionspan per compaction, withapp.compaction.mode=blocking(before the first model request),active(between model calls), orhandoff. The summarygen_ai.chatspan (gen_ai.prompt.name: junior.context_compaction) is its child, so you can group cache reads by mode.app.turn.compaction_wait_msonchat.turnstill counts this wait, because the agent runs inside the Slackchat.turnspan.TELEMETRY.mdnow says thatblockingruns in the agent, andapp.compaction.retained_messagescounts all history messages kept with the summary.modereplaces the earlierretainUserMessagesflag. Ablockingcompaction with a new instruction keeps recent user messages. This is the same behavior as before.#2601's read-back eval and the new probe evals are both in
continuity.eval.ts.Tests / evals
tests/unit/agent/context-summary.test.tscovers request equality with the real Pi loop, the single appended message, and the nudge-then-fail path.check-test-architectureforbids new agent-running tests inpackages/junior/tests.evals/conversation/continuity.eval.tsask "what's still open", "which PR", and "what failed" after a long thread is compacted. Not run: the sandbox has no model keys. They need a run and recordings.via @thomasjiangcy.
--
View Session [Sentry]