buzz-agent: move proactive context handoff to end of turn - #6584
Open
tlongwell-block wants to merge 2 commits into
Open
buzz-agent: move proactive context handoff to end of turn#6584tlongwell-block wants to merge 2 commits into
tlongwell-block wants to merge 2 commits into
Conversation
added 2 commits
August 22, 2026 18:10
The proactive handoff gate ran at the start of every tool-loop round. That is the worst moment to compact: the turn's working set (file reads, tool results) is exactly what the model is using, and summarizing it away mid-task forces a re-read storm that refills the window. The round-start gate also projected history growth at a conservative bytes-per-token rate, so one large tool result could trip it on its own. Run the gate once, after `run()` returns, so the NEXT turn starts on a fresh context and the current turn keeps what it was working with. - handoff.rs: `maybe_handoff(&mut attempts)` → `end_of_turn_handoff()`. Gates on `max_handoffs == 0` (disabled) and `should_handoff()`, which reads the final successful request's measured usage. `handoff()` takes a `Reseat` flag: end of turn does not re-append the already-answered prompt; the reactive context-400 path keeps `Reseat::LivePrompt` because it continues the same turn. - agent.rs: drop the per-turn `handoff_attempts` counter and the round- start gate; `truncate_history` is now unconditional after `drain_steers()`. Mid-turn overflow is handled only by the reactive context-400 ladder. - lib.rs: call `end_of_turn_handoff()` after `ctx.run()` when the result is `Ok` and the stop reason is not `Cancelled` — `run()` reports cancellation as `Ok(StopReason::Cancelled)`, so an `is_ok()` gate would spend a summarize round trip the user just asked us to stop. - config.rs / README.md: `BUZZ_AGENT_MAX_HANDOFFS` is now a switch — 0 disables proactive compaction, any positive value enables it; the gate runs at most once per turn by construction so the magnitude no longer bounds anything. Reactive recovery bypasses it as before. - The 90% threshold is unchanged; timing and threshold are not tuned in the same change. Tests (tests/regressions.rs) rewritten for the new boundary: `token_usage_over_budget_triggers_handoff` asserts the summarize lands inside the over-threshold turn and that the next request is [handoff block, new prompt] with no re-seated answered prompt; `history_growth_mid_turn_does_not_trigger_handoff` proves a 6 KB tool result mid-turn fires nothing and that the gate reads the final request's usage, not the turn's cumulative input; `cancelled_turn_skips_end_of_turn_handoff` cancels during an in-flight provider call; `handoff_fires_on_each_over_threshold_turn`, `max_handoffs_zero_disables_end_of_turn_handoff`, and `failed_end_of_turn_summarize_leaves_history_intact` replace the per-turn-cap tests. Existing handoff fixtures shift one request because the summarize now lands at the end of the seeding turn. Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz> Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Review finding (Wren): `should_handoff()` still routed its measured-usage
arm through `projected_handoff_input_tokens()`, which added a 1-byte/token
estimate of history appended since the usage was recorded. At end of turn
the only such history is the final assistant reply, so an under-threshold
turn could compact merely because its answer was long (8,500 measured +
a 500-byte reply crossed a 9,000 gate). That contradicted the documented
contract ("final request used 90%") and kept the estimator this change
set out to retire.
- handoff.rs: the `Some(measured_tokens)` arm compares the provider's
count directly against `token_threshold`. `projected_handoff_input_tokens`
is deleted; the handoff log line's "before" count uses the measurement
when present, else the byte-derived bound. The `None` byte fallback is
unchanged. Doc comments no longer claim "nothing is appended after" the
final request.
- agent.rs / lib.rs: remove `last_request_history_bytes` (RunCtx field,
Session field, acquire/run_prompt plumbing, the two clears) — it existed
only to feed the estimator.
- tests/regressions.rs: add `large_final_reply_does_not_trigger_end_of_turn_handoff`
(usage 8500 < 9000, 2 KB reply, exactly one request). Fails on the
parent commit; passes here. Reword
`reactive_reset_clears_usage_baseline_so_the_gate_is_not_blind` — the
scenario still holds (a stale sub-threshold reading blinds the gate),
the "paired bytes / grown" mechanics it described do not.
Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Move
buzz-agent's proactive context handoff from the start of every tool-loop round to once, at the end of the turn.Problem. The round-start gate compacted while the turn's working set (file reads, tool results) was exactly what the model was still using. Summarizing it away mid-task forced a re-read storm that refilled the window. The gate also projected history growth at 1 byte/token, so a single large tool result could trip it on its own.
Change.
handoff.rs:maybe_handoff(&mut attempts)→end_of_turn_handoff(). Gates onmax_handoffs == 0(disabled) andshould_handoff(), which now compares the provider's measured input usage for the turn's final request directly against the 90% threshold. The growth estimator (projected_handoff_input_tokens) and its byte baseline (last_request_history_bytes) are deleted end-to-end. The pre-usage byte fallback is unchanged.handoff()takesenum Reseat { LivePrompt, None }: end of turn does not re-append the already-answered prompt; the reactive context-400 path keepsLivePromptbecause it continues the same turn.agent.rs: per-turnhandoff_attemptscounter and round-start gate removed;truncate_historyis unconditional afterdrain_steers(). Mid-turn overflow is handled only by the reactive context-400 ladder (unchanged: in-loop retry, round refund, independent 3-rung bound).lib.rs:end_of_turn_handoff()runs afterctx.run()when the result isOkand the stop reason is notCancelled—run()reports cancellation asOk(StopReason::Cancelled), so anis_ok()gate would spend a summarize the user just asked us to stop.BUZZ_AGENT_MAX_HANDOFFSis now a switch:0disables proactive compaction, any positive value enables it (≤1 per turn by construction). Reactive recovery bypasses it as before.config.rs/README.mdupdated.Deliberately not changed: the 90% threshold. Timing and threshold are not tuned in the same PR; measure first.
Related issue
None found. Supersedes the relay-hosted draft (
buzz://PR #1 by Pinky/Brain) that proposed the same move.Testing
tests/regressions.rsrewritten for the new boundary (54 regressions, all green):token_usage_over_budget_triggers_handoff— summarize lands inside the over-threshold turn; next request's user items are exactly[Context Handoff], new prompt (no re-seated answered prompt).history_growth_mid_turn_does_not_trigger_handoff— 6 KB tool result mid-turn fires nothing; gate reads the final request's usage (8500+8900 cumulative > 9000 gate, final 8900 < 9000 → no handoff).large_final_reply_does_not_trigger_end_of_turn_handoff— usage 8500 < 9000 with a 2 KB reply → exactly one request. Fails on the first commit, passes on the second (this was the review blocker: the old estimator charged the reply at 1 byte/token).cancelled_turn_skips_end_of_turn_handoff— cancel during an in-flight provider call →stopReason: cancelled, no summarize.handoff_fires_on_each_over_threshold_turn,max_handoffs_zero_disables_end_of_turn_handoff,failed_end_of_turn_summarize_leaves_history_intactreplace the per-turn-cap tests.Mutants caught:
is_ok()gate;Reseat::LivePromptat end of turn; mid-turn gate restored; reply-bytes growth term re-added.Verification at
7f3b63ea:cargo test -p buzz-agentfull package green (456 unit / 54 regressions); fmt + clippy-D warningsclean; pre-push rust-tests + desktop-tauri-checks green.Independent review: Wren — Minimalness 9/10, Elegance 9/10, Correctness 9/10 at
7f3b63ea, no findings (full suite re-run in a clean worktree).Live e2e (Max): built from a clean worktree at
7f3b63ea, real OpenAIgpt-5.5through a recording proxy. 4,000-token window: request 1 reported 4,701 input tokens, 14 KB tool result, turn completed with no mid-turn handoff; summarize +handoff #1only after the final response; next turn's provider history was exactly[Context Handoff]+ new prompt. Cancel mid-provider-call →cancelled, no handoff. 8,000-token window: 1,154 output tokens appended on a turn ending at measured input 6,413 → no handoff. Receipt:.scratch/eot-live-7f3b63ea/on the dev Mac.