Skip to content

[Feat] Agents screen their own diff for risky hunks before they ship - #3141

Merged
mrubens merged 4 commits into
developfrom
feat/diff-risk-hints
Sep 22, 2026
Merged

mrubens merged 4 commits into
developfrom
feat/diff-risk-hints

Conversation

@mrubens

@mrubens mrubens commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

What

A get_diff_risk_hints tool for coding tasks. During its self-review, before it pushes or opens a pull request, the agent calls it once. It runs the same decision-model pre-screen pull request reviews use (#3119, #3130) on the branch's diff and returns up to three changed hunks most likely to contain a defect, each with the kind of issue suspected. The agent re-reads those hunks and fixes what the code confirms.

The point is catching an issue one review round earlier: fixed before the push, instead of found by the reviewer, picked up by the agent, and fixed in another push.

Why this shape

It replaces the completion check removed in #3139 and avoids what made that fragile:

  • Advisory. No verdict, nothing held, blocked, or reopened. A weak hint costs a glance.
  • Diff only. No transcript, command history, or test-evidence reconstruction.
  • Reuses a tuned component. The pre-screen already has its own eval and fails open. This PR exports screenReviewHunks and wraps it with author-facing wording; the review path is unchanged.
  • The agent decides when. No harness hooks or plugins; a prompt instruction in the implement-changes self-review step.

How

  • Sandbox MCP tool computes the diff locally: branch against origin/HEAD merge-base, including uncommitted and untracked files (what the pull request will show).
  • POST /api/mcp/tasks/runs/:runId/diff_risk_hints (run token, run must match) screens it with the task title; the judgment model key stays on the API.
  • Registered for every task run except pull request reviews, whose reviewer already receives these hints. Returns available: false when there is no hosted judgment model.

Measuring it

Worth checking after a week on nightly: the share of agent pull requests whose first review finds issues, before and after.

Validation

  • Server wrapper (title and changed files passed through, empty-hint wording, unavailable path), API route (auth, run match, size caps), sandbox tool against real temporary git repos (committed, uncommitted, untracked; no call when nothing changed).
  • pnpm lint:fast, pnpm check-types:fast, pnpm knip; cloud-agents workflows and worker MCP suites.

@roomote-community

roomote-community Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • packages/cloud-agents/src/server/diff-risk-hints.ts:33 now fails open when the decision model errors.
  • packages/cloud-agents/src/server/diff-risk-hints.ts:44 no longer logs raw upstream errors that can include submitted diff content.

Reviewed ee6e123

Comment thread packages/cloud-agents/src/server/diff-risk-hints.ts Outdated
Comment thread packages/cloud-agents/src/server/diff-risk-hints.ts Outdated
@mrubens
mrubens marked this pull request as ready for review September 22, 2026 21:20
@mrubens
mrubens merged commit 0acb4e1 into develop Sep 22, 2026
18 checks passed
@mrubens
mrubens deleted the feat/diff-risk-hints branch September 22, 2026 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant