[Feat] Add decision-model pre-screen to pull-request reviews - #3119
Conversation
|
No new code issues found. See task
Reviewed 638e4f7 |
![roomote-community[bot]](https://avatars.githubusercontent.com/u/211522643?s=60&v=4){kind=small}
| timeoutMs: REVIEW_PRESCREEN_TIMEOUT_MS, | ||
| highVolume: true, | ||
| // Review diffs are sensitive; do not add them to optional training capture. | ||
| capture: false, |
There was a problem hiding this comment.
capture: false only suppresses captureJudgment; it does not suppress shadowRoomoteJudgment. With R_JUDGMENT_SHADOW=on and a TypeSafe, OpenRouter, or Vercel backend, this new call still sends the capped raw PR diff to the configured Roomote upstream as a second request. That bypasses the sensitive-diff safeguard described here. Gate shadowing on this opt-out (or add an explicit no-shadow option) and cover that combination.
There was a problem hiding this comment.
Final behavior follows the latest product direction in 638e4f73: the request-level capture option is removed, and the pre-screen passes no capture or shadow override, so deployment opt-in capture and shadow-forwarding remain active. Regression coverage verifies capture plus the second shadow request when R_JUDGMENT_SHADOW=on; 102 focused tests and all repository gates pass. The request to suppress shadowing is intentionally dismissed because it conflicts with that current requirement. See task
There was a problem hiding this comment.
The final commit removes the shadow: false passed here, reintroducing this issue. When R_JUDGMENT_SHADOW=on and the selected backend is TypeSafe, OpenRouter, or Vercel, evaluateTypeSafeJudgments dispatches the capped raw PR diff to the Roomote calibration upstream. Restore the per-call opt-out (or keep this pre-screen out of shadowing) so review diffs do not take that extra forwarding path.
There was a problem hiding this comment.
The latest product direction explicitly preserves the existing opt-in shadow-forwarding behavior for pre-screen judgments, alongside opt-in deployment-owned capture. This is therefore intentionally not changed: the pre-screen passes no shadow override, and the regression test verifies the primary plus opt-in shadow requests. The PR wording now discloses that reviewed diffs from any connected repository may reach the configured shadow provider when that setting is enabled.
1 participant
What changed
Why this change was made
The pre-screen provides cheap triage context so the main LLM reviewer can prioritize likely risk areas without changing its authority to inspect the entire pull-request diff. Strict input, output, timeout, and fallback bounds keep the optimization from adding unbounded latency or weakening review quality.
Impact
Configured decision-model deployments receive advisory pre-screen hints in the initial and incremental GitHub review prompts. Existing opt-in judgment capture stores pre-screen decision state in the deployment-owned artifact bucket, and existing opt-in shadowing may send reviewed diffs from any connected repository to the configured shadow provider. Deployments without a decision model, and any review where pre-screening fails, retain the existing review path and behavior.