Skip to content

pr-review-security lane reports success and marks the head reviewed after permission denials stop it early #6421

Closed

Description

@kyle-sexton

Problem

On #6382 head f1d3233, the pr-review-security lane run ended success after 29 turns with 7 permission denials. Its checklist stopped before the hunter step and it posted no findings, yet it wrote a "reviewed through f1d3233" marker. Later pushes therefore got only an incremental review of new commits, so the full diff was never security-reviewed by the lane (a local review stood in).

Expected

A run that did not complete its review steps must not write the reviewed-through marker and should not conclude success (fail or neutral with a reason), so the next push gets a full review.

Evidence

🤖 Generated with Claude Code

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 5, 2026
  2. github-actions commented on Oct 5, 2026

    @github-actions
    Contributor

    Thanks for the report. Automated triage handed this issue to a maintainer.

    Automated triage. A maintainer reviews it.

  3. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 5, 2026
  4. kyle-sexton commented on Oct 11, 2026

    @kyle-sexton
    MemberAuthor

    The same failure hits the code-review lane, not only the security lane. On PR #7060, .github/workflows/pr-review-hosted.yml, job pr-review / claude-review-status:

    • Run 38119559231 (head 660c0bc, job 114411329359): the SDK result was subtype:"success", is_error:false, num_turns:42, permission_denials_count:5.
    • Run 38120506944 (head 61ab590, job 114414070418): subtype:"success", num_turns:22, permission_denials_count:5.

    In both runs the progress comment stayed at "Code review in progress", with "Fan out hunters + adversarial verification" and "Post findings / summary" unchecked. Even so, "Report review outcome" logged "Claude review completed", the reviewed-head marker was written, and the check went green. Neither run timed out or hit the turn cap (75). Other runs on the same PR (38121164028 with 8 denials, 38121689718, 38119866298) did post, so the denials alone don't always stop it.

    The issue title could drop "security" so it covers both lanes.

    🤖 Generated with Claude Code

  5. added theissue type on Oct 12, 2026
  6. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 12, 2026
  7. kyle-sexton commented on Oct 13, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. The owner approved applying independently verified recommendations (high-risk set, reviewed) on 2026-10-13.

    Both AI review lanes concluded success and wrote the reviewed-head marker even when the run stopped before posting findings. ci-workflows #740 (merged 2026-10-12) makes a run with no completed review fail as incomplete without a marker, and both hosted review workflows here pin that commit (v0.41.0).

    Verification notes:

    • Assumption that denial-truncated runs are caught by the new verify step is plausible from the step's design (no completed review -> incomplete, no marker), but no run has been observed under the v0.41.0 pin yet.

    Verified by an independent fresh-context check (confirmed, risk high): gh issue view 6421 incl. owner comment 2026-10-11 (same failure on the code-review lane); melodic-software/ci-workflows#740 MERGED 2026-10-12, merge commit a621ef8112677a519b662ec5d2547475c0101f7e = tag v0.41.0; origin/main .github/workflows/pr-review-hosted.yml:84 and pr-review-security-hosted.yml:85 pin a621ef8; ci-workflows@a621ef8 pr-review.yml (:64, :151 incomplete class) and pr-review-security.yml (:480-483 'Verify the review completed', no marker on incomplete); #7257 OPEN umbrella

    Closing: Shipped upstream: ci-workflows #740 adds a 'Verify the review completed' step that fails both review lanes as incomplete and writes no marker, and this repo pins that commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions