Skip to content

fix(ci): handle oversized PR diff fallback safely - #9678

Open
Krrish Mittal (takyyon) wants to merge 1 commit into
Azure:mainfrom
takyyon:fix/pr-ai-validation-oversized-diff
Open

Krrish Mittal (takyyon) wants to merge 1 commit into
Azure:mainfrom
takyyon:fix/pr-ai-validation-oversized-diff

Conversation

@takyyon

@takyyon Krrish Mittal (takyyon) commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Commit Type

  • feature - New functionality
  • fix - Bug fix
  • refactor - Code restructuring without behavior change
  • perf - Performance improvement
  • docs - Documentation update
  • test - Test-related changes
  • chore - Maintenance/tooling

Risk Level

  • Low - Minor changes, limited scope
  • Medium - Moderate changes, some user impact
  • High - Major changes, significant user/system impact

What & Why

PR AI Validation fails before assessment when GitHub rejects a raw diff over 20,000 lines. This is the required-check failure observed on #9677.

Fall back to bounded, paginated native file patches only for the structured HTTP 406 PullRequest/diff/too_large error. Preserve normal raw-diff retrieval, explicitly identify unavailable patches, and propagate unrelated errors.

Keep the existing 4,000-line gather bound and 2,000-line/100,000-byte prompt limits. Apply the line and byte limits through an intermediate temporary file instead of a producer pipe: large inputs otherwise cause SIGPIPE under Actions' strict Bash settings.

Run actual-workflow regression tests from the existing trusted checkout. Permissions, models, verdict handling and the pull_request_target trust boundary remain unchanged; no PR-head checkout is introduced.

Impact of Change

  • Users: No product UX or API changes.
  • Developers: Oversized PRs can reach assessment without weakening validation or silently accepting missing input.
  • System: Changes only the validator workflow and its regression test script. Because the workflow runs from the trusted base, this fix must be integrated there before a fresh PR event can use it.

Test Plan

  • Unit tests added/updated

  • E2E tests added/updated

  • Manual testing completed

  • Tested in: local focused workflow/test checks.

  • node --test scripts/pr-ai-validation.test.mjs: 30 passed, 0 failed.

  • Tests execute the actual extracted workflow scripts, including strict Bash with two-megabyte inputs, pagination bounds, missing patches, exact line/byte limits, input/temp-file failures and trusted-checkout test wiring.

Contributors

Krrish Mittal (@takyyon)

Screenshots/Videos

N/A (CI workflow/test-only change).

Add a guarded fallback for the structured PR raw-diff-too-large 406 path so AI PR validation can gather bounded data from listFiles patches without relaxing normal raw-diff behavior.\n\nInclude strict regression coverage that executes the extracted workflow JavaScript and extracted bash snippet under pipefail, including large input and error-path contracts.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>\nCopilot-Session: c0d9bc12-a47f-4e97-9b1e-274b5574525d
@takyyon Krrish Mittal (takyyon) added the risk:medium Medium risk change with potential impact label Oct 30, 2026
Copilot AI balanced review requested due to automatic review settings October 30, 2026 16:39
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AI PR Validation Report

PR Review Results

Thank you for your submission! Here's detailed feedback on your PR title and body compliance:

✅ PR Title

  • Current: fix(ci): handle oversized PR diff fallback safely
  • Issue: The title is specific, follows a conventional format, and accurately describes the CI behavior change.
  • Recommendation: No changes required.

✅ Commit Type

  • Properly selected (fix).
  • Only one commit type is selected, and the workflow bug fix is consistent with this classification.

✅ Risk Level

  • The Medium risk level is selected in the body and matches the risk:medium label.
  • The advised risk level is medium, based on changes to a pull_request_target validation workflow, GitHub API error handling, pagination, and prompt-input processing. The documented trust boundary is preserved, so a high-risk classification is not required.

✅ What & Why

  • Current: Clearly explains the oversized-diff failure, the structured fallback behavior, sampling limits, temporary-file change, testing approach, and preserved security assumptions.
  • Issue: No compliance issues identified.
  • Recommendation: No changes required.

✅ Impact of Change

  • The impact section identifies the absence of product UX/API changes and explains the effects on developers and the validation workflow.
  • Recommendation:
    • Users: The stated absence of product UX or API changes is appropriate.
    • Developers: The explanation of oversized PR handling and validation behavior is sufficient.
    • System: The workflow scope, trusted checkout behavior, and unchanged trust boundary are documented appropriately.

✅ Test Plan

  • Unit tests are marked and a test file is added in the diff (scripts/pr-ai-validation.test.mjs). The described 30 passing tests cover normal diff retrieval, structured oversized-diff fallback, pagination limits, unavailable patches, error propagation, strict Bash sampling, and trusted-checkout wiring.
  • Manual testing is also documented with the command and result. The test plan passes because unit tests are present in the diff.

⚠️ Contributors

  • @takyyon is credited.
  • Consider adding PMs, designers, reviewers, or other contributors if they helped shape or validate this change; this is optional and does not block approval.

✅ Screenshots/Videos

  • Correctly marked as not applicable because this is a CI workflow and test-only change with no visual product changes.

Summary Table

Section Status Recommendation
Title ✅ No changes required.
Commit Type ✅ fix is correctly selected.
Risk Level ✅ Medium matches both the body and risk:medium label.
What & Why ✅ Explanation is clear and sufficiently detailed.
Impact of Change ✅ User, developer, and system impacts are covered.
Test Plan ✅ Unit tests are present in the diff and manual testing is documented.
Contributors ⚠️ Add other contributors if applicable.
Screenshots/Videos ✅ Correctly identified as not applicable.

The PR title and body pass review. No required updates are needed.


Powered by: Logic App (fallback) | Last updated: Fri, 30 Oct 2026 16:45:50 GMT

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the oversized-pull-request validation changes.

No open issues 🎉

What changed in this PR

This PR repairs AI PR Validation for oversized pull requests while preserving normal diff handling and existing validation policy.

Changes:

  • Falls back to bounded, paginated file patches only for GitHub’s structured oversized-diff error; other errors still propagate.
  • Separates shell sampling reads to avoid SIGPIPE failures while retaining line and byte limits.
  • Adds regression tests and runs them from the trusted checkout before gathering PR data.
File Description
scripts/​pr-ai-validation.test.mjs Tests fallback handling, sampling limits, failure propagation, and trusted-checkout configuration.
.github/​workflows/​pr-ai-validation.yml Adds the guarded fallback, fixes shell sampling, and integrates regression tests.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-validated risk:medium Medium risk change with potential impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants