Skip to content

ci(ci-status): pass a draft's skipped lanes instead of failing it - #7348

Merged
kyle-sexton merged 3 commits into
mainfrom
ci/draft-ci-status-passes
Oct 13, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
ci/draft-ci-status-passes

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: the owner approved retiring the "draft PRs fail ci-status" rule in session.

Merge only after all three prerequisites hold:

  1. github-iac#692 is merged and applied (https://github.com/melodic-software/github-iac/pull/692). It removes the OrganizationAdmin bypass from this repository's merge-queue ruleset. Until it is applied, a green draft-era ci-status could admit a direct bypass merge.
  2. This PR's own ci-status is green on its head after the ready flip.
  3. The owner's autoMode "Gated PR Merge" guard (clause 6, in dotfiles) gains a "review checks not SKIPPED" clause, merged by the owner. pr-review-hosted and pr-review-security-hosted trigger only on pull_request (never merge_group), and their jobs skip on drafts. Right after a ready flip, the head can show this PR's green draft-era ci-status beside review checks COMPLETED as SKIPPED from the draft run. The guard today only requires each check named review to be COMPLETED, so a SKIPPED one passes and an agent could enqueue a PR nobody reviewed. This PR does not touch dotfiles.

Summary

A draft's ci-status now passes, with the notice draft: lanes not run; the ready_for_review run lints and tests this SHA, instead of failing. Lanes still skip on drafts, AI review lanes still skip drafts, and ready_for_review and merge_group behave as before.

Why it was red: between the flip to ready and the ready_for_review run's own ci-status, the draft's ci-status is the newest on the SHA, so a green one could let a merge through. The merge queue (ALLGREEN, squash) re-runs every lane on the merge_group commit, so once the queue has no bypass (prerequisite 1) no merge can rest on a draft-era result.

Fix

  • .github/workflows/pr-require-checks.yml
    • ci-status: removes the Fail a draft step, including its contract-only live-state re-check (994925b), which served only draft-red. Adds Note a draft, which writes the notice and a step-summary line.
    • Aggregate: treat-skipped-as: ${{ github.event.pull_request.draft && 'pass' || 'fail' }}, the form f0ffaf3 replaced. Outside a draft a skip is still red, apart from the ordered-skip conversion.
    • Comments: the lane-gate comment and the aggregate comment now say a draft passes. The header's "no bypass actor" statement now points at github-iac GovernedRepositories.cs; it is true once prerequisite 1 is applied.
  • scripts/check-docs-only-gate.sh property 11 accepts fail or exactly the draft form. check-docs-only-gate.test.sh now allows the draft form and rejects the flipped form draft && 'fail' || 'pass'.
  • Deletes scripts/ci-fail-a-draft.test.sh and its scripts/suite-seconds.txt row. No other registry or plan references it.
  • docs/ci-runner-routing.md and docs/conventions/pr-pipeline/README.md (with CHANGELOG 1.10.5) say a draft passes.

Unchanged: rerun-contract-only-siblings stays off on drafts.

Verification

  • actionlint .github/workflows/pr-require-checks.yml: clean.
  • scripts/check-docs-only-gate.test.sh: PASS=88 FAIL=0. check-docs-only-gate.sh --check on the shipped workflow: all references in the sanctioned form.
  • Every scripts/*.test.sh that reads pr-require-checks.yml passed: affected-tests, check-conformance-registry, check-detector-eval-coverage, check-docs-only, check-hooks-description, check-lane-coverage, ci-hold-merge-group, ci-ordered-skips, lint-shell-filter, plan-test-lanes, post-merge-demotion-alert, workflow-self-paths. The exception is selection-audit, which failed 1 of 12 because strace is missing on this host (environmental, unrelated).
  • shellcheck on the gate script and test: clean. markdownlint on the three changed docs: 0 issues. typos: clean.
  • Not yet observed live: this draft's own ci-status should go green with the notice.

Related

  • github-iac#692: the prerequisite queue-bypass removal.
  • f0ffaf3, a90848a and 994925b: the commits that made drafts red. This PR reverses the draft-red part only.
  • Other copies: pr-require-checks.yml is not synced into this repository; each repository owns its copy. melodic-software/architecture has its own Refuse a draft step. It should stay unless that repository gets a merge queue with no bypass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WVW9hAwnSBrnFsR3kzXcB4

kyle-sexton and others added 2 commits October 12, 2026 19:10
A draft's ci-status failed so that a green draft-era ci-status, the
newest on the SHA for a few minutes after the flip to ready, could not
admit a merge. With the merge-queue ruleset's owner bypass removed
(github-iac#692), every merge to main waits for the merge-group run,
which runs every lane, so that hole is closed by the queue.

- ci-status: the `Fail a draft` step and its contract-only live-state
  re-check go; a `Note a draft` step prints "draft: lanes not run; the
  ready_for_review run lints and tests this SHA" as a notice and in the
  step summary. The aggregate passes skips only on a draft
  (`treat-skipped-as: ${{ github.event.pull_request.draft && 'pass' ||
  'fail' }}`). Lanes still skip on drafts, and ready_for_review and
  merge_group behave as before.
- check-docs-only-gate.sh property 11 accepts `fail` or exactly that
  draft form; its tests allow the form and reject a flipped one.
- scripts/ci-fail-a-draft.test.sh and its suite-seconds entry go.
- The workflow header, docs/ci-runner-routing.md and the pr-pipeline
  convention README say a draft passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WVW9hAwnSBrnFsR3kzXcB4
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 13, 2026 01:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T01:17:25.418460Z 2d85e1e Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 6m 58s —— View job


Security review: CI security review (/review:security-review)

  • Gather context (PR diff, changed files)
  • Review .github/workflows/pr-require-checks.yml changes for logic/trust-boundary issues
  • Review scripts/check-docs-only-gate.sh / test changes
  • Check instruction-exception-register against the deleted "fail a draft" guardrail
  • Review docs changes for accuracy vs. actual behavior
  • Post findings
    · branch ci/draft-ci-status-passes

Finding (CRITICAL): this PR's own merge-hold intent does not engage the one mechanism built to enforce it

The PR body states three prerequisites that must hold before this change is safe to merge — most
critically, that github-iac#692 (removing the OrganizationAdmin merge-queue bypass) must land
first, since until it does, this diff's own change (making a draft's ci-status pass instead of
fail — pr-require-checks.yml#L2847-L2850, #L2933)
reopens exactly the bypass-merge window the author describes: a bypass actor could merge on a
green, lanes-skipped draft-era ci-status.

The PR currently carries no labels and the body's hold is phrased as prose — **Merge only after all three prerequisites hold:** — not as a line that starts with Hold:. The repo has two,
and only two, mechanisms that make a body hold or label hold actually binding:

  • do-not-merge label: the only cross-lane hold enforced server-side via ci-status's
    check-contract step (docs/conventions/loop-lane/README.md, "The do-not-merge label is the
    only cross-lane hold").
  • A body line matching BODY_MERGE_HOLD_RE in the babysit-prs merge gate
    (babysit_merge.py#L194):
    at this PR's head SHA the pattern is \bdo(?:n['’]?t| not)\s+merge\b only (no Hold: support
    yet — that lands in fix(source-control): hold the babysit merge on a body line starting Hold: #7349, merged into main after this PR's head commit). Even with that
    later Hold: support merged in, the pattern requires hold: to start the line (optional
    leading whitespace only); "**Merge only after all three prerequisites hold:**" has hold:
    mid-line, so it still would not match.

Net effect: neither GitHub's required-check gate nor the repo's own automated merge lane
recognizes this PR's explicit "do not merge yet" condition. Per this repo's own review rule
(AGENTS.md, "When a pull request is superseded or must not merge, apply the do-not-merge
label... before writing any explanation"), a PR in exactly this state — openly stating unmet
merge prerequisites — should carry that label now, or the body's hold line should be rewritten to
start with Hold: (e.g. Hold: merge only after github-iac#692 is merged and applied) so the
babysit lane's own hold pattern actually fires. Relying on prose that no mechanism reads leaves
this PR mergeable by any unattended/autonomous lane before its own stated preconditions are met —
the precise agent-authority / bypass-merge risk the PR body itself raises.

No other security-relevant issues found in the diff. The treat-skipped-as ternary and
check-docs-only-gate.sh property 11's exact-string check are internally consistent with the
PR's stated intent and are well covered by the updated test suite; the scripts/ci-fail-a-draft.test.sh
deletion leaves no stale references anywhere in the tree.

claude-lane-summary: lane=claude-security-review sha=ecf5b6d3af2fd503b33a671e10f0c529f9af60aa run=38331138715

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WVW9hAwnSBrnFsR3kzXcB4
@claude

claude Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 5m 3s —— View job


Code review complete

  • Gather context (PR diff, changed files)
  • Review .github/workflows/pr-require-checks.yml changes
  • Review scripts/check-docs-only-gate.sh / test changes
  • Review docs changes for accuracy vs. actual behavior
  • Check deleted test file / registry references
  • Post findings

No correctness or maintainability issues found in this diff.

What I checked:

  • treat-skipped-as: ${{ github.event.pull_request.draft && 'pass' || 'fail' }} — verified the GHA ternary idiom resolves correctly for the draft ('pass'), non-draft/ready_for_review ('fail'), and merge_group (no pull_request context → falsy → 'fail') cases, matching the stated intent.
  • Confirmed the removed Fail a draft step's special-cased live-draft re-check (gh api .../pulls/$PR --jq .draft) is no longer needed: that logic existed only to stop a stale-payload contract-only re-run from hard-failing the job outright. With the step replaced by a non-failing Note a draft notice and treat-skipped-as now passing a draft's skips, there's no remaining path where the stale github.event.pull_request.draft payload on a re-run could wrongly block a ready PR.
  • Verified scripts/check-docs-only-gate.sh's new SKIP_AS_DRAFT literal (scripts/check-docs-only-gate.sh:1066) matches the actual workflow line byte-for-byte (.github/workflows/pr-require-checks.yml:2933), so property 11 accepts the real file.
  • scripts/check-docs-only-gate.test.sh adds a case for the exact draft form and a case for the polarity-flipped form (fail/pass swapped) being rejected — good adversarial coverage of the one way this gate could be silently defeated.
  • Grepped the repo for leftover references to the deleted scripts/ci-fail-a-draft.test.sh and the old Fail a draft step name — none found outside this diff, consistent with the PR's own claim.
  • Doc updates (docs/ci-runner-routing.md, docs/conventions/pr-pipeline/README.md) accurately describe the new passing behavior and are consistent with the workflow changes.

Not independently verified: this sandboxed lane could not execute shell commands (bash -n, the .test.sh suites), so I read the scripts/workflow statically rather than re-running the author-claimed test results in the PR description.

One pre-existing condition worth naming (not a defect in this diff, already called out in the PR body as a merge prerequisite): the workflow comment at .github/workflows/pr-require-checks.yml:11 and the docs now assert in present tense that "the merge-queue ruleset has no bypass actor." That's only true once github-iac#692 is merged and applied — the PR description already gates the merge on that prerequisite, so no action needed here.

claude-lane-summary: lane=claude-review sha=1146e72960c5855cf67e6d6988d1012ade8c7aa1 run=38331534957

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review has reviewed this pull request through 1146e72; a later push is reviewed from there.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review has reviewed this pull request through 1146e72; a later push is reviewed from there.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit 69bb850 Oct 13, 2026
49 of 51 checks passed
@kyle-sexton
kyle-sexton deleted the ci/draft-ci-status-passes branch October 13, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant