Skip to content

ci(release): fix recurring release failures since the v3 cut (CLI-2913) - #7465

Merged
Coly010 merged 3 commits into
developfrom
columferry/cli-2913-replace-the-fixed-30-minute-npm-propagation-wait-in-the
Nov 3, 2026
Merged

Coly010 merged 3 commits into
developfrom
columferry/cli-2913-replace-the-fixed-30-minute-npm-propagation-wait-in-the

Conversation

@Coly010

@Coly010 Coly010 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Most Release runs since the v3 cut have failed, and the next channel has quietly stopped releasing. There are four separate causes, plus two smaller release-notes failures.

next and develop tag the same commits. Between majors next has no commits of its own, so the sync fast-forwards it onto develop and every fix or feature releases on both channels from one SHA. semantic-release reads channel notes per commit and gives every tag on a commit the first note (semantic-release#4073), so v3.1.0-next.1 reads as a beta release. Every next run since has recomputed 3.1.0-next.1, published nothing and gone green. The same shared commits break beta's release-notes backfill: its note seeding looked for notes on the annotated tag object (which never has one), force-wrote a note for every tag, and on a shared commit the next note won, so backfilling v3.1.0-beta.2 computed v3.1.0-beta.1.

  • A new next-ahead job compares next with develop. A next push with no commits beyond develop skips Test next and the release, and a channel=next dispatch in that state is refused up front. The plan checks again after the tests and the stable-release wait, so a major cut landing in between can't make next tag a commit develop also tags. Once a breaking change lands, next releases from its own commits again. Until then supabase@next stays on 3.1.0-next.1 and supabase@beta carries the newest code.
  • The backfill drops other channels' prerelease tags (and non-CLI tags) in its temporary clone and rebuilds one channel note per tagged commit from the tag names, instead of reading or overwriting notes.

The setup-cli v3 smoke test races npm. setup-cli v3 installs through npm. npm lists a new version a few minutes after publish, and the registry CDN then caches each packument for five minutes, separately per region and per Accept header. One run failed with a 404 several seconds after npm had already recorded the publish. A single wait job on one runner can't cover the macOS, Windows and Alpine runners, so:

  • setup-cli v3 is pinned to v3.2.0, and each v3 job first runs .github/scripts/wait-for-npm-version.sh, which polls npm view and the abbreviated packument from that runner for both supabase and the runner's platform package, for up to 15 minutes.
  • Since v3.1.0, setup-cli falls back to the GitHub release whenever the npm install fails, so the v3 jobs now fail unless install-source is npm. One extra ubuntu job forces the fallback and checks it still works.

The platform-package wait is too short. publish.ts waits for every platform package to become installable before publishing the umbrella. That has taken anywhere from 6 to more than 15 minutes, and two releases timed out at the 15-minute limit. The ceiling is now 30 minutes and the publish job timeout 60. It's a poll, so a quick npm costs nothing extra.

A beta planned during a stable release misses the stable tag. main pushes its tag only after the npm publish, so a develop push in that window computed a stale beta version (3.0.0-beta.18 was planned while 3.0.0 was publishing). The develop and next plans now wait, for up to 120 minutes, until the publish job of any in-flight Release run on main has finished. A shared concurrency group would have let a pending develop run cancel a pending main release.

A forced maintenance version gets no release notes. The 2.122.1 maintenance release was dispatched with an explicit version, and its only commits were ci: commits, so the backfill's semantic-release dry run computed no release and left the GitHub release body empty. The backfill now gives semantic-release an extra commit analyzer that returns the release type implied by the tag's own version, so commit analysis still wins when it computes more and a forced version still gets its notes.

Proposing release notes fails to mint a token. propose-release-notes.yml asked for an issues: read permission the releaser app doesn't have, so token creation returned 422. The request is dropped; the notes agent reads public issues without it.

This supersedes #7439. These are release-infra paths, so v2.x needs the setup-cli, platform-wait, backfill and propose-release-notes changes before its next maintenance release; v2.x has no next channel, so the next-ahead gate doesn't apply there.

Closes CLI-2913, CLI-2914, CLI-2915, CLI-2916, CLI-2917

🤖 Generated with Claude Code

- Skip next's test and release when next has no commits beyond develop,
  since semantic-release cannot tell a next tag from a beta tag on the
  same commit (semantic-release#4073); refuse such a next dispatch.
- Rebuild the backfill's channel notes per commit from this channel's
  tags, and floor its release type at the tag's own version so forced
  maintenance versions get notes.
- Pin setup-cli v3.2.0 in the smoke test, wait per runner for npm to
  serve the version and platform package, and assert the npm install
  path; add a forced GitHub-release fallback job.
- Raise the platform-package wait to 30 minutes.
- Make develop/next plans wait for main's in-flight publish job.
- Drop the issues permission propose-release-notes cannot get.

Closes CLI-2913, CLI-2914, CLI-2915, CLI-2916, CLI-2917
@Coly010
Coly010 requested review from a team and supabase-oss as code owners November 3, 2026 12:57
@Coly010
Coly010 requested a review from Prashansa-K November 3, 2026 12:57
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

CLI benchmark (Lightweight Commands): no changes found

Commit fea7b9e against develop at a5e1884 · 10 runs each · report only, never blocks merging

Command Workload Before After Change
supabase --version version 253 ms 246 ms no real change
supabase --help help 220 ms 224 ms no real change
supabase init init 229 ms 221 ms no real change
supabase migration new bench migration-new 217 ms 212 ms no real change
supabase gen types --lang typescript --project-id replayfixtureprojref gen-types 233 ms 237 ms no real change
supabase projects list projects-list 213 ms 218 ms no real change
supabase functions list --project-ref replayfixtureprojref functions-list 219 ms 217 ms no real change

API calls: unchanged for all 7 commands.

Binary size: 134.0 MB, unchanged.

Details: timing ranges, API calls per command, slowest phases

Timing ranges

Command Workload Before range After range Change Verdict
supabase --version version 216-292 ms 230-282 ms -7 ms (-3%) within noise
supabase --help help 201-243 ms 212-264 ms +4 ms (+2%) within noise
supabase init init 187-258 ms 196-271 ms -8 ms (-4%) within noise
supabase migration new bench migration-new 199-257 ms 188-233 ms -5 ms (-2%) within noise
supabase gen types --lang typescript --project-id replayfixtureprojref gen-types 215-259 ms 209-254 ms +4 ms (+2%) within noise
supabase projects list projects-list 196-237 ms 187-237 ms +5 ms (+2%) within noise
supabase functions list --project-ref replayfixtureprojref functions-list 204-236 ms 195-238 ms -2 ms (-1%) within noise

API calls per command

Command Workload Before After Endpoint changes
supabase --version version 0 0
supabase --help help 0 0
supabase init init 0 0
supabase migration new bench migration-new 0 0
supabase gen types --lang typescript --project-id replayfixtureprojref gen-types 2 2
supabase projects list projects-list 1 1
supabase functions list --project-ref replayfixtureprojref functions-list 2 2

Slowest phases

No command moved beyond noise, so there are no phase deltas.

Workflow run

Comment /bench to run the lightweight benchmark (this comment) again on the latest commit, or /bench full to also run the integration-heavy benchmark, which posts in its own comment.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Verified all five findings against the checked-out code and trusted conventions: four confirmed and one refuted. No findings overlap. Focused checks reproduced the prerelease comparison errors and polling-budget overrun; workflow sequencing was verified statically.

Findings

Severity Location Category Sources Claim
🟠 MAJOR .github/workflows/release.yml:95 release-sequencing codex The cached ahead check allows a next release to proceed after a major cut has advanced develop to the candidate commit.
🟡 MINOR .github/workflows/release.yml:130 reliability claude The 60-minute stable-release wait can expire while main's release is progressing within its allowed timeouts, failing the prerelease plan and requiring a manual rerun.
🟡 MINOR .github/scripts/wait-for-npm-version.sh:19 timeout-handling codex An npm probe can exceed the script's default 15-minute polling budget and potentially exhaust the smoke job's timeout.
🟡 MINOR apps/cli/scripts/release-channels.ts:69 correctness codex releaseTypeBetween returns patch for prerelease downgrades, contrary to its documented requirement to return null when the target version is not later.
Refuted findings (kept for transparency, not posted as review comments)
  • apps/cli/scripts/backfill-release-notes.e2e.test.ts:112 (style): The Bun executable guard throws a plain Error inside Effect.gen instead of following the repository's claimed requirement to use an explicit Effect failure or defect.
    Refuted: The trusted Effect guidance requires tagged errors for expected typed failures, but does not require every test prerequisite failure to be typed. Trusted tests already use this identical guard. A throw inside Effect.gen becomes a defect, as the suggested Effect.die would; no distinct failure-handling problem was identified.

Stats

Claude findings: 2 · Codex findings: 3 · Confirmed: 4 · Refuted: 1 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread .github/workflows/release.yml Resolved
Comment thread .github/workflows/release.yml Resolved
Comment thread .github/scripts/wait-for-npm-version.sh Resolved
Comment thread apps/cli/scripts/release-channels.ts Resolved
- Re-check that next is ahead of develop after the tests and the stable
  wait, so a major cut in between cannot tag a shared commit.
- Wait up to 120 minutes for main's publish job.
- Bound each npm probe and stop probing once the budget is spent.
- Return no release type for equal or earlier prerelease versions.
@Coly010 Coly010 self-assigned this Nov 3, 2026
@Coly010

Coly010 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

/ai-review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Reviewed all four findings against the surrounding code and trusted conventions. Confirmed two minor issues: failed GitHub API polls abort release planning, and interrupted tests leave subprocesses running. The 2.x package-compatibility claim remains uncertain without published package history; the PR-description mismatch remains uncertain because the description was not supplied. No findings overlap.

Findings

Severity Location Category Sources Claim
🟡 MINOR .github/workflows/setup-cli-smoke-test.yml:71 ci-robustness claude The unconditional platform-package wait would make manual smoke tests of 2.x versions without @supabase/cli-* packages time out, and could affect backports to v2.x.
🟡 MINOR .github/workflows/release.yml:132 error-handling claude A failed GitHub API poll immediately aborts the stable-release wait and fails release planning instead of retrying within its 120-minute budget.
🟡 MINOR apps/cli/scripts/backfill-release-notes.e2e.test.ts:27 resource-cleanup codex The new subprocess helper leaves its child running when the test effect is interrupted, including on timeout.
⚪ NIT .github/workflows/release.yml:130 documentation claude The PR description reportedly promises a 60-minute stable-release wait, while the implementation and release documentation specify 120 minutes.

Stats

Claude findings: 3 · Codex findings: 1 · Confirmed: 2 · Refuted: 0 · Uncertain: 2


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread .github/workflows/release.yml Resolved
Comment thread .github/workflows/release.yml Resolved
@Coly010
Coly010 added this pull request to the merge queue Nov 3, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 3, 2026
@Coly010
Coly010 added this pull request to the merge queue Nov 3, 2026
Merged via the queue into develop with commit 497bf42 Nov 3, 2026
58 checks passed
@Coly010
Coly010 deleted the columferry/cli-2913-replace-the-fixed-30-minute-npm-propagation-wait-in-the branch November 3, 2026 13:55
Coly010 added a commit that referenced this pull request Nov 5, 2026
…2.x (#7507)

Cherry-picks #7467 and #7465 onto `v2.x`, as the maintenance line needs
both before its next release.

- **#7467:** attests every GitHub Release asset with build provenance.
`attestations: write` is granted to the `publish` job and to the
`release.yml` and `release-smoke-test.yml` callers. The smoke test
conflicted, so the permission goes in its top-level block.
- **#7465:** the platform-package wait polls for up to 30 minutes with a
60-minute `publish` job timeout. `backfill-release-notes` drops other
channels' prerelease tags, rebuilds one channel note per tagged commit
from tag names, and forces the release type implied by the tag's own
version so a forced maintenance version still gets notes. `v2.x`'s
script is plain async, so those changes are ported by hand rather than
merged.

Left out of #7465 because `v2.x` has no `next` channel and its
`release.yml` only runs as a maintenance dispatch: the `next-ahead`
gate, the in-flight stable release wait, the setup-cli v3 smoke test
changes (`v2.x` covers v1 and v2 only, so `wait-for-npm-version.sh`
would be unused) and the matching docs. `propose-release-notes.yml`
already requests no `issues: read`, so there is nothing to remove.

---------

Co-authored-by: Julien Goux <hi@jgoux.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants