Skip to content

ci: publish all packages from a single pypi job - #7294

Open
u9g wants to merge 1 commit into
mainfrom
jason/publish-single-job
Open

u9g wants to merge 1 commit into
mainfrom
jason/publish-single-job

Conversation

@u9g

@u9g u9g commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What

Collapse the per-package publish matrix in publish.yml into a single job that downloads every dist-* artifact and runs pypa/gh-action-pypi-publish once.

Why

The pypi environment now has required reviewers (see the hardening thread in #team-agents-tickets). Each matrix leg is its own pending deployment, so the v1.8.2 release raised one review request per package and the GitHub Slack app posted 100+ messages to #updates-code, each tagging all six reviewers.

With one publish job there is one pending deployment per release: one approval click, one Slack message.

Notes

  • Trusted publishing mints a token scoped to every project whose publisher config matches the workflow and environment, so one job can upload all packages.
  • skip-existing: true keeps republishes working. Before, a package that was already on PyPI only failed its own matrix leg. Now all dists upload from one job, so already-published files must be skipped instead of aborting the rest.
  • The dist-* pattern also matches the per-OS intermediate artifacts for blockguard and durable. They contain the same wheel filenames as the merged artifact, so merging them is a no-op.
  • Downstream jobs (docs, deploy-examples, dispatch-downstream-bumps) already keyed off the aggregate needs.publish.result, so their behavior is unchanged.

The pypi environment now has required reviewers, and with the publish job
as a per-package matrix every release raised one deployment review per
package, each one notifying every reviewer. One job means one approval.

skip-existing keeps republishes working now that all dists upload together.
@u9g
u9g requested a review from a team as a code owner September 15, 2026 18:25

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Devin Review

Comment on lines +227 to +229
pattern: dist-*
path: dist/
merge-multiple: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Partial builds produce successful releases

When a package build fails, pattern downloads the remaining artifacts and publishing can succeed. The ignored build failure lets downstream deployment and version bumps proceed after an incomplete release.

Learn more

The build job is a matrix over every discovered package. Its aggregate result becomes failure or cancelled when a package build does not complete, but the publish job runs whenever discovery succeeded. A wildcard artifact download only requires at least one matching artifact and cannot detect which expected package is absent. The publish action can therefore upload the available distributions and return success. The dependent docs, examples, and downstream-bump jobs then treat the incomplete release as complete.

Example: Suppose 78 package builds upload artifacts and livekit-plugins-openai fails before upload. The wildcard downloads the 78 available artifacts, publishes them, and returns success. The workflow bumps downstream repositories even though the new OpenAI package version is absent from PyPI.

Recommended fix: Gate the publish job on needs.build.result == 'success' while retaining always() if needed for explicit result checks. Alternatively, validate the downloaded artifacts against needs.discover.outputs.packages before publishing and fail on every missing package.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@u9g

u9g commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Local verification of the upload behavior (the OIDC step can't run outside Actions):

  • actionlint on the workflow: no new findings, the 4 shellcheck warnings it reports are already on main.
  • Against a local pypiserver with twine 7.0.0 (the version pinned by the pypa action), built wheels for livekit-agents, plugins-deepgram, plugins-openai:
    • one twine upload with all three wheels: all uploaded, exit 0.
    • republish with a 4th wheel (plugins-cartesia) and no skip-existing: aborted on the first duplicate, cartesia never uploaded, exit 1. This is the regression skip-existing guards against.
    • same republish with skip-existing: the three duplicates skipped with warnings, cartesia uploaded, exit 0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant