Skip to content

npm registries as a built-in format plugin: hosted, proxy and group repositories with npm login - #463

Merged
schmitch merged 3 commits into
mainfrom
claude/41-npm-registry
Oct 24, 2026
Merged

schmitch merged 3 commits into
mainfrom
claude/41-npm-registry

Conversation

@schmitch

@schmitch schmitch commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Fixes #41

npm registries for npm, pnpm, Yarn and Bun at /npm/<repository>/: hosted, proxy and group repositories of a new built-in format plugin, crates/formats/npm, which implements FormatPlugin and is served by the {mount}/{repo} dispatcher like NuGet and raw. No npm code lives in the server: everything npm needed from the host went into the plugin contract as a general feature.

The crate was asked for at plugins/npm; #456 has since moved every format to crates/formats/<id>/ (AGENTS.md), so it sits next to crates/formats/nuget and crates/formats/raw, and like them it describes itself with a Rust DESCRIPTOR (#458).

What npm clients get

  • Packuments, abbreviated (application/vnd.npm.install-v1+json) and full (publish times in time, the README of latest), version documents, tarballs, npm search, npm ping.
  • npm publish: the base64 tarball streams out of the JSON document into the store; the tarball must match the integrity, shasum, name and version the document claims. The deployment policy decides on republishing (409 by default).
  • Dist-tags, npm deprecate, npm unpublish (a version or the package; the delete permission). latest cannot be removed and moves to the highest version left.
  • Logins: npm login is approved in the browser on a new web UI page (/cli-login/<secret>, password or OIDC sign-in), yarn npm login checks a password through the realm chain; both mint an API token (90 days, [cli_login]) that npm logout revokes; npm whoami. API tokens are accepted as Authorization: Bearer on every plugin repository.
  • Proxies of registry.npmjs.org or any registry: Basic credentials or a Bearer token (to the remote's origin only), the upstream's dist-tags and deprecations, tarballs verified against the published digests, the publication delay dated from the upstream's publish times, stale serving while the upstream is down.
  • Groups: merged versions, dist-tags of the first member that has them, reserved scopes (@acme) kept away from proxy members, a writable member that takes publishes and changes through the group, npm audit forwarded to the proxy member without the reserved names.
  • Retention never deletes a version a dist-tag names.
  • Host errors reach npm clients as {"error": "..."}, which they print.

Contract (forge:format@0.4.0, unreleased) and host

Additions to the 0.4.0 contract #456 introduced, all general:

  • Action::ReadBody (bodies on demand, bounded, with the permission their use needs), Action::Change (unlist, relist, delete, deprecate, tag, untag; every permission a set needs is checked before any change), Action::Auth (logins performed by the host, rendered by auth_response), Action::Forward (a POST to the upstream with the proxy's credentials).
  • Upload::Document with unpack_upload / parse_upload / published: a package inside a document, unpacked while it streams.
  • Query::Tags, Query::DatedPackages (publish times even where a proxy pays for the full document), UpstreamOp::Tags, UpstreamAnswer::Located (a package URL with its digests), Fetch::accept, error_response, readme, reserved_prefix, and Descriptor::upstream_token (Bearer for npm: a proxy's password without a username is sent as a Bearer token).
  • Package formats' actions are authorized one by one (a login needs nothing, an audit read, a group publish write on its writable member); files formats keep the up-front check by method that Move the format crates from plugins/ to crates/formats/ #456 introduced.
  • New tables package_tag and cli_login, component.deprecation (migration 0047).
  • A proxy whose upstream refuses its credentials (401/403) or serves a package it does not take answers 502 instead of 500, for NuGet too.

Admin API and web UI

  • The deployment policy and a group's writable member now apply to every package format (NuGet too); allow_redeploy follows the policy. The repository form offers both for package formats, and the edit and create bodies send them.
  • The package page shows a package's dist-tags, deprecated versions and README; a repository's page shows the .npmrc / .yarnrc.yml lines and install commands; /cli-login/<secret> approves a CLI login.

Tests

  • crates/formats/npm: 37 unit tests and 15 protocol tests (testing::serve).
  • crates/server/tests/npm.rs: 12 integration tests (publish, packuments, the deployment policy, dist-tags, deprecation, unpublishing, permissions, logins, proxies with a mock upstream, groups and the dependency-confusion guard, retention).
  • New CI job npm-e2e (.github/scripts/npm-e2e.sh): npm's browser login, Yarn's password login, logout, publishes with npm and pnpm (also through the group), installs with npm and pnpm (and Bun when installed) from hosted, proxy and group repositories, dist-tags, deprecation, unpublishing, the guard and npm audit. Branch protection needs npm-e2e added to the required checks by hand (it is in REQUIRED_JOBS and DEVELOPMENT.md).

Run locally after merging main (#458, #460):

  • cargo fmt --all --check; cargo clippy --workspace --all-targets --all-features -- -D warnings; cargo sqlx prepare --workspace --check; cargo xtask changelog --check
  • cargo test of every workspace crate (.github/scripts/test-parallel.sh --workspace, 108 binaries) against one shared Postgres: 103 passed in the parallel run, and the 5 that collided over the shared database (retention_worker, stopped midway by admin_retention's instance-wide switch; auth_realm, dao, encryption, workers) passed run alone. Not run: the gated forge-e2e suite.
  • .github/scripts/npm-e2e.sh with npm 11.21 and 10.9, pnpm 11.5 and 10.28, Yarn 4.18 and Bun: passed (npm 11 refuses a known version itself, so the republish check uses --force to reach the server)
  • .github/scripts/raw-e2e.sh (FORGE_E2E_SKIP_GITHUB=1), since raw shares the dispatcher this changes: passed
  • web: tsc, vitest (417)

Docs and changelog

docs/npm.md (new), docs/format-plugins.md (the 0.4.0 additions), crates/formats/README.md, README.md, DEVELOPMENT.md, forge.toml.example ([npm], [cli_login]); fragments 41.feature.md, 41-package-options.feature.md, 41.fixed.md, 41.limitation.md under docs/changes/2.0.0/.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BaGpNGgkQV54XvgL5D9ay2

npm, pnpm, Yarn and Bun against /npm/<repository>/: hosted, proxy and
group repositories of the new format plugin crates/formats/npm, served by the
{mount}/{repo} dispatcher like NuGet.

- Packuments, abbreviated and full (with publish times and the README of
  latest), version documents, tarballs, search, ping.
- npm publish: the base64 tarball streams out of the JSON document into
  the store (Upload::Document, unpack_upload), checked against the
  integrity and version it claims; the deployment policy decides on
  republishing.
- Dist-tags, deprecations and unpublishing as host changes (package_tag,
  component.deprecation); retention never deletes a tagged version.
- Logins: npm's browser login approved on /cli-login/<secret>, a password
  login (yarn npm login), logout and whoami, minting API tokens; Bearer
  API tokens are accepted on every plugin repository.
- Proxies: Bearer or Basic credentials, the upstream's dist-tags, tarballs
  verified against the published digests, publish times for the full
  packument (Query::DatedPackages), 502 for an upstream that refuses the
  proxy or serves a package that does not hold up.
- Groups: dist-tags from the first member that has them, reserved scopes,
  a writable member for publishes and changes, audits forwarded upstream
  without reserved names.
- Contract 0.4.0: request bodies on demand, document uploads, changes,
  logins, forwards, tags, host errors in the format's shape.
- Web UI: a package's dist-tags, deprecations and README, the npm connect
  snippets, the browser login approval page, and the deployment policy and
  writable member for every package format.
- Docs (docs/npm.md, format-plugins.md), the npm-e2e CI job with the real
  clients, changelog fragments.

Fixes #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BaGpNGgkQV54XvgL5D9ay2
npm 11 (CI's Node 26) checks the registry's versions before publishing
and refuses a known one itself, so the republish never reached the
server and its message did not match. With --force the client skips its
check and the server's 409 is what refuses the republish, with npm 10
and 11 alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BaGpNGgkQV54XvgL5D9ay2
Resolve the conflicts with #458 (formats describe themselves in Rust,
plugin.toml is gone) and #460 (access levels):

- npm describes itself with a DESCRIPTOR like NuGet and raw; plugin.toml's
  upstream_token key becomes the Descriptor field `upstream_token`
  (UpstreamToken::Bearer for npm, None for NuGet and raw).
- The dispatcher's per-action authorization uses authz::Level.
- The package_tag / cli_login migration becomes 0047: main's access
  levels took 0046.
- The cli-login web page's path is a reserved mount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BaGpNGgkQV54XvgL5D9ay2
@schmitch
schmitch enabled auto-merge October 24, 2026 14:58
@schmitch
schmitch added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit dd60323 Oct 24, 2026
23 checks passed
@schmitch
schmitch deleted the claude/41-npm-registry branch October 24, 2026 15:14
schmitch pushed a commit that referenced this pull request Oct 24, 2026
main now holds #463, this branch's former base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BaGpNGgkQV54XvgL5D9ay2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

npm registry: hosted, proxy and group repositories with npm login

2 participants