Skip to content

[rush] Add PNPM global virtual store support via RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE - #6005

Open
Jérémy Dieuze (QuanticPotatoes) wants to merge 3 commits into
microsoft:mainfrom
QuanticPotatoes:feature/d-11160-pnpm-gvs-pr
Open

Jérémy Dieuze (QuanticPotatoes) wants to merge 3 commits into
microsoft:mainfrom
QuanticPotatoes:feature/d-11160-pnpm-gvs-pr

Conversation

@QuanticPotatoes

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #5830 (currently conflicts with main). Original work by EscapeB (@EscapeB); this PR rebases it onto current main and keeps the env-var design from that review.

When RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1, workspace installs set enableGlobalVirtualStore: true in the generated pnpm-workspace.yaml. Package instances live in the shared PNPM store instead of being recreated under each worktree's common/temp/node_modules/.pnpm.

The lockfile does not change. The package-manager install-lock skip stays in #5844.

Details

  • Opt-in via RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE, not pnpm-config.json. A per-subspace config flag would allow mixed layouts in one repo (called out on [rush] Add PNPM global virtual store support for rush install #5830).
  • Requires a shared store: pnpmStore global, or RUSH_PNPM_STORE_PATH pointing outside the repo. A worktree-local store throws. A path override inside the repo warns and still enables the flag (store stays local, so no worktree win).
  • Requires PNPM >= 10.12.1.
  • Incompatible with usePnpmSyncForInjectedDependencies (pnpm-sync still assumes node_modules/.pnpm in the worktree).
  • Last-install flag records the setting so flipping it forces a reinstall.
  • Adapted to the current PnpmWorkspaceFile / EnvironmentConfiguration shape (public workspace fields, saveAsync, module-level env state, globalPnpmfile).

How it was tested

  • Existing unit tests from [rush] Add PNPM global virtual store support for rush install #5830, updated for the current APIs (PnpmWorkspaceFile.test.ts, WorkspaceInstallManager.test.ts, BaseInstallManager.test.ts, EnvironmentConfiguration.test.ts, LastInstallFlag.test.ts).
  • rush-package-manager-integration-test coverage from [rush] Add PNPM global virtual store support for rush install #5830 kept.
  • Dogfood on a 7-subspace Rush repo that already uses RUSH_PNPM_STORE_PATH (shared content store). The four installed subspaces still keep a worktree-local virtual store: 5.4 GB / 531k inodes under node_modules/.pnpm (api alone: 2.0 GB / 204k inodes / 2606 instances). With this PR and RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1, rush install --subspace api wrote enableGlobalVirtualStore: true. Worktree common/temp/api/node_modules/.pnpm dropped to 1.5 MB / 2354 inodes (2 top-level entries). Package instances moved to ~/.rush-pnpm-store/v10/links (+2.0 GB). A second worktree on the same store installed in 6.2s; store size did not grow; .pnpm stayed 1.5 MB. Lockfile unchanged.

EscapeB (@EscapeB), happy to close this if you already have a rebase in flight. Otherwise this is meant to land #5830.

Fixes #5830

Comment thread libraries/rush-lib/src/api/EnvironmentConfiguration.ts Outdated
Comment thread libraries/rush-lib/src/api/LastInstallFlag.ts Outdated
Comment thread libraries/rush-lib/src/logic/installManager/WorkspaceInstallManager.ts Outdated
Comment thread libraries/rush-lib/src/logic/test/WorkspaceInstallManager.test.ts Outdated
Comment thread libraries/rush-lib/src/logic/test/WorkspaceInstallManager.test.ts Outdated
Comment thread libraries/rush-lib/src/logic/test/WorkspaceInstallManager.test.ts Outdated
Comment thread libraries/rush-lib/src/logic/test/WorkspaceInstallManager.test.ts Outdated
Comment thread libraries/rush-lib/src/logic/installManager/WorkspaceInstallManager.ts Outdated
When RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1, workspace installs set
enableGlobalVirtualStore in the generated pnpm-workspace.yaml so package
instances live in the shared PNPM store instead of each worktree's
node_modules/.pnpm.

Follow-up to microsoft#5830, rebased onto current main.

Co-authored-by: chenzhelong.sirius <EscapeB@users.noreply.github.com>
microsoft#5994 exported this regexp for TrimRushEnvironmentVariablesPlugin.
The GVS follow-up accidentally dropped it and inlined /^RUSH_/i.
@QuanticPotatoes

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (was CONFLICTING).

Conflict resolution:

  • TestHelper.ts: kept the inode-identity check for workspace links (needed when GVS hardlinks) and fs.realpath for the error paths, matching the main-side helper change.
  • rush-lib.api.md: restored trimRushEnvironmentVariablesForOperations that the rebase auto-merge dropped.

Local verification on the rebased tip:

  • rush test --to @microsoft/rush-lib: 948 tests, 0 failures (API Extractor included)
  • PNPM 10.12.1 GVS integration test: enable, switch off without --purge, switch back, lockfile unchanged, build OK

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs triage

Development

Successfully merging this pull request may close these issues.

2 participants