[rush] Add PNPM global virtual store support via RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE - #6005
Open
Jérémy Dieuze (QuanticPotatoes) wants to merge 2 commits into
Open
Conversation
When RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1, workspace installs set enableGlobalVirtualStore in the generated pnpm-workspace.yaml so package instances live in the shared PNPM store instead of each worktree's node_modules/.pnpm. Follow-up to microsoft#5830, rebased onto current main. Co-authored-by: chenzhelong.sirius <EscapeB@users.noreply.github.com>
microsoft#5994 exported this regexp for TrimRushEnvironmentVariablesPlugin. The GVS follow-up accidentally dropped it and inlined /^RUSH_/i.
| * If true, enables PNPM's global virtual store during workspace installs. | ||
| * See {@link EnvironmentVariableNames.RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE} | ||
| */ | ||
| public static get pnpmGlobalVirtualStore(): boolean { |
There was a problem hiding this comment.
Suggested change
| public static get pnpmGlobalVirtualStore(): boolean { | |
| public static get enablePnpmGlobalVirtualStore(): boolean { |
Along with renaming _pnpmGlobalVirtualStore. As-written, this sounds like it may be a path.
| const oldPnpmGlobalVirtualStore: boolean = oldState.pnpmGlobalVirtualStore === true; | ||
| const newPnpmGlobalVirtualStore: boolean = newState.pnpmGlobalVirtualStore === true; | ||
| if (oldPnpmGlobalVirtualStore !== newPnpmGlobalVirtualStore) { | ||
| throw new Error( |
There was a problem hiding this comment.
Pnpm can't automatically figure this out?
| this.rushConfiguration.experimentsConfiguration.configuration?.usePnpmSyncForInjectedDependencies | ||
| }); | ||
| if (globalVirtualStoreWarning) { | ||
| this._terminal.writeWarningLine(Colorize.yellow(globalVirtualStoreWarning)); |
There was a problem hiding this comment.
Warnings are printed in yellow by default
Suggested change
| this._terminal.writeWarningLine(Colorize.yellow(globalVirtualStoreWarning)); | |
| this._terminal.writeWarningLine(globalVirtualStoreWarning); |
| }); | ||
|
|
||
| describe('prepareCommonTempAsync', () => { | ||
| const fixtureRepoPath: string = path.resolve(__dirname, 'repoWithSubspacesCatalogs'); |
There was a problem hiding this comment.
Suggested change
| const fixtureRepoPath: string = path.resolve(__dirname, 'repoWithSubspacesCatalogs'); | |
| const fixtureRepoPath: string = `${__dirname}/repoWithSubspacesCatalogs`; |
| let originalPnpmGlobalVirtualStoreEnvValue: string | undefined; | ||
|
|
||
| beforeEach(() => { | ||
| originalPnpmStorePathEnvValue = process.env[EnvironmentVariableNames.RUSH_PNPM_STORE_PATH]; |
There was a problem hiding this comment.
Can process.env properties be mocked by Jest so you don't have to do this manual record+reset?
| variant: undefined, | ||
| subspace: rushConfiguration.defaultSubspace, | ||
| terminal | ||
| } as unknown as IInstallManagerOptions; |
There was a problem hiding this comment.
Is the as unknown actually required?
|
|
||
| function prepareFixtureRepo(options: { pnpmStore?: PnpmStoreLocation }): RushConfiguration { | ||
| const repoPath: string = `${tempFolderPath}/repo`; | ||
| FileSystem.copyFiles({ |
There was a problem hiding this comment.
Use the async variants for filesystem operations.
| ); | ||
| } | ||
|
|
||
| if (options.pnpmStore === 'local' && !options.pnpmStorePathOverride) { |
There was a problem hiding this comment.
Destructure options.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #5830 (currently conflicts with
main). Original work by EscapeB (@EscapeB); this PR rebases it onto currentmainand keeps the env-var design from that review.When
RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1, workspace installs setenableGlobalVirtualStore: truein the generatedpnpm-workspace.yaml. Package instances live in the shared PNPM store instead of being recreated under each worktree'scommon/temp/node_modules/.pnpm.The lockfile does not change. The package-manager install-lock skip stays in #5844.
Details
RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE, notpnpm-config.json. A per-subspace config flag would allow mixed layouts in one repo (called out on [rush] Add PNPM global virtual store support for rush install #5830).pnpmStoreglobal, orRUSH_PNPM_STORE_PATHpointing outside the repo. A worktree-local store throws. A path override inside the repo warns and still enables the flag (store stays local, so no worktree win).>= 10.12.1.usePnpmSyncForInjectedDependencies(pnpm-sync still assumesnode_modules/.pnpmin the worktree).PnpmWorkspaceFile/EnvironmentConfigurationshape (public workspace fields,saveAsync, module-level env state,globalPnpmfile).How it was tested
PnpmWorkspaceFile.test.ts,WorkspaceInstallManager.test.ts,BaseInstallManager.test.ts,EnvironmentConfiguration.test.ts,LastInstallFlag.test.ts).rush-package-manager-integration-testcoverage from [rush] Add PNPM global virtual store support for rush install #5830 kept.RUSH_PNPM_STORE_PATH(shared content store). The four installed subspaces still keep a worktree-local virtual store: 5.4 GB / 531k inodes undernode_modules/.pnpm(api alone: 2.0 GB / 204k inodes / 2606 instances). With this PR andRUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1,rush install --subspace apiwroteenableGlobalVirtualStore: true. Worktreecommon/temp/api/node_modules/.pnpmdropped to 1.5 MB / 2354 inodes (2 top-level entries). Package instances moved to~/.rush-pnpm-store/v10/links(+2.0 GB). A second worktree on the same store installed in 6.2s; store size did not grow;.pnpmstayed 1.5 MB. Lockfile unchanged.EscapeB (@EscapeB), happy to close this if you already have a rebase in flight. Otherwise this is meant to land #5830.
Fixes #5830