Skip to content

vfs: add minimal node:vfs subsystem#63115

Open
mcollina wants to merge 32 commits into
nodejs:mainfrom
mcollina:vfs-minimal
Open

vfs: add minimal node:vfs subsystem#63115
mcollina wants to merge 32 commits into
nodejs:mainfrom
mcollina:vfs-minimal

Conversation

@mcollina
Copy link
Copy Markdown
Member

@mcollina mcollina commented May 4, 2026

Adds an experimental node:vfs builtin (gated behind --experimental-vfs) with VirtualFileSystem, VirtualProvider, MemoryProvider, and RealFSProvider. No integration with node:fs, the module loader, or SEA those are intended to land in follow-up PRs.

Extracted from: #61478

Approximate line counts: code ~4k / docs ~1k / tests ~5k — total ~10k lines, with tests being the largest share.

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/config
  • @nodejs/loaders
  • @nodejs/startup

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels May 4, 2026
@bakkot
Copy link
Copy Markdown
Contributor

bakkot commented May 4, 2026

No integration with node:fs

The docs in this PR claim that you can call myVfs.mount('/virtual') and subsequently regular node:fs operations which read from paths beginning with /virtual will read from the VFS. That sounds like integration to me. Are the docs wrong, or am I misunderstanding what "no integration" means?

@mcollina mcollina force-pushed the vfs-minimal branch 2 times, most recently from 30f5755 to 779fc37 Compare May 5, 2026 09:30
@mcollina mcollina marked this pull request as ready for review May 5, 2026 10:04
@mcollina
Copy link
Copy Markdown
Member Author

mcollina commented May 5, 2026

No integration with node:fs

The docs in this PR claim that you can call myVfs.mount('/virtual') and subsequently regular node:fs operations which read from paths beginning with /virtual will read from the VFS. That sounds like integration to me. Are the docs wrong, or am I misunderstanding what "no integration" means?

Fixed, good spot.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

❌ Patch coverage is 96.69149% with 191 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.31%. Comparing base (8d3245e) to head (1088e5f).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/vfs/providers/memory.js 93.64% 61 Missing and 4 partials ⚠️
lib/internal/vfs/file_system.js 96.68% 38 Missing ⚠️
lib/internal/vfs/watcher.js 95.20% 32 Missing and 1 partial ⚠️
lib/internal/vfs/providers/real.js 96.01% 19 Missing ⚠️
lib/internal/vfs/streams.js 95.46% 16 Missing ⚠️
lib/internal/vfs/provider.js 98.38% 10 Missing ⚠️
lib/internal/vfs/file_handle.js 98.88% 7 Missing and 1 partial ⚠️
lib/internal/vfs/fd.js 97.70% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #63115      +/-   ##
==========================================
+ Coverage   90.14%   90.31%   +0.16%     
==========================================
  Files         718      730      +12     
  Lines      227984   233756    +5772     
  Branches    42835    43803     +968     
==========================================
+ Hits       205522   211121    +5599     
- Misses      14235    14400     +165     
- Partials     8227     8235       +8     
Files with missing lines Coverage Δ
lib/internal/bootstrap/realm.js 96.29% <100.00%> (+0.06%) ⬆️
lib/internal/modules/cjs/loader.js 98.05% <100.00%> (+<0.01%) ⬆️
lib/internal/process/pre_execution.js 97.93% <100.00%> (+0.14%) ⬆️
lib/internal/vfs/dir.js 100.00% <100.00%> (ø)
lib/internal/vfs/errors.js 100.00% <100.00%> (ø)
lib/internal/vfs/stats.js 100.00% <100.00%> (ø)
lib/vfs.js 100.00% <100.00%> (ø)
src/node_builtins.cc 76.37% <100.00%> (+0.23%) ⬆️
src/node_options.cc 76.67% <100.00%> (+0.02%) ⬆️
src/node_options.h 98.00% <100.00%> (+<0.01%) ⬆️
... and 8 more

... and 36 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollina mcollina added help wanted Issues that need assistance from volunteers or PRs that need help to proceed. request-ci Add this label to start a Jenkins CI on a PR. labels May 6, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 6, 2026
@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label May 7, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 7, 2026
@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Comment thread doc/api/vfs.md Outdated
Comment thread doc/api/vfs.md Outdated
Comment thread lib/internal/bootstrap/realm.js Outdated
@aduh95
Copy link
Copy Markdown
Contributor

aduh95 commented May 7, 2026

Can you add a test for #63158?

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label May 11, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 11, 2026
mcollina and others added 23 commits May 23, 2026 14:47
Adds an --experimental-vfs runtime option that gates loading of the
node:vfs builtin module, matching the pattern used by node:quic and
node:stream/iter. Without the flag, require('node:vfs') / import
'node:vfs' throw ERR_UNKNOWN_BUILTIN_MODULE.

All VFS test files are updated to pass --experimental-vfs.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Fixes the JS lint warnings on the VFS subsystem and tests:
primordial alphabetical ordering, em-dash → hyphen, error-codes
multiline destructuring, removal of unused JSDoc @returns, and the
test-side mustSucceed / async-iife-no-unused-result rules.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
internal/vfs/fd.js doesn't require any internal/vfs/* modules so
there's no circular dependency to defer. Replace the getLazy wrapper
with a direct import.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Removes mount/unmount, virtualCwd, overlay mode, fs/module
integration sections, SEA usage, and worker-thread guidance
since none of that ships in this PR.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Adds 'vfs' to the C++ cannot_be_required list so existing tests
(test-code-cache, test-process-get-builtin, test-require-resolve)
treat it like other flagged experimental modules. Adds the flag to
doc/node.1 and reorders the entry in doc/api/cli.md.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
The first block used persistent: false plus setTimeout to trigger the
write. The watcher's poll timer was unref'd, so on slow runners the
write timer could fire before the first poll and the change event
would be missed. Use the same await-once pattern as the other blocks
in the file with a content-length change so the size-based stat-change
detector always fires.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Mirrors the recently-merged handling for node:ffi (see nodejs#63158): when
--experimental-vfs is not set, node:vfs is filtered out of the public
Module.builtinModules list. Adds matching coverage in test-vfs-flag.

Refs: nodejs#63158

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
The getter exposed the raw real-fs file descriptor of a virtual file
handle, which is a leaky abstraction: it lets user code bypass the
VFS layer. The only consumer was a test that closed the real fd to
trigger the EBADF rejection paths in the async fd ops; those branches
are defensive and unreachable from the public API, so the test block
is dropped along with the getter.

Assisted-by: Claude-Opus4.7
Signed-off-by: Matteo Collina <hello@matteocollina.com>
* file_handle: import SymbolDispose from primordials so the
  SymbolDispose prototype binding doesn't throw at load time.
* providers/real: import getValidatedPath and setOwnProperty that
  the code-review rewrite of the rootPath validator referenced
  without importing.
* test-vfs-real-provider: getValidatedPath rejects non-strings
  with ERR_INVALID_ARG_TYPE (not ERR_INVALID_ARG_VALUE) and now
  accepts the empty string, so the assertions are updated and the
  empty-string case dropped.
* test-vfs-memory-provider-dynamic: the object-literal rewrite
  left a stray ');' instead of '}', which crashed the parser.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Tracks the suggestion from review to delegate the async fd ops in
RealFileHandle to FileHandle methods instead of manually wrapping
fs.read/write/fstat/ftruncate/close in Promises. The refactor is
blocked on a way to wrap an existing numeric fd in a FileHandle so a
sync-opened handle can share one underlying handle for async ops.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
RealFSProvider.realpath fed the resolved path through StringPrototype
startsWith against the stored rootPath plus a separator. On Windows
fs.realpathSync is a JS implementation that preserves case while
fs.promises.realpath calls the native binding, which canonicalizes the
drive letter (and potentially other components). The async path then
fails the startsWith check and rejects with EACCES, even for valid
paths inside the root.

Use path.relative to test containment instead, which treats the
comparison as case-insensitive on Windows. Both realpathSync and
realpath now share the same helper.

Fixes failure in test-vfs-real-provider-symlinks on
test-binary-windows-js-suites.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Matteo Collina <matteo.collina@gmail.com>
* test-vfs-stream-errors: drop the unused `assert` import.
* test-vfs-streams: capitalize a lowercase comment to satisfy
  capitalized-comments.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
spawnSyncAndAssert handles the expected stderr/status, so the
top-level `assert` is no longer referenced.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
The currentPath segment used for symlink resolution and for the
#ensurePopulated call below was being computed twice in the loop
body. Compute it once at the top of the iteration and reuse it in
both branches.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label May 23, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 23, 2026
@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lib / src Issues and PRs related to general changes in the lib or src directory. semver-minor PRs that contain new features and should be released in the next minor version.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants