Skip to content

Commit 977cc3d

Browse files
committed
fs: add virtual file system support
Add a read-only virtual file system (VFS) that can be mounted at a specific path prefix, enabling standard fs APIs to work transparently with in-memory files. Key features: - fs.createVirtual() to create VFS instances - Support for files, directories, and symbolic links - Full async/sync/promise API support (readFile, stat, readdir, etc.) - File descriptor operations (open, read, close) - createReadStream() support - fs.glob() integration - CJS require() and ESM import() support via module hooks - Virtual process.chdir() for relative path resolution - SEA integration via sea.getVfs() and sea.hasAssets() - Test runner mock.fs() for file system mocking The VFS is read-only by design and uses virtual file descriptors (10000+) to avoid conflicts with real file descriptors.
1 parent 637bda0 commit 977cc3d

30 files changed

+6824
-0
lines changed

doc/api/fs.md

Lines changed: 590 additions & 0 deletions
Large diffs are not rendered by default.

doc/api/single-executable-applications.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -238,6 +238,94 @@ const raw = getRawAsset('a.jpg');
238238
See documentation of the [`sea.getAsset()`][], [`sea.getAssetAsBlob()`][],
239239
[`sea.getRawAsset()`][] and [`sea.getAssetKeys()`][] APIs for more information.
240240
241+
### Virtual File System (VFS) for assets
242+
243+
> Stability: 1 - Experimental
244+
245+
Instead of using the `node:sea` API to access individual assets, you can use
246+
the Virtual File System (VFS) to access bundled assets through standard `fs`
247+
APIs. The VFS automatically populates itself with all assets defined in the
248+
SEA configuration and mounts them at a virtual path (default: `/sea`).
249+
250+
To use the VFS with SEA:
251+
252+
```cjs
253+
const fs = require('node:fs');
254+
const sea = require('node:sea');
255+
256+
// Check if SEA assets are available
257+
if (sea.hasAssets()) {
258+
// Initialize and mount the SEA VFS
259+
const vfs = sea.getVfs();
260+
261+
// Now you can use standard fs APIs to read bundled assets
262+
const config = JSON.parse(fs.readFileSync('/sea/config.json', 'utf8'));
263+
const data = fs.readFileSync('/sea/data/file.txt');
264+
265+
// Directory operations work too
266+
const files = fs.readdirSync('/sea/assets');
267+
268+
// Check if a bundled file exists
269+
if (fs.existsSync('/sea/optional.json')) {
270+
// ...
271+
}
272+
}
273+
```
274+
275+
The VFS supports the following `fs` operations on bundled assets:
276+
277+
* `readFileSync()` / `readFile()` / `promises.readFile()`
278+
* `statSync()` / `stat()` / `promises.stat()`
279+
* `lstatSync()` / `lstat()` / `promises.lstat()`
280+
* `readdirSync()` / `readdir()` / `promises.readdir()`
281+
* `existsSync()`
282+
* `realpathSync()` / `realpath()` / `promises.realpath()`
283+
* `accessSync()` / `access()` / `promises.access()`
284+
* `openSync()` / `open()` - for reading
285+
* `createReadStream()`
286+
287+
#### Loading modules from VFS in SEA
288+
289+
The default `require()` function in a SEA only supports loading Node.js
290+
built-in modules. To load JavaScript modules bundled as assets, you must use
291+
[`module.createRequire()`][]:
292+
293+
```cjs
294+
const { createRequire } = require('node:module');
295+
const sea = require('node:sea');
296+
297+
// Initialize VFS
298+
sea.getVfs();
299+
300+
// Create a require function that works with VFS
301+
const seaRequire = createRequire('/sea/');
302+
303+
// Now you can require bundled modules
304+
const myModule = seaRequire('/sea/lib/mymodule.js');
305+
const utils = seaRequire('/sea/utils/helpers.js');
306+
```
307+
308+
This is necessary because SEA uses a special embedder require that doesn't go
309+
through the standard module resolution hooks that VFS registers.
310+
311+
#### Custom mount prefix
312+
313+
By default, the VFS is mounted at `/sea`. You can specify a custom prefix
314+
when initializing the VFS:
315+
316+
```cjs
317+
const fs = require('node:fs');
318+
const sea = require('node:sea');
319+
320+
const vfs = sea.getSeaVfs({ prefix: '/app' });
321+
322+
// Assets are now accessible under /app
323+
const config = fs.readFileSync('/app/config.json', 'utf8');
324+
```
325+
326+
Note: `sea.getVfs()` returns a singleton. The `prefix` option is only used
327+
on the first call; subsequent calls return the same cached instance.
328+
241329
### Startup snapshot support
242330
243331
The `useSnapshot` field can be used to enable startup snapshot support. In this
@@ -553,6 +641,7 @@ to help us document them.
553641
[Mach-O]: https://en.wikipedia.org/wiki/Mach-O
554642
[PE]: https://en.wikipedia.org/wiki/Portable_Executable
555643
[Windows SDK]: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
644+
[`module.createRequire()`]: module.md#modulecreaterequirefilename
556645
[`process.execPath`]: process.md#processexecpath
557646
[`require()`]: modules.md#requireid
558647
[`require.main`]: modules.md#accessing-the-main-module

doc/api/test.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2334,6 +2334,94 @@ test('mocks a counting function', (t) => {
23342334
});
23352335
```
23362336

2337+
### `mock.fs([options])`
2338+
2339+
<!-- YAML
2340+
added: REPLACEME
2341+
-->
2342+
2343+
> Stability: 1.0 - Early development
2344+
2345+
* `options` {Object} Optional configuration options for the mock file system.
2346+
The following properties are supported:
2347+
* `prefix` {string} The mount point prefix for the virtual file system.
2348+
**Default:** `'/mock'`.
2349+
* `files` {Object} An optional object where keys are file paths (relative to
2350+
the VFS root) and values are the file contents. Contents can be strings,
2351+
Buffers, or functions that return strings/Buffers.
2352+
* Returns: {MockFSContext} An object that can be used to manage the mock file
2353+
system.
2354+
2355+
This function creates a mock file system using the Virtual File System (VFS).
2356+
The mock file system is automatically cleaned up when the test completes.
2357+
2358+
## Class: `MockFSContext`
2359+
2360+
The `MockFSContext` object is returned by `mock.fs()` and provides the
2361+
following methods and properties:
2362+
2363+
* `vfs` {VirtualFileSystem} The underlying VFS instance.
2364+
* `prefix` {string} The mount prefix.
2365+
* `addFile(path, content)` Adds a file to the mock file system.
2366+
* `addDirectory(path[, populate])` Adds a directory to the mock file system.
2367+
* `existsSync(path)` Checks if a path exists (path is relative to prefix).
2368+
* `restore()` Manually restores the file system to its original state.
2369+
2370+
The following example demonstrates how to create a mock file system for testing:
2371+
2372+
```js
2373+
const { test } = require('node:test');
2374+
const assert = require('node:assert');
2375+
const fs = require('node:fs');
2376+
2377+
test('reads configuration from mock file', (t) => {
2378+
const mockFs = t.mock.fs({
2379+
prefix: '/app',
2380+
files: {
2381+
'/config.json': JSON.stringify({ debug: true }),
2382+
'/data/users.txt': 'user1\nuser2\nuser3',
2383+
},
2384+
});
2385+
2386+
// Files are accessible via standard fs APIs
2387+
const config = JSON.parse(fs.readFileSync('/app/config.json', 'utf8'));
2388+
assert.strictEqual(config.debug, true);
2389+
2390+
// Check file existence
2391+
assert.strictEqual(fs.existsSync('/app/config.json'), true);
2392+
assert.strictEqual(fs.existsSync('/app/missing.txt'), false);
2393+
2394+
// Use mockFs.existsSync for paths relative to prefix
2395+
assert.strictEqual(mockFs.existsSync('/config.json'), true);
2396+
});
2397+
2398+
test('supports dynamic file content', (t) => {
2399+
let counter = 0;
2400+
const mockFs = t.mock.fs({ prefix: '/dynamic' });
2401+
2402+
mockFs.addFile('/counter.txt', () => {
2403+
counter++;
2404+
return String(counter);
2405+
});
2406+
2407+
// Each read calls the function
2408+
assert.strictEqual(fs.readFileSync('/dynamic/counter.txt', 'utf8'), '1');
2409+
assert.strictEqual(fs.readFileSync('/dynamic/counter.txt', 'utf8'), '2');
2410+
});
2411+
2412+
test('supports require from mock files', (t) => {
2413+
t.mock.fs({
2414+
prefix: '/modules',
2415+
files: {
2416+
'/math.js': 'module.exports = { add: (a, b) => a + b };',
2417+
},
2418+
});
2419+
2420+
const math = require('/modules/math.js');
2421+
assert.strictEqual(math.add(2, 3), 5);
2422+
});
2423+
```
2424+
23372425
### `mock.getter(object, methodName[, implementation][, options])`
23382426

23392427
<!-- YAML

lib/fs.js

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3206,6 +3206,19 @@ function globSync(pattern, options) {
32063206
return new Glob(pattern, options).globSync();
32073207
}
32083208

3209+
const lazyVfs = getLazy(() => require('internal/vfs/virtual_fs').VirtualFileSystem);
3210+
3211+
/**
3212+
* Creates a new virtual file system instance.
3213+
* @param {object} [options] Configuration options
3214+
* @param {boolean} [options.fallthrough] Whether to fall through to real fs on miss
3215+
* @param {boolean} [options.moduleHooks] Whether to enable require/import hooks
3216+
* @returns {VirtualFileSystem}
3217+
*/
3218+
function createVirtual(options) {
3219+
const VirtualFileSystem = lazyVfs();
3220+
return new VirtualFileSystem(options);
3221+
}
32093222

32103223
module.exports = fs = {
32113224
appendFile,
@@ -3223,6 +3236,7 @@ module.exports = fs = {
32233236
cp,
32243237
cpSync,
32253238
createReadStream,
3239+
createVirtual,
32263240
createWriteStream,
32273241
exists,
32283242
existsSync,

lib/internal/test_runner/mock/mock.js

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,10 @@ const {
5050
const { MockTimers } = require('internal/test_runner/mock/mock_timers');
5151
const { Module } = require('internal/modules/cjs/loader');
5252
const { _load, _nodeModulePaths, _resolveFilename, isBuiltin } = Module;
53+
const { join } = require('path');
54+
55+
// Lazy-load VirtualFileSystem to avoid loading VFS code if fs mocking is not used
56+
let VirtualFileSystem;
5357
function kDefaultFunction() {}
5458
const enableModuleMocking = getOptionValue('--experimental-test-module-mocks');
5559
const kSupportedFormats = [
@@ -402,6 +406,72 @@ class MockPropertyContext {
402406

403407
const { restore: restoreProperty } = MockPropertyContext.prototype;
404408

409+
/**
410+
* Context for mocking the file system using VFS.
411+
*/
412+
class MockFSContext {
413+
#vfs;
414+
#prefix;
415+
416+
constructor(vfs, prefix) {
417+
this.#vfs = vfs;
418+
this.#prefix = prefix;
419+
}
420+
421+
/**
422+
* Gets the underlying VirtualFileSystem instance.
423+
* @returns {VirtualFileSystem}
424+
*/
425+
get vfs() {
426+
return this.#vfs;
427+
}
428+
429+
/**
430+
* Gets the mount prefix for the mock file system.
431+
* @returns {string}
432+
*/
433+
get prefix() {
434+
return this.#prefix;
435+
}
436+
437+
/**
438+
* Adds a file to the mock file system.
439+
* @param {string} path - The path of the file.
440+
* @param {string|Buffer|Function} content - The file content.
441+
*/
442+
addFile(path, content) {
443+
this.#vfs.addFile(path, content);
444+
}
445+
446+
/**
447+
* Adds a directory to the mock file system.
448+
* @param {string} path - The path of the directory.
449+
* @param {Function} [populate] - Optional callback to populate the directory.
450+
*/
451+
addDirectory(path, populate) {
452+
this.#vfs.addDirectory(path, populate);
453+
}
454+
455+
/**
456+
* Checks if a path exists in the mock file system.
457+
* @param {string} path - The path to check (relative to prefix).
458+
* @returns {boolean}
459+
*/
460+
existsSync(path) {
461+
const fullPath = join(this.#prefix, path);
462+
return this.#vfs.existsSync(fullPath);
463+
}
464+
465+
/**
466+
* Restores the file system to its original state.
467+
*/
468+
restore() {
469+
this.#vfs.unmount();
470+
}
471+
}
472+
473+
const { restore: restoreFS } = MockFSContext.prototype;
474+
405475
class MockTracker {
406476
#mocks = [];
407477
#timers;
@@ -725,6 +795,45 @@ class MockTracker {
725795
});
726796
}
727797

798+
/**
799+
* Creates a mock file system using VFS.
800+
* @param {object} [options] - Options for the mock file system.
801+
* @param {string} [options.prefix] - The mount prefix for the VFS.
802+
* @param {object} [options.files] - Initial files to add (path: content pairs).
803+
* @returns {MockFSContext} The mock file system context.
804+
*/
805+
fs(options = kEmptyObject) {
806+
validateObject(options, 'options');
807+
const { prefix = '/mock', files } = options;
808+
if (files !== undefined) {
809+
validateObject(files, 'options.files');
810+
}
811+
812+
VirtualFileSystem ??= require('internal/vfs/virtual_fs').VirtualFileSystem;
813+
const vfs = new VirtualFileSystem({ __proto__: null, moduleHooks: true });
814+
815+
// Add initial files if provided
816+
if (files) {
817+
const paths = ObjectKeys(files);
818+
for (let i = 0; i < paths.length; i++) {
819+
const path = paths[i];
820+
vfs.addFile(path, files[path]);
821+
}
822+
}
823+
824+
// Mount the VFS at the specified prefix
825+
vfs.mount(prefix);
826+
827+
const ctx = new MockFSContext(vfs, prefix);
828+
ArrayPrototypePush(this.#mocks, {
829+
__proto__: null,
830+
ctx,
831+
restore: restoreFS,
832+
});
833+
834+
return ctx;
835+
}
836+
728837
/**
729838
* Resets the mock tracker, restoring all mocks and clearing timers.
730839
*/

0 commit comments

Comments
 (0)