Switch from Jest to Node.js built-in test runner

[shadowspawn]

Problem

I do like writing tests using Jest, but the ESM support is still experimental (relies on a Node.js experimental flag). I hit this quickly when looking at converting Commander to esm-only.

• https://jestjs.io/docs/ecmascript-modules

Solution

The radical approach in this PR is to switch to native Node.js test runner! Zero dependencies compared with large Jest install. Fast test runner. Potential downsides are simpler API patterns and switch to node:assert.

The Node.js test runner does have test coverage support but still experimental. Given we do not currently enforce coverage percentage, I don't think that is a blocker.

• https://nodejs.org/docs/latest-v22.x/api/test.html#collecting-code-coverage

Another alternative is we could switch to vitest which has a similar API to Jest and natively supports ESM. I have not used vitest, but seems to be fast and well regarded.

ToDo

• confirm does work with ESM before going too far [only a simple check, but fine]
• revisit runtime test of exports from Typescript, not achieving anything? (ts-imports.test.js)
• compare total number of tests before/after in case some tests not recognised by runner after conversion (paranoia)
• turn back on no-undef eslint rule (when finish conversion)
• delete .example-conversion.test.js (when finish conversion)
• reenable unit tests (when finish conversion)

ChangeLog

• (dev) switch from Jest to node:test test runner

[shadowspawn]

Example test run including coverage using node v24.

[shadowspawn]

I have converted all the tests starting with a. The node:assert library does have some useful convenience routines so not too basic, like assert.throws(). Mocking is similar.

[cjihrig]

hitting problems with glob expansion in node 20

Just a heads up in case you're running into problems - there is no globbing in Node 20.

[shadowspawn]

Thanks @cjihrig for clarifying. I was doing some quick hacking to reduce build failures with current state of files and hadn't nailed down the difference in behaviour with node version.

[cjihrig]

So, Node 20 goes EOL in a couple months. In the interim, I've seen people do a few different things (assuming you want to use node:test):

1. Drop Node 20 - not advocating for this, but it's an option.
2. Create a simple glob script.
3. Use a lightweight wrapper like borp that papers over the differences between versions until Node 20 goes away.

[shadowspawn]

A difference to Jest. The default test progress shows the test names but not the test filename. That is currently (somewhat deliberately) left up to user or a custom reporter.

I was initially a little dismayed. However, when using default spec reporter the errors do show the filename. So the success progress is less rich that we are used to, but the failure output has the context when it really matters.

Not planning to rework test names or wrap files in describe for now. See how it feels with default support. The dot reporter is very compact for seeing if any problems (but does not show the filename for failures).

---

Edit: I noticed some of the files do already have describe wrappers which give tests names context (without filename), so maybe will go through and be disciplined about combination of describe and test being globally unique and human readable.

commander.js/tests/help.visibleCommands.test.js

Line 6 in 0692be5

describe('visibleCommands', () => {

[shadowspawn]

Main conversion done, and in particular can run node --test now without errors.

Jest counts on release/15.x branch:

Test Suites: 110 passed, 110 total
Tests:       1369 passed, 1369 total

node:test counts on PR branch:

ℹ tests 1349
ℹ suites 156
ℹ pass 1349
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0

There were 8 tests in ts-imports.test.ts. Will have a look for the other 12 difference in tests count. Might be a loop moved inside a test, but intended pattern for porting was test.each() turned into a describe wrapper for a loop calling test on each test case.

Edit: found missing true/false loop in command.allowExcessArguments.test.js for 10 test difference.

[shadowspawn]

For reference, this was the prompt and example file I was using with agents

Convert the open file from using Jest to node:test. Refer to tests/.exampe-conversion.test.js for examples and convention

const commander = require('../');
const { test, describe } = require('node:test');
const assert = require('node:assert/strict'); // strict assertion mode

// Key patterns for conversion from Jest to node:test
// - Jest comments are for example file only and original code does not need to be retained in comments
// - use assert.equal for primitives, and assert.deepEqual for objects
// - use assert.equal rather than assert.strictEqual (already in strict assertion mode)
// - use assert.deepEqual rather than assert.strictDeepEqual (already in strict assertion mode)
// - convert Jest describe to node:test describe
// - convert jest.fn() to use node:test t.mock.fn()
// - convert expect().toHaveBeenCalled() to assert.equal(callCount, 1)

test('when arguments includes -- then stop processing options', () => {
  const program = new commander.Command();
  program
    .option('-f, --foo', 'add some foo')
    .option('-b, --bar', 'add some bar')
    .argument('[args...]');
  program.parse(['node', 'test', '--foo', '--', '--bar', 'baz']);
  // More than one assert, ported from legacy test
  const opts = program.opts();
  // Jest was: expect(opts.foo).toBe(true);
  assert.equal(opts.foo, true);
  // Jest was: expect(opts.bar).toBeUndefined();
  assert.equal(opts.bar, undefined);
  // Jest was: expect(program.args).toEqual(['--bar', 'baz']);
  assert.deepEqual(program.args, ['--bar', 'baz']);
});

describe('variadic argument', () => {
  test('when no extra arguments specified for program then variadic arg is empty array', (t) => {
    // Jest was: const actionMock = jest.fn();
    const actionMock = t.mock.fn();
    const program = new commander.Command();
    program.arguments('<id> [variadicArg...]').action(actionMock);

    program.parse(['node', 'test', 'id']);

    // Jest was: expect(actionMock).toHaveBeenCalledWith('id', [], program.opts(), program);
    const callArgs = actionMock.mock.calls[0].arguments;
    assert.equal(callArgs[0], 'id');
    assert.deepEqual(callArgs[1], []);
    assert.equal(callArgs[2], program.opts()); // check same object
    assert.equal(callArgs[3], program); // check same object
  });
});

test('when use alias then action handler called', (t) => {
  const program = new commander.Command();
  const actionMock = t.mock.fn();
  program.command('list').alias('ls').action(actionMock);
  program.parse(['ls'], { from: 'user' });
  // Jest was: expect(actionMock).toHaveBeenCalled();
  assert.equal(actionMock.mock.callCount(), 1);
});

// Jest was:
// test.each([
//   { flags: '-s' }, // single short
//   { flags: '--long' }, // single long
// ])('when construct Option with flags %p then do not throw', ({ flags }) => {
//   expect(() => {
//     new Option(flags);
//   }).not.toThrow();
// });
//
// Wrap loop in a describe block to group related tests.
describe('when construct Option with various flags then do not throw', () => {
  const flagsList = [
    { flags: '-s' }, // single short
    { flags: '--long' }, // single long
  ];
  for (const { flags } of flagsList) {
    test(`when construct Option with flags ${flags} then do not throw`, () => {
      assert.doesNotThrow(() => {
        new commander.Option(flags);
      });
    });
  }
});

test('when command name conflicts with existing alias then throw', () => {
  // Jest was:
  // expect(() => {
  //   const program = new commander.Command();
  //   program.command('one').alias('1');
  //   program.command('1');
  // }).toThrow('cannot add command');
  assert.throws(
    () => {
      const program = new commander.Command();
      program.command('one').alias('1');
      program.command('1');
    },
    { message: /cannot add command/ },
  );
});

[shadowspawn]

Future work, but does not need to be part of this PR unless preferred now:

• add describe() wrappers and revisit test names so have full context in testing output without filenames (style difference between Tap/Ava/node:test vs Jest)
• use .configureOutput() instead of mock spies for suppressing stdout/stderr
  • (I wonder whether a public program.configureForTests() would be useful, but will start with private utility routines.)

[shadowspawn]

Mega merge! Feel free to ask style questions. I used AI agents for lots of conversion and while I did review each file, there may be some weirdness I ignored, or some patterns we would like to use consistently.

[shadowspawn]

Added createTestCommand() configured for silently throwing exception. Using built-in support for suppressing output instead of mocks on process.stdout.write and process.stderr.write.

(One of ideas in #2463 (comment))