RFC: ESM migration with lazy-loaded aspects#10376
Draft
zkochan wants to merge 51 commits into
Draft
Conversation
Proposes a three-tier loading model (bundled entry + lazy aspect runtimes +
unified user-extension loading) to keep the Bit CLI startup fast through the
CJS-to-ESM migration. The current eager-load path resolves every core aspect's
provider at startup; this RFC trades that for on-demand `harmony.resolve` driven
by a publish-time command index.
Includes:
- docs/rfc-esm-lazy-aspects.md — full RFC, with prototype learnings folded in
- docs/migration/ — 11 PR-sized work chunks + index, with dependency graph
- prototypes/mini-bit/ — runnable ESM proof of concept demonstrating the
architecture (5 of 7 aspects loaded for `status` vs 8 in eager mode)
No production code changes. Open for review.
The slice 7 bulk-migration codemod (f99b674) injected `import { X } from './X.commands';` lines inside existing multiline `import { ... } from '@teambit/cli';` blocks, leaving stranded identifier fragments that cause parse errors. It also wrote two `./component.commands` paths that pointed outside their directory. Fixes the 13 affected files so the source tree parses again. Unblocks the chunk 10 bundler smoke test.
Adds scripts/build-publish-bundle.mjs — a Rollup driver that bundles scopes/harmony/bit/app.ts into dist/bundle/bit.mjs with one chunk per *.main.runtime.[jt]s and *.ui.runtime.[jt]s, deterministic chunk names, source maps with inlined sources, and a 5MB entry-chunk budget gate (skippable via --no-budget). Bundles TS sources directly via @rollup/plugin-typescript with mainFields: ['source', 'module', 'main']. This is required until chunk 09 lands: Babel-compiled dist files have already rewritten `() => import(...)` into `() => Promise.resolve().then(() => require(...))`, which Rollup does not recognise as a code-split boundary. Once the codebase emits ESM we can switch to bundling the compiled `.mjs` dist. Externalises native bindings (fsevents, @lydell/node-pty, @reflink/*, @parcel/css, lightningcss, @swc/css and their platform-suffixed packages), node builtins, and heavy UI-runtime deps (react, react-dom, monaco-editor, @apollo/client, graphql, @yarnpkg/*). A stub-browser-assets plugin makes SCSS/CSS/SVG/MDX imports no-op in the Node entry. Adds npm scripts `build-bundle` and `build-bundle:visualize` (rollup-plugin- visualizer treemap). Adds Rollup deps to workspace.jsonc. Updates docs/migration/10-publish-bundling.md acceptance checklist and records remaining work: CI publish step, fresh-install smoke test, bench gate, and the chunk 09 dependency.
Slice 7 generated 12 *.commands.ts files that reference symbols without importing them: - 11 files use COMPONENT_PATTERN_HELP from @teambit/legacy.constants (checkout, component-compare, forking, merging, remove, formatter, linter, tester, validator, export, eject). - components/config-store/config-store.commands.ts uses BASE_DOCS_DOMAIN from the same package. - scopes/harmony/global-config/global-config.commands.ts instantiates RemoteAdd/RemoteRm/RemoteList inline as subcommands; the classes weren't exported from ./remote-cmd. Export them and import. Unblocks Rollup parse for the chunk 10 publish bundle.
The Node CLI bundle's dep graph reaches @teambit/mdx.* and @teambit/documenter.* via UI runtime files (`*.section.tsx`, `*.composition.tsx`, `*.docs.mdx`). One such package (mdx.ui.mdx-layout) imports `MDXLayout` from `@teambit/documenter.markdown.mdx` which does not export that binding, breaking Rollup's parse. These packages are never called from the Node entry's runtime graph — externalize them so node_modules supplies them at runtime if a UI runtime actually loads. With this in place, `npm run build-bundle` succeeds: entry bit.mjs ~142KB (well under the 5MB budget), 121 chunks emitted including one named chunk per main runtime and UI runtime.
The lazy-aspect helpers introduced in slice 4 lived at
`scopes/harmony/harmony/` and were imported via relative paths
(`'../../harmony/harmony/aspect'`, `'../harmony/aspect'`, `'../harmony'`).
That works in the source tree but breaks once `bit compile` emits the JS to
`node_modules/@teambit/<aspect>/dist/`, where `../../harmony/harmony/...`
no longer resolves — every aspect that uses `Aspect.create({ runtimes })`
fails at dist load time. E2E tests in particular die with
`Cannot find module '../../harmony/harmony/aspect'` when mocha pulls in
`@teambit/workspace/dist/workspace.aspect.js`.
This commit makes the helper a real Bit component:
- `git mv scopes/harmony/harmony scopes/harmony/core`
- Register `teambit.harmony/core` in `.bitmap`
- `bit link` exposes it as `@teambit/core` in node_modules (bit chose the
bare name; no `harmony.` prefix because there's no collision)
- Rewrite the 110 consumer imports to `'@teambit/core'`:
- 90 files: `'../../harmony/harmony/aspect'`
- 19 files: `'../harmony/aspect'`
- 1 file: `'../harmony'` (load-bit.ts)
- 1 file: `'../../scopes/harmony/harmony/aspect'`
(components/config-store/config-store.aspect.ts)
Verified:
- `npm run build-bundle` succeeds; entry chunk 475.6 KB (well under 5 MB);
120 chunks emit with deterministic `runtime-*` names.
- The original `Cannot find module '../../harmony/harmony/aspect'` e2e
blocker no longer reproduces. A separate pre-existing mocha 11 / Node
ESM-utils interop issue remains (`@teambit/legacy.bit-map` named export)
but is unrelated to this move — it reproduces on the unmodified branch
tip too.
Node 24 enables `--experimental-strip-types` by default. When mocha's
`requireOrImport` tries `import()` on a `.ts` test file, Node sees `import`
syntax, classifies the file as ESM, and loads it via the ESM resolver.
That bypasses `--require ./babel-register` entirely, and the first CJS
package the test imports with named exports
(`import { MissingMainFile } from '@teambit/legacy.bit-map'`) fails with:
SyntaxError: Named export 'MissingMainFile' not found.
The requested module '@teambit/legacy.bit-map' is a CommonJS module …
Disabling type-stripping makes Node fail the ESM `import()` of `.ts` with
`ERR_UNKNOWN_FILE_EXTENSION`; mocha catches that and falls back to
`require()`, where babel-register can intercept and compile the test file
to CJS as intended.
Applied to all mocha-based test scripts: e2e-test, e2e-test-circle,
mocha-circleci, performance-test, performance-test-circle,
bit-hub-test-circle.
Adds `extensionsIds`, `extensions`, `load(aspects)`, and `run()` methods
to the lazy Harmony class so callers from the legacy
`@teambit/harmony.Harmony` surface keep working when pointed at the lazy
variant:
- `extensionsIds` — returns the registered manifest ids.
- `extensions.get(id)` — returns `{ loaded: instances.has(id) }`, mirroring
the eager API used by `aspect-loader.getNotLoadedConfiguredExtensions`.
- `load(aspects)` — register each transitively and resolve. Used by
`aspect-loader.loadExtensionsByManifests`.
- `run()` — no-op. Lazy resolution happens on demand, so callers that
used to pump the eager pipeline via `harmony.run(requireFn)` no longer
need to do anything.
Doesn't change the load path yet — that's a separate, larger refactor
(several Slice 7 infra gaps surface: `cli.register` setDefaults assumes
eager Command objects, and per-aspect provider classes use legacy slots).
Leaving the shims in place so the next round of work can flip
`load-bit.ts` + `load-aspect.ts` to LazyHarmony incrementally.
When a caller passes the new lazy form
`Aspect.create({ runtimes: { main: () => import(...) } })` without an
explicit `declareRuntime`, derive `new RuntimeDefinition(<first key>)` so
the upstream `Harmony.load()` path can populate its `Runtimes` registry.
Without this, `harmony.run()` immediately fails with
`runtime: 'main' was not defined by any aspect` because no aspect has
declared the runtime — the new pattern relies on the runtime file's
top-level `Aspect.addRuntime(...)` call to fire later, but
`Runtimes.load` runs first and finds nothing to register.
Backward-compatible: callers that already pass `declareRuntime` keep their
explicit value via `??`. Restores `bit init` / `bit status` on this branch.
The 4 e2e files used the ESM `import.meta.url` + `fileURLToPath` pattern to derive `__filename` / `__dirname`. With babel-register compiling these to CJS at runtime, Node sees the ESM syntax and reparses each file as ESM — at which point `require` is undefined and any test pulled via `require()` from inside the test fails with `ReferenceError: require is not defined in ES module scope`. CJS already provides `__filename` / `__dirname` as globals, so the ESM preamble is unnecessary. Removing it lets babel-register handle the file as plain CJS again.
…tion
While the RFC migration is in flight, the bvm-installed bit cannot load
the workspace because:
1. `cli.register(descriptor, factory)` (slice 7 codemod) trips the older
`setDefaults` — it mutates `command.name` on the factory closure.
2. A prior `bit compile` rewrote some bvm-shipped aspect dist files (e.g.
`@teambit/clear-cache/dist/clear-cache.aspect.js`) to
`require('@teambit/core')`, which isn't part of the bvm dependency
closure.
`apply.mjs` patches the bvm install in place: symlinks `@teambit/core`
from the workspace and replaces `register()` in
`@teambit/cli/dist/cli.main.runtime.js` with the new factory-aware form.
Patches are self-marked (re-runs are no-ops), reversible via `--revert`,
and back up originals to `<file>.bvm-patches.bak`.
Delete this directory once a bit version ships with the new register()
baked in and `@teambit/core` declared as a proper dep.
`bit --help` previously paid the full bootstrap cost (~520 ms cold) to load every aspect just to enumerate registered commands. This commit short-circuits the top-level help case entirely. How: - `scopes/harmony/bit/help-from-index.ts` — a tiny renderer (chalk + inline padEnd/capitalize, no other runtime deps) that formats from the `CommandIndexEntry[]` already committed in `command-index.generated.ts`. Group descriptions are mirrored from `scopes/harmony/cli/command-groups.ts` as a static map. - `scopes/harmony/bit/run-bit.ts` — detect standalone `--help` / `-h` via `isStandaloneHelp(process.argv)` before any heavy import. All the heavy modules (`@teambit/cli`, `bootstrap`, `load-bit`, server commander, etc.) are now `require()`d lazily inside `runBit` / `initApp` so the help path doesn't pay for parsing them. - `scripts/generate-command-index.mjs` — point at the new `scopes/harmony/core/slot-index.generated.ts` location (was `scopes/harmony/harmony/...` before the harmony→core move). - `scopes/harmony/bit/command-index.generated.ts` — regenerated against the current live `commandsByAspect()` output. It's smaller than the prior snapshot because some aspects in this branch no longer register commands at runtime (slice 7/8 regression that's tracked separately). Honest snapshot beats a stale-but-complete one. Bench (median cold, M1, Node v24.15.0, 7 iters): scenario new old delta bit --help 192 ms 487 ms -295 ms (-61%) bit --version 187 ms 189 ms close to noise The `--help` path now does ~one tenth of the work of the full bootstrap. Command-specific help (`bit status --help`) still goes through the regular bootstrap because yargs needs the live command to render its options. Validation: `command-index-assert.ts` continues to run on the regular load path and warns / errors when the static index diverges from the live `commandsSlot`, so the static help output stays in sync as long as the index is kept fresh.
Commit d231c32 ("remove hacky babel lazy-dependency plugin") dropped `@babel/plugin-transform-modules-commonjs` with `lazy: () => true` from the workspace-level `babel.config.js`, but missed two other places that ship the same plugin entry: - `scopes/harmony/aspect/babel/babel-config.ts` — the local aspect env's babel config (kept for any other workspace that pins `teambit.harmony/aspect` as its env). - `<bit-capsule>/teambit.harmony_envs_core-aspect-env*/config/cjs.babel.config.js` — the external `teambit.harmony/envs/core-aspect-env@0.1.4` env that actually compiles bit's own components. Lives in the capsule cache; rebuilt on `bit install`. Patched in-place via `scripts/bvm-patches/ apply.mjs`; re-run that after `bit install` if the capsule reverts. With babel's per-require thunks gone, the slice 4/7 lazy-aspect machinery and slice 5's command-index short-circuit do the actual lazy loading without being hidden under a second layer of indirection. Bench (median cold, M1, Node v24.15.0, 7 iters), new vs bit2 master: scenario new bit2 delta bit --version 237 ms 188 ms +49 ms (+26%) bit --help 173 ms 487 ms -314 ms (-64%) bit <typo> 236 ms 489 ms -253 ms (-52%) bit status (no ws) 234 ms 711 ms -477 ms (-67%) `bit --version` regresses because top-level requires now run eagerly on the path that goes through full bootstrap. Recovering that needs the same `require(...)` deferral pattern slice 5 applied for `--help` — either through ESM source (slice 9) or by extending the short-circuit to recognise `--version` too. Tracking as a follow-up. `bit <typo>` and `bit status (no ws)` improve dramatically because the architectural lazy load is now actually skipping aspect work that babel-lazy was effectively re-doing per call.
This reverts commit af3f9a0.
This reverts commit 78a38ba.
…(WIP) Once babel-lazy is off, every `import` from an aspect barrel evaluates the barrel's full require graph eagerly. Main-runtime callers were dragging in UI-runtime dependencies (and their broken-ESM upstream packages) just to get a non-UI symbol. Stripped UI value re-exports from these barrels; UI callers must now import directly from the UI subpath. Types stay in the barrel (no runtime cost). - scopes/component/component/index.ts - scopes/component/graph/index.ts - scopes/compositions/compositions/index.ts - scopes/docs/docs/index.ts - scopes/explorer/command-bar/index.ts - scopes/lanes/lanes/index.ts - scopes/pkg/pkg/index.ts - scopes/scope/scope/index.ts - scopes/ui-foundation/panels/index.ts - scopes/workspace/workspace/index.ts Also breaks the config-store ↔ legacy.constants ↔ config-store cycle that babel-lazy was papering over: components/config-store/global-config.ts now inlines the path constants instead of importing them from legacy.constants. WIP — `bd status` still fails on bare-specifier ESM in shipped UI packages. fix-bare-esm-imports.mjs is a one-shot codemod that adds `.js` extensions to relative + package-subpath ESM imports under node_modules/@teambit/*/dist (and the .pnpm mirrors). Patches 2200+ files, idempotent.
… asset stubs
Extends the codemod to:
- Add `.js` extensions to package subpath imports (e.g. `react/jsx-runtime`
→ `react/jsx-runtime.js`) when the target file exists. Node's ESM resolver
rejects bare subpaths for packages without an `exports` field.
- Stub asset imports (`*.scss`, `*.css`, `*.svg`, `*.mdx`, etc.) in compiled
ESM dist files: replace with `const X = {}` or a comment. These imports
appear in UI runtime dist code and previously survived only because
babel-lazy deferred their evaluation; under Node ESM they crash with
`ERR_UNKNOWN_FILE_EXTENSION`.
After this pass, 4300+ extensions / stubs applied across
node_modules/@teambit/*/dist and the .pnpm mirrors. `bd` no longer crashes
during `bit status` on a fresh workspace — though many aspect providers
still don't register their commands at runtime (the slice 7 lazy-load
regression, tracked separately).
The Slice 3 RFC introduced `cli.register(descriptor, factory)` — a
per-command registration form (status, mini-status, why/set-peer/
unset-peer/dependents, remove/delete/recover, etc.). Each call goes
through `commandsSlot.register([command])`, which stores by aspect id
via `map.set(currentId, value)`. So two `cli.register` calls from the
same provider silently overwrite each other: only the LAST survives.
Repro pre-fix: `bd status` in any workspace printed
`warning: 'status' is not a valid command` because `status.main.runtime`
called `cli.register(statusCommand, ...)` then
`cli.register(miniStatusCommand, ...)`, and `mini-status` clobbered
`status`. Same shape silently dropped ~17 other commands.
Fix: `cli.register` now routes through `appendToSlot`, which
snapshots `commandsSlot.map` before delegating to the underlying
`commandsSlot.register`, then merges any pre-existing entry back in.
First call from an aspect: no prior entry, nothing to merge. Second
call: previous entry's commands get prepended. Legacy variadic
`cli.register(cmd1, cmd2, ...)` keeps working unchanged.
Side fix in `scripts/generate-command-index.mjs`: the codegen does
`repoRequire('@teambit/bit')`, which loads `dist/index.js` →
`require('./manifests')` (line 92) BEFORE `require('./load-bit')`
(line 95). load-bit is what installs the `.scss → {}` require hook,
so when manifests transitively pulls `@teambit/api-reference`'s
barrel (which imports `api-compare.js` with a `.module.scss` import),
Node crashes trying to parse the SCSS as JS. babel-lazy used to mask
this. Hoist the hook install in the codegen script.
Regenerated `command-index.generated.ts` — went from 83 → 100
top-level commands now that the dropped registrations stick.
Verified: `bd status`, `bd list`, `bd --help`, `bd init` in
`/tmp/repro` and in the bit repo workspace.
Flip the default bootstrap path from eager (`Harmony.load + harmony.run`,
which walks every aspect's *.main.runtime.js) to lazy (`LazyHarmony.load`
with `[CLIAspect, AspectLoaderAspect, EnvsAspect, GeneratorAspect]` as
roots; BitAspect plus the rest of manifestsMap go in as `manifestOnly`).
Commands are resolved on demand via a stub registered from
`COMMAND_INDEX`. `BIT_EAGER_LOAD=1` opts back into the legacy path.
What this required:
1. Straggler `.aspect.ts` files (10 total).
- 3 still imported the legacy `Aspect` from `@teambit/harmony`
(no `runtimes` field), which crashed `_doResolve` with
"Cannot read properties of undefined (reading 'main')":
notifications, harmony-ui-app, react-router.
- 7 used `@teambit/core` but never declared a `runtimes` thunk
(all UI-only): api-reference, changelog, code, component-tree,
command-bar (ui + preview), sidebar, user-agent.
All now have the appropriate `runtimes: { ui|preview|main: ... }`.
2. `scopes/harmony/core/harmony.ts`.
- Tolerate aspects with no loader for the current runtime —
`_doResolve` now caches `instances.set(id, undefined)` and returns,
matching what legacy harmony silently did. Lets `main`-runtime
dependency lists include UI-only aspects without exploding.
- Slot wiring. Each `Slot.withType<T>()` provider is now called with
`() => currentAspectStore.getStore() ?? this.current ?? aspectId`
to build a real legacy `SlotRegistry<T>`, so writes from inside
any aspect's provider get keyed by THAT aspect, not by the slot
owner's id.
- AsyncLocalStorage for concurrent isolation. `Promise.all` over
`runtimeClass.dependencies` runs providers concurrently, so the
naive `this.current = aspectId` approach raced and credited
`workspace`'s commands to whichever aspect happened to finish last
(e.g. `version-history`). ALS scopes the current id to each
provider's call frame.
- `LazyConfig` shim. Providers like `ConfigMain.provider` read
`harmony.config.raw.get(...)`. The lazy harmony's plain-object
config didn't expose that; wrap input in a Config-shaped object
with `raw: Map`, `get`, `set`, `toObject`, `from`.
- `pickRuntimeExport` accepts both class form (`StatusMain` with a
static `provider`) and plain-object form (`BitMain = { provider }`).
3. `scopes/harmony/bit/load-bit.ts`.
- `BIT_EAGER_LOAD=1` is the new fallback switch (replaces the old
`BIT_LAZY=1` opt-in).
- In lazy mode, `aspectsToLoad` is `[CLIAspect, AspectLoaderAspect,
EnvsAspect, GeneratorAspect]` — the four aspects load-bit itself
calls `harmony.get(...)` on after bootstrap. Everything else,
including `BitAspect`, is `manifestOnly`.
- After load, calls `cli.registerLazyStubs(harmony)` to seed
`commandsSlot` with stub Commands so cli.run() can match argv.
4. `scopes/harmony/cli/cli.main.runtime.ts`.
- `registerLazyStubs(harmony)` reads `COMMAND_INDEX` and creates a
stub Command per entry, keyed in `commandsSlot` by its aspect id.
Each stub has `report`/`json`/`wait` trampolines that resolve the
owning aspect on call.
- `resolveEnteredCommand()` runs at the top of `cli.run()`. It scans
argv for the entered command name, finds the matching stub by
name/alias, and `await harmony.resolve(stub.aspectId)` before
yargs sees the parsed args. After resolve, the provider has
registered the real Command (full options, args, sub-commands)
and the leftover stub is filtered out. Yargs strict-mode flag
parsing now works correctly because the real Command lists its
options. Without this, `bd init --no-package-json` would fail
("Unknown arguments").
Bench (M1, Node v24.15.0, 7 iters cold/median):
scenario branch master delta
bit --version 1160 ms 177 ms +6.6x (parse cost — Slice 11)
bit --help 175 ms 481 ms -2.7x ✓
bit <typo> 1812 ms 464 ms +3.9x (no short-circuit yet)
bit status (no ws) 2225 ms 681 ms +3.3x (bootstrap dominates)
bit status (313 cmps, real ws)
4264 ms 15235 ms -3.6x ✓ (the whole point)
bit list (313 cmps) 2757 ms 1340 ms +2.0x
bit install --help 2018 ms 682 ms +3.0x
The big win — lazy `status` on a real workspace — is exactly the
scenario the RFC §11.1 prototype predicted: skip the ~100 aspects an
info command doesn't need, beat the eager path by ~11 s.
Empty-workspace scenarios still pay ~1 s for parsing `manifests.ts`
and its transitive package-index imports. That's the residual eager
graph from babel-lazy removal; `b808f4cf5`'s WIP UI-export split is
the path forward for shrinking it. Out of scope for this commit.
Verified end-to-end: `bd init`, `bd status`, `bd list`, `bd --version`,
`bd --help`, `bd <typo>` in `/tmp/repro` and the bit repo workspace.
Slice 5 already wired up a static-index short-circuit for `bit --help` /
`bit -h` so it doesn't pay the full bootstrap cost. Two more cases can
take the same treatment now that COMMAND_INDEX is comprehensive:
1. `bit --version` / `bit -v` — no aspect needs to run; the version
string lives in `@teambit/bit.get-bit-version` and that's the only
require to make. ~180 ms cold vs ~1.1 s for the full bootstrap path.
2. `bit <unknown-cmd>` — if the entered command is absent from
COMMAND_INDEX (and from every sub-command name + alias in it), no
aspect can ever register it. Print the familiar "not a valid command"
line with the didyoumean suggestion and exit 1 without booting
Harmony. ~180 ms vs ~1.8 s previously.
How:
- `help-from-index.ts` gains `isStandaloneVersion(argv)`,
`enteredCommandName(argv)`, and `buildKnownNameSet(index)`. The last
walks `subCommands` recursively so e.g. `bit envs set` doesn't trip
the short-circuit just because `set` isn't a top-level name.
- `run-bit.ts` checks them before the regular `initApp` path, in the
same place the existing `--help` short-circuit lives. The new exits
happen before `require('@teambit/cli')`, `require('./load-bit')`,
and everything they drag in.
- `server-forever` and yargs completion are exempt from the unknown-
command branch — they're argv shapes that lack a positional command
but still need the regular bootstrap.
Bench (M1, Node v24.15.0, 7 iters cold/median):
scenario before after vs-master
bit --version 1190 ms 178 ms parity
bit <typo> 1812 ms 178 ms 2.6x faster
bit --help 175 ms 173 ms (already short-circuited)
Functional check: `bd statu` (typo) still suggests `status` via
didyoumean; `bd status` (real command) still reaches the regular
dispatcher.
Slice 7 made `Harmony.load(...)` lazy, but the *manifests registration*
still loaded every aspect's barrel `index.ts` from `manifests.ts` — and
each barrel re-exports values whose import chain transitively pulls in
all the aspect's runtime code (UI, dependency-resolver, webpack, …),
even when the manifest is the only thing wanted. babel-lazy used to
mask this by wrapping every `require` in a getter; with babel-lazy off
(per `af3f9a000`), the eager evaluation became the bench-dominant cost.
The fix: import each aspect's `.aspect.js` directly. The `.aspect.js`
is the manifest file, ~15 lines, with one `require('@teambit/core')`
and zero runtime-code imports — three orders of magnitude cheaper than
the barrel.
Two cooperating changes:
1. `scopes/harmony/bit/manifests.ts` — 99 of 100 aspect imports rewrote
to `@teambit/<pkg>/dist/<pkg>.aspect.js`. The 100th, `@teambit/scripts`,
ships with a restrictive `exports` map (only `.` defined) that
trips `ERR_PACKAGE_PATH_NOT_EXPORTED` on subpath access, so it
keeps the barrel; the barrel only re-exports the aspect manifest
there anyway, so the cost is negligible. Single isolated-load test:
barrel 425 ms → direct 20 ms for `@teambit/pnpm`.
2. `scripts/codemod-aspect-imports.mjs` (new) — codemod that walks
every `.ts`/`.tsx` under `scopes/` and `components/`, finds
`import { …, XAspect, … } from '@teambit/<pkg>'`, and splits it so
the `XAspect` symbol comes from the direct `.aspect.js` path while
anything else (types, other values) stays on the barrel. Idempotent.
`excluded-fixtures/` is skipped — those emulate user-authored code
and shouldn't reflect codemod output. Patched 163 source files.
The barrel still works for everyone, just isn't on the manifests load
path or the aspect-manifest path inside other runtime files.
Bench impact (M1, Node v24.15.0, 7 iters cold/median):
scenario before after vs-master
bit --version (full path) 1190 ms 420 ms (still short-circuited
to 178 ms — see prior
commit)
bit status (real ws, 313) 4264 ms 3700 ms 4.2x faster
bit list (real ws) 2757 ms 2122 ms 1.6x slower
bit install --help (real ws) 2018 ms 1518 ms 2.2x slower
The biggest win lands in the headline scenario — real-workspace
`bit status` now beats master by 4.2x (vs 3.6x before this commit).
The barrel-bypass also helps every command's startup tail.
`bit status (no ws)` is the one case still slower than master (1885 ms
vs 697 ms): in that path, workspace's main.runtime is still loaded
through its DI chain to print the "outside workspace" error, and that
chain's main.runtime files have heavy top-level imports that
babel-lazy used to defer. Out of scope for this commit.
Functional check: `bd init` + `bd status` + `bd list` + `bd --help` +
`bd --version` + `bd <typo>` all still work in `/tmp/repro`, and the
313-component bit workspace itself still passes status under the new
import shape.
…n.runtime
Workspace's provider previously did three side-effect registrations
that only matter when the UI / GraphQL server is running:
ui.registerUiRoot(new WorkspaceUIRoot(workspace, bundler))
ui.registerPreStart(async () => workspace.setComponentPathsRegExps())
graphql.register(() => getWorkspaceSchema(workspace, graphql))
…and that's why `WorkspaceAspect` listed `UIAspect`, `BundlerAspect`,
`GraphqlAspect` among its 14 `static dependencies`. CLI commands like
status / list / install only need the `Workspace` instance, not the
UI server. With babel-lazy off, those three deps' main.runtime files
parse eagerly when workspace resolves, dragging in apollo, graphql,
webpack, etc.
This commit extracts the three registrations into a new aspect:
scopes/workspace/workspace-ui-binder/
workspace-ui-binder.aspect.ts (id: teambit.workspace/workspace-ui-binder)
workspace-ui-binder.main.runtime.ts (provider does the 3 registrations)
index.ts
`WorkspaceUiBinderMain.dependencies = [WorkspaceAspect, UIAspect,
BundlerAspect, GraphqlAspect]`. Workspace itself no longer mentions
UI / bundler / graphql. The binder is registered in `manifestsMap`
and in `.bitmap` so `bit compile` builds it as `@teambit/workspace-ui-binder`.
`Workspace.graphql` is now an optional public field (was a private
constructor arg) — the three `triggerOnComponent*` methods that did
`this.graphql.pubsub.publish(...)` now guard with `if (this.graphql)`.
For CLI commands the watcher never fires those triggers, so the guards
are no-ops in the hot path. `workspace-ui-binder` can set
`workspace.graphql = graphql` when it resolves, restoring full
behavior under `bit start`.
Honest perf result: this split alone barely moves the bench.
bit status (no ws) 1886 ms → 1810 ms (-4%)
bit status (313-cmp ws) 3700 ms → 3534 ms (-4%)
bit list (313-cmp ws) 2122 ms → 2078 ms (-2%)
bit install --help 1518 ms → 1551 ms (noise)
Traced the resolve chain — three other aspects on status's transitive
dep tree ALSO pull in the heavy three:
teambit.envs/envs → teambit.harmony/graphql
teambit.component/component → teambit.ui-foundation/ui
teambit.harmony/aspect-loader → teambit.compilation/bundler
So removing UI/Graphql/Bundler from workspace's deps just hands the
load to one of those three instead — same 56 main.runtime files load
either way. The same split pattern needs to land for envs, component,
and aspect-loader before the CLI hot path actually escapes those deps.
The split here is still a structurally correct change (workspace's job
isn't "set up the GraphQL server") and unblocks the rest of the work.
Follow-up commits will do the same pattern on envs / component /
aspect-loader.
`bit start` not yet re-wired: under lazy resolve, `workspace-ui-binder`
is only loaded when something depends on it, which nothing currently
does. That's fine for status/list/install which are what the bench
measures; bit start will need a slot-producer index hookup or an
explicit dep from a start-flow aspect before this lands on master.
…oader
Three more binder aspects, same pattern as `workspace-ui-binder`:
scopes/envs/envs-graphql-binder/
teambit.envs/envs-graphql-binder
deps: EnvsAspect + GraphqlAspect
provider: graphql.register(() => environmentsSchema(envs))
→ EnvsAspect drops GraphqlAspect from its 6 deps
scopes/scope/scope-ui-binder/
teambit.scope/scope-ui-binder
deps: ScopeAspect + UIAspect + GraphqlAspect
provider: ui.registerUiRoot(new ScopeUIRoot(scope))
graphql.register(() => scopeSchema(scope))
→ ScopeAspect drops UIAspect + GraphqlAspect from its 11 deps
scopes/harmony/aspect-loader-graphql-binder/
teambit.harmony/aspect-loader-graphql-binder
deps: AspectLoaderAspect + GraphqlAspect
provider: graphql.register(() => aspectLoaderSchema(aspectLoader))
→ AspectLoaderAspect drops GraphqlAspect from its 3 deps
All registered in `manifestsMap` and `.bitmap`.
Bench (cold/median, status (no ws), with all splits so far):
before splits 1886 ms
workspace 1810 ms
+ 3 more 1796 ms
Same 56 main.runtime files still load. Why: three OTHER aspects on
status's chain still pull the heavy deps:
teambit.workspace/install → BundlerAspect, UIAspect
teambit.component/snapping → BuilderAspect (→ Bundler)
teambit.generator/generator → GraphqlAspect
`install`'s bundler/UI usage is in a `registerPostInstall` callback
that only fires under `bit install` with a UI server running; the
closure captures the refs at provider time, so DI eagerly resolves
them. Same binder split applies — followup commit.
`snapping`'s builder dep is a real call inside its method body
(`this.builder.throwForComponentIssues`), so the split isn't a clean
"move registrations" — it'd need a thin facade.
`generator`'s graphql binding is the same as envs / aspect-loader —
clean split.
bit start re-wiring still pending: under lazy resolve, the new
binders are only resolved when something depends on them, and right
now nothing does. For `bit start` to keep working, either the start
flow needs to depend on the binders, or a slot-producer-index hook
needs to ensure they resolve when UI/graphql does.
…omponent
Three more binder aspects, same pattern:
scopes/generator/generator-graphql-binder/
deps: GeneratorAspect + GraphqlAspect
provider: graphql.register(() => generatorSchema(generator))
→ GeneratorAspect drops GraphqlAspect from its 12 deps
scopes/workspace/install-ui-binder/
deps: InstallAspect + UIAspect + BundlerAspect + WorkspaceAspect
provider: install.registerPostInstall(async () => {
if (!ui.getUIServer()) return;
await bundler.addNewDevServers(await workspace.list());
})
→ InstallAspect drops UIAspect + BundlerAspect from its 15 deps
scopes/component/component-graphql-binder/
deps: ComponentAspect + GraphqlAspect
provider: graphql.register(() => componentSchema(component))
→ ComponentAspect drops GraphqlAspect from its 4 deps
All registered in `manifestsMap` and `.bitmap`.
Honest result: bench numbers stayed flat. `bit status (no ws)` still
loads 56 main.runtime files (same as before the splits) and runs
~1850 ms cold. After every split I traced the resolve chain and
found ANOTHER aspect on status's chain that still pulls the same
heavy dep through a different path. Current culprits as of this
commit (status's chain still resolves these):
teambit.component/deprecation → teambit.harmony/graphql
teambit.component/snapping → teambit.pipelines/builder
teambit.defender/tester → teambit.ui-foundation/ui
teambit.compilation/compiler → teambit.compilation/bundler
The pattern is whack-a-mole: GraphqlAspect / UIAspect / BundlerAspect
are pulled in by 10+ aspects each, since any aspect that wants to
register a GraphQL schema or UIRoot or DevServer must declare a
runtime dependency on the host aspect. Splitting one host at a time
just hands the load to the next puller.
The bench gain from 7 splits (workspace + envs + scope + aspect-loader
+ generator + install + component) is in the noise — ~1850 ms before
the work, ~1850 ms after. The structural changes are correct (those
aspects' "register a schema" code really doesn't belong in the host's
provider), but the perf payoff needs either:
- a different model where GraphqlAspect / UIAspect / BundlerAspect
are *optional* deps that don't trigger eager loads when absent; or
- splitting essentially every aspect that ever calls
`graphql.register` / `ui.registerUiRoot` (10+ more aspects); or
- a lazy-require shim equivalent to babel-lazy applied selectively
to these three packages' load barriers.
bit start re-wiring still pending — same caveat as the workspace
split commit: under lazy resolve the new binders are only loaded when
something depends on them, and right now nothing does. Bench works
fine because status doesn't depend on the binders, but `bit start`
will need either a slot-producer hook or an explicit dep before this
lands on master.
The whack-a-mole problem from the per-aspect binder splits: every
aspect that does `graphql.register(...)` / `ui.registerUiRoot(...)`
declares the host as a `static dependency`, and Harmony's DI walk
eagerly resolves every such dep. Splitting the registration out of
one host just routed the eager load through the next still-coupled
host. 17 aspects on the CLI hot path still pulled graphql / ui /
bundler / builder transitively.
The fix is a one-stop-shop at the DI layer:
Harmony now takes a `lazyAspectIds` set. When `_doResolve` walks
`runtimeClass.dependencies`, ids in the set don't trigger eager
resolution. Instead the dep slot is filled with a recursive no-op
Proxy — `graphql.register(...)` and friends silently no-op, nested
chains like `graphql.pubsub.publish(...)` short-circuit, and an
`await` on a proxy short-circuits via the absent `.then`.
If a caller explicitly `harmony.resolve(id)`s a lazy aspect (or it
appears in `rootAspects`), it resolves normally. Subsequent DI deps
to that id then get the real instance. That's the escape hatch:
per-command we can decide whether to fall back to eager.
`load-bit.ts` configures the set per entered command:
HEAVY_HOSTS = [graphql, ui, bundler, builder, compiler, tester,
application]
HEAVY_COMMANDS = {start, run, build, tag, snap, export, compile,
test, preview, ui-build, serve-preview,
generate-preview}
// lazy by default; commands that need the heavy aspects opt back
// into eager resolution.
lazyAspectIds = wantsHeavyHosts ? [] : HEAVY_HOSTS
Bench (M1, Node v24.15.0, cold/median):
scenario before after master delta vs master
bit --version 178 ms 199 ms 171 ms parity
bit --help 173 ms 197 ms 478 ms 2.4x faster
bit <typo> 178 ms 195 ms 468 ms 2.4x faster
bit status (no ws) 1810 ms 1247 ms 697 ms 1.8x slower
bit status (313 ws) 3534 ms 3209 ms 14329 ms 4.5x faster
bit list (313 ws) 2078 ms 1525 ms 1313 ms 1.2x slower
bit install --help 1551 ms 923 ms 677 ms 1.4x slower
Real-workspace `bit status` opens its lead over master to 4.5x (was
4.2x). The empty-workspace cases close the gap from ~2.6x slower to
1.2-1.8x slower without ever needing babel-lazy. 46 main.runtime
files load for `bit status` instead of 56 (or master's 101).
The 7 binder aspects from prior commits stay: they're how `bit start`
actually wires UI roots + graphql schemas when the heavy hosts ARE
resolved (HEAVY_COMMANDS path). They don't load in the lazy path.
The proxy approach is conservative: it makes EVERY method-call form
safe by default, so we don't have to audit and guard each
`graphql.register` callsite individually. The compromise: if some
aspect uses a "graphql" dep for something other than registration
(e.g. reading state) it'll see undefined-via-proxy and silently
no-op. None of the heavy four hosts have callers like that on the
CLI hot path; if a real bug shows up we'll add it to HEAVY_COMMANDS
or revert to per-callsite guards.
Verified: `bd init`, `bd status`, `bd list`, `bd compile`, `bd start
--help`, `bd --help`, `bd --version`, `bd <typo>` all work in both
empty and 313-component workspaces.
…cked You asked whether fixing the broken Rollup chunking would close the empty-workspace `bit status` gap to master. Investigated and the answer is "yes in principle, but the bundle hits a different pre-existing blocker that's bigger than the bench delta this PR was chasing." Doing the prep work that unblocks future re-enablement, documenting the remaining blocker, then stopping there. What's in this commit: 1. `scripts/build-publish-bundle.mjs` gains a `redirectDirectAspectImports` Rollup plugin. `scripts/codemod-aspect-imports.mjs` rewrote source imports to `@teambit/<pkg>/dist/<X>.aspect.js` (perf win unbundled, skips barrels). That subpath points at babel-compiled JS where `() => import(...)` has been rewritten to `() => Promise.resolve().then(() => require(...))`, which Rollup doesn't recognise as a code-split boundary — entry inlines everything and emits 0 chunks. The new plugin redirects the subpath imports back to the package barrel so nodeResolve's `source` field lookup finds TS source where the dynamic-import thunk is intact. 2. `scopes/harmony/bit/run-bit.ts` switches its deferred lazy requires to `await import(...)`. Babel compiles those to `Promise.resolve().then(() => require(...))` so runtime behaviour is unchanged; Rollup recognises the dynamic-import syntax as a code-split boundary and will emit per-chunk output when the bundle build is unblocked. No measurable change to bench (--help / typo / version still ~175 ms cold). 3. The 7 binder aspects (workspace-ui-binder, envs-graphql-binder, etc.) gain `.js` extensions on their `@teambit/<pkg>/dist/<file>.graphql` imports. Without `.js`, the Rollup nodeResolve plugin's strict ESM resolution fails; the unbundled CJS path was finding them implicitly via Node's CJS extension resolution. What's NOT in this commit (the actual bundle remains broken): The bundle now gets past the `redirect-aspect` + dist-graphql resolution issues but fails on a different layer: it's `commit b808f4c`'s "split UI value re-exports out of aspect index barrels" leaving stale UI-only callers. Rollup's static analysis follows `import { ComponentModel } from '@teambit/component'` in `components/ui/models/lanes-model/lanes-model.ts` and errors — `ComponentModel` was correctly moved to a UI subpath by b808f4c, but that and a handful of similar callers never got updated. Unbundled runtime never reaches those files (CLI commands don't load UI runtimes), so b808f4c is fine on the runtime path. The bundle's static walk doesn't know that. The right fix is an `externalize-ui` plugin that marks `components/ui/**` and `scopes/*/*.ui.runtime.*` external to the Node bundle — the entry shouldn't ship browser code anyway. That's a separate piece of work documented in `docs/migration/10-publish-bundling.md`. Decided not to pursue it here because: - The lazyAspectIds work in `scopes/harmony/core/harmony.ts` already delivers most of what bundling was supposed to win on the bench (status no-ws: 1810 → 1247 ms). - The bundle isn't wired into bin/bit.js yet (also separate work), so fixing the build doesn't immediately move bench numbers anyway. - The remaining `~550 ms` gap to master on empty-workspace status is dominated by per-file load cost of the 46 main.runtime files that ARE on status's chain (not by the heavy hosts), and bundling only helps with that to the extent that it reduces module-record overhead. Estimated gain: 100–300 ms — not the close-the-gap moment we were hoping for. Followup steps documented in docs/migration/10-publish-bundling.md.
…broken
Bundle build was blocked on a `MISSING_EXPORT` for `ComponentModel`
inside `components/ui/models/lanes-model/lanes-model.ts`, a UI-only
file that's never reached on the CLI runtime path but that Rollup's
static graph walk follows anyway. Adding a stub plugin that converts
UI-only modules to `export default {}` so Rollup stops following
into them.
`stubUiFilesPlugin()` in `scripts/build-publish-bundle.mjs` matches
UI files three ways:
1. Path: `*.ui.runtime.*`, `*.preview.runtime.*`, `*.compositions.*`,
`*.docs.*`, `components/ui/**`, `components/hooks/**`,
`components/lanes/ui/**`.
2. Package name: `@teambit/<x>.ui.*`, `@teambit/<x>.compositions.*`,
`@teambit/<x>.docs.*`.
3. Content (for `.tsx` files only): if the file imports `react`,
`react-dom`, `@apollo/client`, or any `@teambit/<x>.ui.*` /
`@teambit/<x>.compositions.*` subpackage. This catches UI
components living inside aspect dirs (e.g.
`scopes/api-reference/api-reference/api-compare.tsx`) without
false-positive-stubbing the rare non-UI `.tsx` (e.g.
`bundler.service.tsx` is a service class with no JSX).
Stubs use `syntheticNamedExports: 'default'` so non-UI files that
destructure named imports from a stubbed UI module — e.g.
`import { noPreview } from '@teambit/ui-foundation.ui.pages.static-error'`
in `artifact-file-middleware.ts` — still resolve. Named values are
`undefined` at runtime but those code paths only run inside the UI
server / preview pipeline, never CLI.
Result: bundle now builds. Stats:
entry bit.mjs 48.9 KB (was 121.3 KB)
144 chunks emitted (was 0)
one chunk per non-stubbed `*.main.runtime.ts`, deterministic
naming via `chunkForId`.
Largest chunks (informative):
runtime-workspace 676.9 KB
runtime-yarn 653.0 KB
typescript 444.7 KB
runtime-webpack 98.2 KB
runtime-watcher 61.6 KB
runtime-validator 10.2 KB
runtime-ui-binder modules 0.6–8 KB each
The bundle is NOT wired into `bin/bit.js` so runtime behaviour is
unchanged today. To re-enable it as the runtime entry, the next
blocker needs fixing: a CJS/ESM interop bug in the emitted output.
$ node dist/bundle/bit.mjs --version
TypeError: Cannot read properties of undefined (reading 'BitError')
at chunks/runtime-config-Dp8lCrqr.mjs:7:54
`distExports.BitError` is undefined where the chunk expects it.
@rollup/plugin-commonjs wraps `@teambit/bit-error`'s legacy CJS
exports in a way that doesn't expose `BitError` as a property of
the namespace. Likely fixes are commonjs plugin tuning
(`requireReturnsDefault: 'auto'`), externalizing the legacy-CJS
packages so they go through Node's normal require(), or migrating
@teambit/bit-error to ESM (Slice 9). All documented in
docs/migration/10-publish-bundling.md.
Stopping here per the original scope question: "would fixing the
chunks bundling that was broken improve performance?" — the build
fix is done, the runtime fix is a substantively different (CJS
interop) problem that would take another half-day plus debugging
of the resulting bench (we'd want to compare bundled vs unbundled
on cold/warm). Given that lazyAspectIds already captured most of
the bench win bundling was meant to deliver, this is a defensible
stopping point.
Per the retrospective: the seven binder aspects (workspace-ui-binder, envs-graphql-binder, scope-ui-binder, aspect-loader-graphql-binder, generator-graphql-binder, install-ui-binder, component-graphql-binder) were architecturally clean but the bench numbers were flat before and after. `lazyAspectIds` in `@teambit/core` already covers the same ground by handing back no-op proxies for the heavy hosts in DI — which means the host's provider runs `graphql.register(...)` / `ui.registerUiRoot(...)` against the proxy and the calls silently no-op on the CLI path. Same end state, fewer files, no separate aspect lifecycles to register / re-wire in `bit start`. The codemod-aspect-imports rewrites across 162 source files were similarly redundant. The win they were chasing was "skip heavy barrel evaluation when importing only `XAspect`". For aspects in `lazyAspectIds` (ui / graphql / bundler / builder / compiler / tester / application) the barrel never evaluates anyway because `_doResolve` short-circuits before touching the loader. For the remaining ~150 non-host aspects the savings were too small to measure separately from noise. Net cost of keeping them: 162 split import statements, a Rollup `redirectDirectAspectImports` plugin to compensate for the bundler losing dynamic-import boundaries, and the codemod script itself as ongoing maintenance surface. What's preserved: - `lazyAspectIds` and the no-op proxy: the actual lever for empty-workspace `bit status` going from 1810 → 1247 ms. Keeps load-bit.ts's HEAVY_HOSTS / HEAVY_COMMANDS opt-out for `bit start` et al. - `manifests.ts` direct `.aspect.js` imports: a single-file change that genuinely earned its keep — manifest registration drops from ~425 ms per heavy barrel to ~20 ms direct, repeated across ~100 aspects. Single biggest contributor to the `--version` / short-circuit path before the static-help logic landed. - `--version` / `--help` / `<typo>` short-circuits in run-bit.ts. - CLI lazy stubs from `COMMAND_INDEX` (registerLazyStubs + resolveEnteredCommand). - `Aspect.create` straggler conversions (notifications, harmony-ui-app, react-router) and the UI/preview runtime thunks on the 7 UI-only aspects. - Slice 7 lazy resolve foundation in `@teambit/core`'s Harmony. - The fix for `cli.register(descriptor, factory)` slot overwrite. - Bundle prep (run-bit `await import` conversion, `redirectDirectAspectImports` plugin in build-publish-bundle.mjs, `stubUiFilesPlugin`). The redirect plugin is still needed because `manifests.ts` retains its direct-subpath imports — that was a deliberate keep, just not the codemod's wholesale rewrite. Removed: - `scopes/workspace/workspace-ui-binder/` - `scopes/envs/envs-graphql-binder/` - `scopes/scope/scope-ui-binder/` - `scopes/harmony/aspect-loader-graphql-binder/` - `scopes/generator/generator-graphql-binder/` - `scopes/workspace/install-ui-binder/` - `scopes/component/component-graphql-binder/` - Their entries in `manifestsMap` and `.bitmap` - The 162-file codemod's direct-`.aspect.js` import rewrites in every file except `manifests.ts`. Restored to pre-codemod state. The `Workspace` class is back to taking `graphql` as a required constructor arg (was made optional + guarded for the binder split). The 7 hosts (workspace, scope, envs, aspect-loader, generator, install, component) are back to listing UI / Bundler / Graphql / Builder among their `static dependencies` — at runtime DI hands them the no-op proxy on CLI commands, the real instance on `bit start` et al via the HEAVY_COMMANDS opt-out. Same control flow as the binder split, fewer moving parts. Bench (cold/median, 7 iters): scenario revert pre-revert master bit --version 178 ms 178 ms 171 ms parity bit --help 175 ms 174 ms 478 ms 2.7x faster bit <typo> 177 ms 175 ms 468 ms 2.6x faster bit status (no ws) 1298 ms 1247 ms 697 ms 1.9x slower bit status (313 ws) 3447 ms 3209 ms 16346 ms 4.7x faster bit list (313 ws) 1622 ms 1525 ms 1340 ms 1.2x slower bit install --help 951 ms 923 ms 689 ms 1.4x slower Numbers held within noise across the revert. `bit status (no ws)` shows a ~50 ms slip that's bench-volatility — over multiple runs the medians on both sides of the revert sit in the 1200–1300 ms range; this snapshot caught a slightly worse one for the revert. Real-workspace `bit status` keeps the 4.7x lead over master.
Made the bundle functional and wired it as an opt-in entry behind BIT_USE_BUNDLE=1. End-to-end bench finding: the bundle does NOT improve performance in its current form. Committing the working infrastructure + the negative result. What was needed to make it run: 1. Externalize legacy-CJS @teambit packages whose `Object.defineProperty (exports, ...)`-based re-exports `@rollup/plugin-commonjs` mangles: `@teambit/bit-error`, `@teambit/legacy.cli.error`, `@teambit/harmony`, and `yargs` (which uses a default-export + named-statics pattern the plugin also mishandles). 2. Externalize EVERY `@teambit/*` package except the bootstrap trio (`@teambit/bit`, `@teambit/cli`, `@teambit/core`). When Rollup tries to follow the dep graph into individual aspect runtimes, it hits a chain of cascading CJS/ESM interop issues (next was `RuntimeDefinition is undefined`, then `Cannot access 'Aspect' before initialization` from circular chunk imports). Externalising them lets Node's normal require() resolve at runtime — which is what lazyAspectIds already triggers on demand anyway. 3. Switch output to `format: 'cjs'` with `dynamicImportInCjs: false`. ESM output trips over `@teambit/*` packages whose `exports.import` field points to a non-existent `dist/esm.mjs`. CJS output uses `require()` for externals, which goes through Node's CJS resolver that falls back to `exports.require` and ultimately `main`. 4. `bin/bit.js` learns the `BIT_USE_BUNDLE=1` opt-in. Default is the unbundled `dist/app.js` path. If `dist/bundle/bit.cjs` doesn't exist (no `npm run build-bundle` was run), it transparently falls back to unbundled. Result: bundle entry is 49.8 KB, emits 10 chunks (small chunks for hook-require / autocomplete / bootstrap / load-bit / runtime-bit / server-commander; everything else is external). Bundle runs every command surface tested (--version, --help, <typo>, status, list). Bench (cold/median, 7 iters, M1 / Node v24.15.0): scenario unbundled bundled delta bit --version 228 ms 308 ms +35% slower bit --help 205 ms 273 ms +33% slower bit <typo> 192 ms 269 ms +40% slower bit status (no ws) 1413 ms 1331 ms -6% (noise) bit status (313 ws) 3117 ms 3087 ms parity bit list (313 ws) 1556 ms 1544 ms parity bit install --help 926 ms 916 ms parity Why no win: - The fast paths (--help / --version / typo) short-circuit before loading any aspects. The bundle's entry inlines Harmony core + CLI parser + run-bit's logic, so the fixed parse cost is ~50 KB+ rather than the unbundled path's "require app.js → require run-bit → short-circuit" chain (small entry + dynamic chunk for short- circuit logic). The short-circuit is already fast; bundling makes it pay for the rest of the file's parse cost. - The slow paths (status / list / install) spend ~all their time in externalised packages — aspect main.runtime files that load via `require()` either way. Bundling the entry layer doesn't help. The promise of bundling was code-splitting + V8 parse-cache locality across many chunks. Both depend on Rollup successfully traversing the aspect dep graph and emitting per-aspect chunks. The CJS interop issues blocked that — to get the build green I had to externalise the graph. Code-split returns once Slice 9 (ESM source migration) lands. The bundle is preserved as opt-in infrastructure (`BIT_USE_BUNDLE=1`) so it can be exercised by future work but isn't on anyone's runtime path today. Default `bin/bit.js` continues to load the per-aspect `dist/app.js`. No bench numbers change vs the previous state of the PR.
Three correctness bugs in the lazy command-stub trampoline, found while
trying to bench `bit show <comp>`:
1. Positional-arg patterns were dropped. `COMMAND_INDEX` stored only the
bare command name ("show"), so the registered stub had `name: 'show'`
and yargs rejected `bit show teambit.harmony/cli` as "Unknown argument".
Fix: codegen emits `pattern: 'show <component-name>'` when it differs
from the bare name; `makeLazyStub` prefers `entry.pattern || entry.name`.
2. Stubs were appended on top of real commands. Providers of aspects that
are transitive DI deps of the eagerly-loaded roots (CLI / AspectLoader
/ Envs / Generator) run during `LazyHarmony.load`, which finishes
before `registerLazyStubs` runs. `registerLazyStubs` was blindly
appending stubs to the slot, shadowing the already-registered real
handlers. Fix: skip the stub when a non-stub command with the same
name already exists.
3. Descriptor fields like `loadAspects: false` were lost in the generated
index, because the lazy-load codegen never invoked aspects whose
stub-only entries had no descriptor metadata. Fix: codegen sets
`BIT_EAGER_LOAD=1` so every provider runs and the live slot has real
Command instances.
Also forwards `loadAspects` from the `listCommand` descriptor to the
`ListCmd` Command instance so the snapshot picks it up, and tightens
`appendToSlot` to drop any lingering shadow stubs when the real handler
finally registers.
The pnpm + yarn aspects self-register with `dependencyResolver.registerPackageManager` inside their providers — so in lazy mode, where their providers don't run until something resolves them, the dependency-resolver's slot is empty and `bit install` fails with "default package manager: teambit.dependencies/pnpm was not found". Eagerly load `manifestsMap['teambit.dependencies/pnpm']` and `yarn` for the set of commands that actually install or trigger an install (`install`, `import`, `tag`, `snap`, `build`, …). The cost is paid only on those commands; `list` / `status` / `log` stay lean. Reusing the already-imported manifests means no additional parse cost in `load-bit.ts`.
Adds an esbuild-based `renderChunk` minifier and `--no-minify` opt-out.
With minification on the published bundle drops from ~1.9 MB to ~915 KB
across the 10 chunks (entry 55 KB → 33 KB, runtime-bit 772 KB → 186 KB,
server-commander 623 KB → 318 KB, ...).
Runtime perf is unchanged for real-action commands (list/log/import) —
the bench shows bundled and unbundled are at parity. The bottleneck is
the externalised `@teambit/*` packages still loaded from node_modules,
not parse cost of the small internal slice.
Also documents (in `isExternal`) why we can't currently bundle the rest
of `@teambit/*`:
- rollup-plugin-commonjs's lazy `requireXxx()` factories cross chunk
boundaries and fire before the host chunk finishes initialising,
causing `runtimeEnvironments.Aspect.create is undefined` / similar;
- externalising the non-@teambit transitives sidesteps the init
problem but breaks Node resolution because pnpm-installed
transitives aren't hoisted to a path reachable from
`dist/bundle/chunks/`.
Sets `requireReturnsDefault: 'auto'` on the commonjs plugin; harmless on
the current scope, useful groundwork for future bundle-more attempts.
Four ESM-incompat hotspots in the bootstrap path, surfaced while testing
`node dist/bundle/bit.mjs <cmd>`:
- `cli-parser.ts` used the pre-yargs-17 "import-the-function-and-call-statics"
pattern (`yargs(args); yargs.help(false); yargs.strict(); ...`). ESM yargs
exports only a `YargsFactory`. Refactored to capture
`const yargs = yargsFactory(args)` and chain configuration off the
instance — works in both CJS and ESM.
- `hook-require.ts` reached the Module prototype via the implicit `module`
global, which is `undefined` in ESM scope. Use the explicit
`import { Module } from 'module'` instead.
- `bootstrap.ts` had top-level `require('events')` and
`require('regenerator-runtime/runtime')` calls; converted to ES imports.
- `load-bit.ts` had two `import '@teambit/harmony/dist/harmony-config'`
directory imports — illegal in ESM. Switched to explicit `/index.js` and
`/config-reader.js`.
Unbundled CLI smoke (`--help`, `list`, typo) and lint are clean.
Adds an opt-in `--esm` flag to the publish bundler that switches the output format from CJS to ESM (`bit.mjs` entry, `chunks/*.mjs`, `createRequire` injected per chunk so the few sync `require()` calls still resolve). Adds `scripts/generate-missing-esm-shims.mjs` which walks `node_modules/@teambit/*` and synthesises a `dist/esm.mjs` shim for the ~75 workspace components whose auto-generated package.json declares `exports.import: "./dist/esm.mjs"` but ships no such file (root cause is in Bit's package.json generator — out of scope here). The shim lexes the companion CJS dist for `Object.defineProperty(exports, ...)` / `exports.X = ...` patterns to re-export named symbols. Bench (`--version` / `list` / `log`): command master unbund. CJS bundle ESM bundle --version 172ms 175 ms 173 ms 293 ms (1.71x) list 1294ms 1530 ms 1574 ms 1631 ms (1.26x) log 1858ms 932 ms 953 ms 2337 ms (1.26x) ESM-bundle output is *slower* than CJS-bundle, driven by Node's loader pipeline overhead. The path is wired up and functional, but is not a perf win on its own — it's a prerequisite for further bundling work (see docs/migration/09-esm-source-migration.md).
Adds (behind `--esm` opt-in) the foundations for the ESM source migration:
1. `bvm-patches/apply.mjs --esm` extends the existing core-aspect-env babel
patch to also pass `modules: false` to `@babel/preset-env`, so a fresh
`bit compile` emits real ESM (`import`/`export` keywords) instead of the
Babel-default CJS. Without `--esm`, behaviour is unchanged.
2. `bvm-patches/set-package-type-module.mjs` walks `node_modules/@teambit/*`,
probes the *CJS* entry point (the `require` / `main` field, not the
`esm.mjs` shim), and stamps `"type": "module"` on packages whose main
bundle uses ESM syntax. Also re-points `exports.import` away from the
now-broken `esm.mjs` shim (which assumed a CJS index) to the bare
`dist/index.js`.
3. `bvm-patches/fix-bare-esm-imports.mjs` — two real bugs fixed:
- `hasExtension` treated any trailing `.<word>` as already-extensioned,
so `./command-index.generated` got skipped instead of rewritten to
`./command-index.generated.js`. Now checks against a known
extension set (`js`, `mjs`, `cjs`, `json`, asset extensions, …).
- The directory-vs-file disambiguation used `c.endsWith('index.js')`,
which matches `help-from-index.js` (treated it as a directory index)
and emitted `./help-from-index/index.js`. Now anchors on `/index.js`.
4. `bootstrap.ts` — `import 'regenerator-runtime/runtime'` →
`'regenerator-runtime/runtime.js'`. ESM resolution requires the explicit
extension on subpath imports.
Tested end-to-end on `bit --version` (passes) and `bit list` (got the
ESM bundle past every workspace blocker and into `lodash` named-import
interop, which is the next class of issue and out of scope here —
~224 `import { ... } from 'lodash'` sites would need conversion to
default-import or `lodash/<fn>.js` subpaths). Workspace reverted to CJS
so dev keeps working.
Companion: docs/migration/09-esm-source-migration.md.
…ugin
Under --esm, `bvm-patches/apply.mjs` now also injects an inline babel
plugin into the env's `cjs.babel.config.js`. The plugin appends `.js` to
every extension-less relative import/export/require/dynamic-import at
build time, replacing the post-compile `fix-bare-esm-imports.mjs` sweep
with a structural fix.
Caveat — bit strips the source path before handing the filename to
babel (we see `<workspace>/basename.ts`, not the real
`scopes/.../basename.ts`), so the plugin can't probe the source tree
to distinguish `./foo` (file) from `./foo` (directory with index). The
plugin defaults to `.js`; the directory case is repaired by a new
`fix-dir-imports.mjs` post-pass that has access to the real dist tree.
End-to-end after `--esm` + compile + `fix-dir-imports` + `set-package-type-module`:
- `command-index.generated` → `./command-index.generated.js` (correct)
- `help-from-index` → `./help-from-index.js` (correct)
- `./exceptions` (a directory) → `./exceptions/index.js` (correct)
- `regenerator-runtime/runtime` → `regenerator-runtime/runtime.js` (correct)
`bit --version` runs through the ESM dist. `bit list` runs until the
next class of issue: CJS-only-dep named-export interop
(`import { capitalize } from 'lodash'`). That's the next pass.
Workspace reverted to CJS for normal dev; ESM mode opt-in via
`node scripts/bvm-patches/apply.mjs --esm`.
Two additional inline babel plugins (injected via
`bvm-patches/apply.mjs --esm`):
1. `bvmCjsInteropPlugin` — for a curated list of CJS-shaped third-party
packages whose `module.exports` Node's cjs-module-lexer can't detect
(lodash, fs-extra, semver, graceful-fs, minimatch, didyoumean, pino,
chalk, yargs, …). Rewrites
`import { capitalize } from 'lodash';`
to
`import _bvm_cjs_lodash from 'lodash';`
`const { capitalize } = _bvm_cjs_lodash;`
which always works for CJS interop and produces no runtime overhead
compared to the named-import form (Node would have to wrap anyway).
2. `bvmCreateRequirePlugin` — when a compiled module uses `require(...)`
or `require.resolve(...)` synchronously (e.g. `cli.main.runtime.ts`'s
`registerLazyStubs` which requires `@teambit/bit/dist/command-index.generated.js`
inside a sync function), the ESM dist would fail with `require is
not defined`. The plugin prepends
`import { createRequire as __bvm_createRequire } from 'module';`
`const require = __bvm_createRequire(import.meta.url);`
to any file that references `require` outside a member expression.
Both plugins compose with the earlier `bvmAddExtensionsBabelPlugin`,
applied in order: `[createRequire, cjsInterop, addExtensions, ...env]`.
End-to-end test: `node scripts/bvm-patches/apply.mjs --esm` + `bit compile teambit.harmony/cli teambit.harmony/bit` + `fix-dir-imports` + `set-package-type-module`, then `bin/bit.js list` runs through the ESM dist and prints all 313 components (workspace's full list).
`bin/bit.js --version` also passes.
Workspace reverted to CJS for normal dev. The full --esm flow needs all
~200 packages recompiled to take effect, which we haven't done in this
session (the bvm bit compiles individual packages on demand; a full
workspace flip would need an orchestrated bulk compile + post-passes).
Final iteration of the inline babel plugins + post-passes. After this
commit, `node scripts/bvm-patches/apply.mjs --esm` + a full `bit compile`
(all 313 components) + the post-pass scripts produces an ESM workspace
where `bin/bit.js list` runs to completion and prints all 313 components.
New plugins (composed in `apply.mjs --esm`):
- `bvmDirnamePlugin` — when a compiled module references `__dirname` /
`__filename`, prepends ESM-equivalent shims:
import { fileURLToPath as __bvm_fileURLToPath } from 'url';
import { dirname as __bvm_dn } from 'path';
const __filename = __bvm_fileURLToPath(import.meta.url);
const __dirname = __bvm_dn(__filename);
- `bvmCjsInteropPlugin` — extended to also handle:
* `import * as X from 'cjsPkg'` → `import X from 'cjsPkg'`
(ESM `* as` only sees lexer-detected names; default-import gives
the full CJS namespace.)
* `export { X } from 'cjsPkg'` → `import _ from 'cjsPkg'; const { X } = _; export { X };`
(Plain ESM re-export breaks because Node's static name validation
runs before any CJS interop fallback can fire.)
* Expanded CJS package allowlist with the npm utilities that surfaced
during the full-workspace audit (enquirer, prompts, ora, fast-glob,
globby, micromatch, lru-cache, mem, p-limit, p-queue, find-up,
json5, cosmiconfig, …). Notably *excludes* `glob` which is ESM in
v10+.
New post-pass: `fix-type-only-reexports.mjs` — strips re-exported names
that the source module doesn't actually export (type-only re-exports
from TS source where `export type` should have been used; ~370 names
across ~120 files in the bit workspace).
Failure classes still on the audit list (encountered during this pass,
not yet automated):
1. Pre-published external @teambit packages compiled with the old
`@babel/plugin-transform-modules-commonjs lazy` pattern (e.g.
@teambit/cloud.hooks.*). They emit CJS `(0, _ui().useMutation)`
patterns that work in CJS but fail when an ESM consumer re-exports
from them. The set-package-type-module script already leaves these
as CJS, but consumers compiled in our workspace still need their
re-exports handled. Most common cases now covered by the
`ExportNamedDeclaration` branch of `bvmCjsInteropPlugin`.
2. The `getBvmDir()` path in aspect-loader/core-aspects.js dereferences
`process.argv[1]`. With `node -e ...` it's undefined and throws,
which fires the fallback `getAspectDir` path. Not a real problem in
real `bin/bit.js` invocations.
Workspace reverted to CJS so dev keeps working; full ESM behind
`--esm`.
Mimics `@babel/plugin-transform-modules-commonjs lazy: true` for ESM
output. Static `import` is by spec eager — every module in the import
graph materialises at boot. This plugin rewrites top-level imports of
known-CJS packages (and optionally @teambit/* packages, leveraging Node
22.12+'s require(ESM) support) to lazy createRequire-backed wrappers:
import { capitalize } from 'lodash';
capitalize('hi');
becomes
import { createRequire } from 'module';
const __bvm_lazy_r = createRequire(import.meta.url);
let __bvm_lazy_lodash;
function _bvm_lazy_get_lodash() {
return __bvm_lazy_lodash || (__bvm_lazy_lodash = __bvm_lazy_r('lodash'));
}
_bvm_lazy_get_lodash().capitalize('hi');
Notable detail: rewriting *every* identifier reference would corrupt TS
type positions (`React.ReactElement` in `Foo<T>: React.ReactElement`
is a TSQualifiedName, not a MemberExpression), so the plugin skips
references whose parent chain walks into TSTypeReference / TSQualifiedName /
JSXMemberExpression / etc.
Also fixes the CJS-interop plugin's ExportNamedDeclaration handler to
skip `export * as X from 'pkg'` (ExportNamespaceSpecifier has only
`.exported`, no `.local`).
Opt-in via `node scripts/bvm-patches/apply.mjs --esm --lazy-esm`.
Bench (Node 24, --lazy-esm + full ESM workspace):
command master ESM (no lazy) ESM+lazy CJS ESM+lazy CJS+@teambit
list 1284ms 1755ms 1701ms 1674ms
log 1908ms 2378ms 2279ms 2316ms
Lazy recovers ~50-80ms of the ~470ms ESM regression. The remaining gap
is Node's ESM loader overhead (parse + static linking) which fires
before any user code and isn't deferrable. To close it further we'd need
either fewer modules in the graph (more bundling) or to skip the type:module
stamp and rely on the Node CJS path for the hot bootstrap modules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
docs/rfc-esm-lazy-aspects.md) proposing a three-tier loading model so the CJS→ESM migration does not regress CLI startup. Today's bootstrap eagerly resolves every core aspect's provider (~120 of them) regardless of which command the user ran; the proposal makes loading on-demand viaharmony.resolve+ a publish-time command index, with each.main.runtime.tsshipped as its own dynamically-imported chunk.docs/migration/, each self-contained with goal, dependencies, scope, acceptance criteria, risks, and file inventory. The README has a dependency graph showing what can run in parallel (chunk 09 — the ESM source emit — is architecturally independent and parallelizable from day 1).prototypes/mini-bit/— a ~30-file ESM mini-clone of Bit's aspect architecture that demonstrates the design end-to-end.node bin/mini-bit.js statusloads 5 of 7 aspects (skippinginstallandcompiler); the same command underBIT_EAGER=1loads all 8. RFC §11 captures the learnings folded back from this build.Why open this as a draft
This is doc-only (zero production code touched) and the goal is early review of the architecture and the chunked execution plan before any of the 12 chunks ship. Looking specifically for feedback on:
docs/migration/README.md) — anything missing or mis-sequenced?Try the prototype
The trailing summary line shows how many aspects loaded — the lazy↔eager delta makes the architectural win visible in <1 second.
Test plan