Skip to content

Handle loading TypeStrip modules in applications using VM loader#241

Merged
kriszyp merged 9 commits intomainfrom
vm-load-typescript
Mar 18, 2026
Merged

Handle loading TypeStrip modules in applications using VM loader#241
kriszyp merged 9 commits intomainfrom
vm-load-typescript

Conversation

@kriszyp
Copy link
Copy Markdown
Member

@kriszyp kriszyp commented Mar 17, 2026

No description provided.

@kriszyp kriszyp force-pushed the vm-load-typescript branch from 1438b23 to 52ab127 Compare March 17, 2026 15:41
@kriszyp kriszyp force-pushed the vm-load-typescript branch from a727190 to fe3f544 Compare March 17, 2026 15:49
@socket-security
Copy link
Copy Markdown

socket-security Bot commented Mar 17, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedamaro@​1.1.89310010094100

View full report

@kriszyp kriszyp marked this pull request as ready for review March 17, 2026 18:25
@kriszyp kriszyp requested a review from a team as a code owner March 17, 2026 18:25
Copy link
Copy Markdown
Contributor

@dawsontoth dawsontoth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the fallbacks. Could amaro be an optional peer dep, maybe?

Comment thread security/jsLoader.ts Outdated
Comment thread security/jsLoader.ts Outdated
// Use amaro - the library that Node.js uses internally for type stripping
// Try to use the internal one first, if we can, otherwise fallback to loading the external one
if (!amaro) {
amaro = await import('internal/deps/amaro/dist/index').then(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not recommend relying on the bundled amaro version just like I don't recommend on relying on the bundled version of undici.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't we rather be consistent with with host's typestripping? If there are actually diffs between whatever amaro we are using and node's, it doesn't seem clear to be me that it is desirable to be inconsistent with our host.
But the more salient point is that import('amaro') will load multiple MBs of extra code. For someone complaining about 20KB in another ticket, I thought this would be pretty important consideration ;).

relying on the bundled version of undici

That's literally what we do every time we call (global) fetch isn't it?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it matters to be consistent with the host's typestripping nearly as much as it matters to be consistent with what we support. The fallback regex gauntlet proves that type stripping isn't rocket science.

I was unaware of how big amaro is (3.23MB), but that's a distribution size thing that is only relevant for install.

At runtime, whether you import amaro or internal/deps/amaro/dist/index, it loads roughly the same amount of code. The only time this becomes an issue is when you enable Node.js' type stripping/transform types, then it would load two different versions of amaro. This is obviously not ideal and why your code is nice. However, right now, we don't enable type stripping, so it's not an issue. If we wait long enough, we can drop Node.js 20, use the internal amaro, and hope the path doesn't change too often.

If we were concerned about runtime memory, then we would just use the regex gauntlet.

Regarding undici, it indeed does power global fetch(), but they are certainly not he same. fetch() was designed for convenience, not performance. If you need maximum performance and more control over streaming data, then you use undici. If you need to do a quick HTTP call with a minimal request/response, you can use fetch().

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At runtime, whether you import amaro or internal/deps/amaro/dist/index, it loads roughly the same amount of code.

I see your point; you are saying that by default internal amaro won't be loaded, so should be the same... I think that is true:

> process.memoryUsage()
{
  rss: 56676352,
  heapTotal: 8712192,
  heapUsed: 5667384,
  external: 2472825,
  arrayBuffers: 10533
}
> require('internal/deps/amaro/dist/index')
{ transformSync: [Getter] }
> gc()
undefined
> process.memoryUsage()
{
  rss: 74571776,
  heapTotal: 11149312,
  heapUsed: 9387120,
  external: 8598955,
  arrayBuffers: 10705
}
> 
(To exit, press Ctrl+C again or Ctrl+D or type .exit)
> 
kzyp@kzyp-XPS-15-9520:~/dev/harper-pro/core$ node --expose-gc --expose-internals
Welcome to Node.js v24.10.0.
Type ".help" for more information.
> process.memoryUsage()
{
  rss: 56250368,
  heapTotal: 7663616,
  heapUsed: 5589456,
  external: 2472819,
  arrayBuffers: 10527
}
> require('amaro')
{ transformSync: [Getter] }
> gc()
undefined
> process.memoryUsage()
{
  rss: 72826880,
  heapTotal: 17776640,
  heapUsed: 12546496,
  external: 5141239,
  arrayBuffers: 10705
}

This is actually really confusing; external amaro adds 3 more MBs of heap than internal amaro. But internal amaro uses more "external" memory, and I guess they end up being the same.

what we support

But we don't explicitly support anything specific in TypeStrip, we just vaguely support TypeStrip as a general principle, which is exactly the level of vagueness the internal module provides.

But anyway, pretty easy to go with amaro as a package. Not fully convinced of the benefit, but I can switch that, maybe its slightly safer.

Comment thread security/jsLoader.ts Outdated
);
amaro ??= await import('amaro').catch(() => null);
}
if (amaro?.transformSync) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use amaro from npm, we can guarantee that transformSync exists.

As a bonus, we can strongly type the amaro variable above.

Comment thread security/jsLoader.ts Outdated

// Fall back to basic regex-based stripping
// Remove type annotations from parameters and variables
source = source.replace(/:\s*[A-Za-z_$][\w$<>[\]|&,\s]*(?=[,)\]=;])/g, '');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for Node.js 20? Node.js 20 becomes EOS in like 4 weeks.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. I'll remove in 4 weeks. Although no guarantees that we won't need to limp someone along on v20, obviously has happened before.

Copy link
Copy Markdown
Member

@Ethan-Arrowood Ethan-Arrowood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it works, I'm okay with it.

I think the regex is a bit excessive. Realistically, when would we fail to load the external amaro? I think if that happens we should properly error not fallback to some alternate less-effective procedure. They UX will just not be the same

Comment thread security/jsLoader.ts Outdated
Comment thread security/jsLoader.ts Outdated
Copy link
Copy Markdown
Member

@Ethan-Arrowood Ethan-Arrowood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol you updated it as soon as I reviewed.

Well now I'm even happier with it!

Just want to assert we are okay with the given error experience; otherwise LGTM

Comment thread security/jsLoader.ts Outdated
Comment on lines +120 to +121
amaro = await import('amaro').catch(() => null);
return amaro.transformSync(source, { mode: 'strip-only' }).code;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we okay with this throwing a TypeError: Cannot read properties of null when the package fails to load?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a real dependency now, and Chris removed the unnecessary catch, so I think we are good.

Comment thread security/jsLoader.ts Outdated
Co-authored-by: Chris Barber <chris@harperdb.io>
Copy link
Copy Markdown
Contributor

@cb1kenobi cb1kenobi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it!

@kriszyp kriszyp merged commit 940539d into main Mar 18, 2026
20 of 22 checks passed
@kriszyp kriszyp deleted the vm-load-typescript branch March 18, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants