Skip to content

chore: increase performance of legacy generators #505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ovflowd merged 1 commit into from
Dec 4, 2025
Merged

chore: increase performance of legacy generators #505

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/generators/legacy-html-all/index.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { join, resolve } from 'node:path';

import HTMLMinifier from '@minify-html/node';

import { getRemarkRehype } from '../../utils/remark.mjs';
import { getRemarkRehypeWithShiki } from '../../utils/remark.mjs';
import dropdowns from '../legacy-html/utils/buildDropdowns.mjs';
import tableOfContents from '../legacy-html/utils/tableOfContents.mjs';

Expand Down Expand Up @@ -49,7 +49,7 @@ export default {
const inputWithoutIndex = input.filter(entry => entry.api !== 'index');

// Gets a Remark Processor that parses Markdown to minified HTML
const remarkWithRehype = getRemarkRehype();
const remarkWithRehype = getRemarkRehypeWithShiki();

// Current directory path relative to the `index.mjs` file
// from the `legacy-html` generator, as all the assets are there
Expand Down
4 changes: 2 additions & 2 deletions src/generators/legacy-html/index.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import dropdowns from './utils/buildDropdowns.mjs';
import { safeCopy } from './utils/safeCopy.mjs';
import tableOfContents from './utils/tableOfContents.mjs';
import { groupNodesByModule } from '../../utils/generators.mjs';
import { getRemarkRehype } from '../../utils/remark.mjs';
import { getRemarkRehypeWithShiki } from '../../utils/remark.mjs';

/**
* @typedef {{
Expand Down Expand Up @@ -53,7 +53,7 @@ export default {
const generatedValues = [];

// Gets a Remark Processor that parses Markdown to minified HTML
const remarkRehypeProcessor = getRemarkRehype();
const remarkRehypeProcessor = getRemarkRehypeWithShiki();

const groupedModules = groupNodesByModule(input);

Expand Down
1 change: 1 addition & 0 deletions src/generators/legacy-json/index.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ export default {
const nodes = groupedModules.get(head.api);

const section = buildSection(head, nodes);

generatedValues.push(section);

return section;
Expand Down
4 changes: 3 additions & 1 deletion src/utils/highlighter.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,13 @@ function isCodeBlock(node) {
);
}

export const highlighter = await createHighlighter();
export const highlighter = await createHighlighter({ wasm: true });

/**
* Creates a HAST transformer for Shiki which is used for transforming our codeboxes
*
* @deprecated This is used only for the legacy-html generator, please use `@node-core/rehype-shiki` directly instead.
*
* @type {import('unified').Plugin}
*/
export default function rehypeShikiji() {
Expand Down
24 changes: 17 additions & 7 deletions src/utils/remark.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,24 @@ export const getRemark = () =>

/**
* Retrieves an instance of Remark configured to output stringified HTML code
* including parsing Code Boxes with syntax highlighting
*/
export const getRemarkRehype = () =>
unified()
.use(remarkParse)
// We make Rehype ignore existing HTML nodes (just the node itself, not its children)
// as these are nodes we manually created during the rehype process
// We also allow dangerous HTML to be passed through, since we have HTML within our Markdown
// and we trust the sources of the Markdown files
.use(remarkRehype, { allowDangerousHtml: true, passThrough })
// We allow dangerous HTML to be passed through, since we have HTML within our Markdown
// and we trust the sources of the Markdown files
.use(rehypeStringify, { allowDangerousHtml: true });

/**
* Retrieves an instance of Remark configured to output stringified HTML code
* including parsing Code Boxes with syntax highlighting
*/
export const getRemarkRehypeWithShiki = () =>
unified()
.use(remarkParse)
// We make Rehype ignore existing HTML nodes (just the node itself, not its children)
Expand All @@ -38,8 +53,6 @@ export const getRemarkRehype = () =>
.use(remarkRehype, { allowDangerousHtml: true, passThrough })
// This is a custom ad-hoc within the Shiki Rehype plugin, used to highlight code
// and transform them into HAST nodes
// @TODO: Get rid of @shikijis/rehype and use our own Rehype plugin for Shiki
// since we have CJS/ESM nodes. (Base off from the nodejs/nodejs.org repository)
.use(syntaxHighlighter)
// We allow dangerous HTML to be passed through, since we have HTML within our Markdown
// and we trust the sources of the Markdown files
Expand All @@ -58,10 +71,7 @@ export const getRemarkRecma = () =>
// as these are nodes we manually created during the generation process
// We also allow dangerous HTML to be passed through, since we have HTML within our Markdown
// and we trust the sources of the Markdown files
.use(remarkRehype, {
allowDangerousHtml: true,
passThrough,
})
.use(remarkRehype, { allowDangerousHtml: true, passThrough })
// Any `raw` HTML in the markdown must be converted to AST in order for Recma to understand it
.use(rehypeRaw, { passThrough })
.use(() => singletonShiki)
Expand Down
Loading