Skip to content

perf(build): replace specs glob with fs reads#2959

Open
atharvadeosthale wants to merge 4 commits intomainfrom
website-optimize
Open

perf(build): replace specs glob with fs reads#2959
atharvadeosthale wants to merge 4 commits intomainfrom
website-optimize

Conversation

@atharvadeosthale
Copy link
Copy Markdown
Member

Summary

The 11 import.meta.glob calls in specs.ts (one per Appwrite version) expanded into ~58k lazy import() statements, producing 49,123 server chunks (500 MB) and pushing peak build RSS to 7 GB. The matched .md and .json files are static text — there's no reason to route them through Vite's module graph.

This PR resolves @appwrite.io/specs at runtime via createRequire and reads examples + OpenAPI specs with fs.readFile. The package moves from devDependencies to dependencies so it survives bun install --production and ships to the final image. vite-plugin-dynamic-import is removed (no longer needed).

Build impact

metric before after delta
wall time 192s 113s −41%
peak RSS 7.0 GB 5.6 GB −21%
SSR modules 50,269 4,340 −91%
server chunks 49,123 3,194 −93%
build/server/ size 501 MB 80 MB −84%
final image (build/server + specs in node_modules) 501 MB 376 MB −125 MB

Test plan

  • Build runs clean
  • /docs/references/cloud/server-nodejs/databases renders code examples (Node.js)
  • /docs/references/1.5.x/server-python/storage renders (Python, older version)
  • /docs/references/1.7.x/client-android-kotlin/account renders (exercises the Android special path in loadExample)

The 11 import.meta.glob calls in specs.ts expanded into 58k lazy import
statements, producing 49,123 server chunks (500 MB) and pushing peak
build RSS to 7 GB. The .md and .json files were being treated as JS
modules with full transform/sourcemap/chunk overhead, when they are
just static text.

Resolve @appwrite.io/specs at runtime via createRequire and load
examples + OpenAPI specs with fs.readFile. Move the package from
devDependencies to dependencies so it survives bun install --production
and ships to the final image.

Build wall: 192s -> 113s. Peak RSS: 7.0 GB -> 5.6 GB. Server chunks:
49,123 -> 3,194. Final image net ~125 MB smaller (build/server drops
421 MB, node_modules adds 296 MB).

Drops vite-plugin-dynamic-import which is no longer needed.
@appwrite
Copy link
Copy Markdown

appwrite Bot commented May 4, 2026

Appwrite Website

Project ID: 69d7efb00023389e8d27

Sites (1)
Site Status Logs Preview QR
 website
69d7f2670014e24571ca
Building Building View Logs Preview URL QR Code

Website (appwrite/website)

Project ID: 684969cb000a2f6c0a02

Sites (1)
Site Status Logs Preview QR
 website
68496a17000f03d62013
Queued Queued View Logs Preview URL QR Code


Tip

Trigger functions via HTTP, SDKs, events, webhooks, or scheduled cron jobs

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 4, 2026

Greptile Summary

  • Replaces 11 import.meta.glob calls (which produced ~58k lazy imports, 49k server chunks, and 501 MB build output) with direct fs.readFile + createRequire-based resolution, cutting build time by 41% and server bundle size by 84%.
  • Adds input validation (assertValidVersion, assertValidPlatform) to guard filesystem paths derived from URL route params, and an apiCache Map to avoid re-parsing large OpenAPI JSON files on every request.
  • The apiCache key is ${version}|${platform} but the file actually read is keyed on mode (server/client/console), so multiple platform variants of the same mode/version get separate cache entries for identical file contents — the fix is to key by ${version}|${mode} instead.

Confidence Score: 4/5

Safe to merge after fixing the cache key; all other changes are well-structured and correct.

One P1 defect: the apiCache key uses the full platform string rather than the derived mode, causing the same large JSON to be parsed and stored once per platform variant instead of once per mode. The rest of the PR — glob removal, fs reads, path validation, parallel example loading, and build-time copy — is correct and a clear improvement.

src/routes/docs/references/[version]/[platform]/[service]/specs.ts — specifically the apiCache key on line 414.

Important Files Changed

Filename Overview
src/routes/docs/references/[version]/[platform]/[service]/specs.ts Replaces import.meta.glob with fs.readFile; adds path allowlist guards, an in-memory apiCache, and parallelised example loading — but apiCache keyed on full platform string causes redundant parses for same-mode platforms (P1).
scripts/build.js New post-build step that copies specs data into build/_specs_data so the deployment artifact is self-contained; logic is straightforward and correct.
package.json Moves @appwrite.io/specs from devDependencies to dependencies and removes vite-plugin-dynamic-import.
vite.config.ts Removes the now-unneeded dynamicImport plugin; no other changes.

Reviews (4): Last reviewed commit: "move specs out of node_modules so doesnt..." | Re-trigger Greptile

Comment thread src/routes/docs/references/[version]/[platform]/[service]/specs.ts
Comment thread src/routes/docs/references/[version]/[platform]/[service]/specs.ts Outdated
Comment thread src/routes/docs/references/[version]/[platform]/[service]/specs.ts Outdated
…RL params

- Cache parsed OpenAPI documents in a module-scope Map keyed by
  version|platform. Restores the implicit caching the old import.meta.glob
  path got from Node's ESM module cache, avoids re-parsing multi-MB JSON
  on every request.

- Replace the serial loadExample await loop with Promise.all over the
  prepared method list. Cuts I/O latency on services with many endpoints.

- Validate version and platform against allowlists from references.ts
  before any path.join. URL segments shouldn't reach a filesystem read
  unguarded; a lone .. segment could escape examples/ otherwise.
Comment thread src/routes/docs/references/[version]/[platform]/[service]/specs.ts
getApi only interpolates a derived literal mode (server/client/console)
into the JSON path, never platform itself, so the platform allowlist
there was rejecting valid internal callers (model-markdown.ts and
models/[model]/+page.server.ts hardcode 'console-web' which isn't in
the public Platform enum).

Move the guard to getService where the raw platform value is actually
interpolated into examples/${version}/${platform}/... — that's the
real filesystem boundary. URL paths reaching getService still go
through the [platform] route segment whose values come from the
Platform enum, so the check stays correct there.

Fixes 69 failing redirect tests for /docs/references/cloud/models/*.
Comment on lines +414 to 422
const cacheKey = `${version}|${platform}`;
const cached = apiCache.get(cacheKey);
if (cached) {
return cached;
}

const isClient = platform.startsWith('client-');
const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console';
const filename = `open-api3-${version}-${mode}.json`;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 The apiCache key includes the full platform string (e.g. server-nodejs, server-python), but the JSON file actually read is determined by mode — one of three fixed literals (server, client, console). Every server-side platform for the same version maps to the same open-api3-${version}-server.json file, so each distinct platform string creates a separate cache entry that parses and stores an identical multi-MB document. On a live SSR server where different users request server-nodejs, server-python, server-php, etc. concurrently, the same JSON is parsed and allocated once per platform variant — partially undoing the memory benefit of the cache. The key should be ${version}|${mode} so all server platforms share one entry.

Suggested change
const cacheKey = `${version}|${platform}`;
const cached = apiCache.get(cacheKey);
if (cached) {
return cached;
}
const isClient = platform.startsWith('client-');
const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console';
const filename = `open-api3-${version}-${mode}.json`;
const isClient = platform.startsWith('client-');
const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console';
const cacheKey = `${version}|${mode}`;
const cached = apiCache.get(cacheKey);
if (cached) {
return cached;
}
const filename = `open-api3-${version}-${mode}.json`;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants