yarn -> pnpm #253

sdqede · 2025-11-04T02:05:39Z

Key Changes

Replace Lerna scripts with pnpm native commands
Add custom workspace binary symlinking script for pnpm
Fix Jest module resolution with moduleNameMapper for workspace dependencies
Add missing devDependencies for TypeScript compilation
Update build scripts and Jest snapshots

Testing

✅ All local tests passing

CI/CD

workflows already use pnpm

…fig.json for TypeScript compilation, and preventing .js and .d.ts files from being generated in the src folder

Replace lerna run commands with pnpm -r to avoid NX dependency issues and simplify workspace script execution.

Replace symlink-workspace package with custom Node.js script to support pnpm workspace bin linking, as symlink-workspace doesn't work with pnpm's isolated node_modules structure.

Ensure TypeScript compiler uses correct config file to prevent output files from being generated in source directories instead of dist folder.

Add @launchql/logger, pg, and @types/pg to dependencies. Yarn automatically hoists dependencies to workspace root, allowing packages to access dependencies not declared in their package.json. pnpm requires explicit declarations, preventing phantom dependencies and ensuring better reproducibility

Add explicit dependencies to packages that were relying on yarn hoisting: - cli: @launchql/logger - url-domains: express, @types/express - introspectron: pg, @types/pg - pg-codegen: @launchql/logger, pg-cache, pg-env - pgsql-test: @launchql/logger Remove resolutions field from all packages (pnpm uses root overrides). Yarn hoists dependencies automatically, but pnpm requires explicit declarations in each package.json for better dependency isolation and reproducibility

- Add pg and @types/pg as devDependencies in packages/cli/package.json Required for TypeScript tests that import pg in pnpm's isolated structure - Fix module template build script to use 'tsc -p tsconfig.json' instead of 'tsc' Ensures consistent build behavior across generated modules - Update test snapshots to reflect corrected template output and file ordering

… pnpm - Add graphql@15.10.1 as devDependency in packages/launchql-gen/package.json - Required for TypeScript tests that import graphql in pnpm's isolated structure - GraphQL 15 includes built-in type definitions, no @types/graphql needed

…ith pnpm - Add @types/babel__generator@^7.6.8 as devDependency in packages/pg-codegen/package.json - Required for TypeScript tests that import @babel/generator in pnpm's isolated structure - Fixes TS7016 error: Could not find a declaration file for module '@babel/generator'

- Add glob@^11.0.2 and @types/glob@^8.1.0 as devDependencies in packages/s3-streamer/package.json - Required for TypeScript tests that import glob in pnpm's isolated structure - Update test snapshots to reflect hash value changes

…ependencies - Add moduleNameMapper configuration in packages/pgsql-test/jest.config.js - Maps @launchql/* packages to packages/*/dist directories for Jest resolution - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import @launchql/core and other workspace packages

- Add moduleNameMapper configuration in packages/cli/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pg-env and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Update test snapshots to match corrected template build script format

- Add moduleNameMapper configuration in packages/introspectron/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pgsql-test, graphile-test, pg-env, and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import workspace packages like pgsql-test and graphile-test

- Add moduleNameMapper configuration in packages/launchql-gen/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pgsql-test, graphile-test, pg-env, and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import graphile-test which depends on pgsql-test

- Add moduleNameMapper configuration in packages/pg-codegen/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pgsql-test, graphile-test, pg-env, and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import pgsql-test which depends on @launchql/logger

pyramation · 2025-11-05T01:27:10Z

packages/cli/__tests__/__snapshots__/init.install.test.ts.snap

+  "../../extensions/@webql/base32/revert/schemas/base32/procedures/decode.sql",
+  "../../extensions/@webql/base32/deploy/schemas/base32/schema.sql",
+  "../../extensions/@webql/base32/deploy/schemas/base32/procedures/encode.sql",
+  "../../extensions/@webql/base32/deploy/schemas/base32/procedures/decode.sql",


do you know why these got out of order?

I did some research, and found the order changed because glob.sync() in packages/cli/tests/init.install.test.ts doesn't guarantee file order—it depends on file system traversal, which differs between Yarn's flat node_modules and pnpm's symlinked structure.
I found out two options:
Option 1: Keep depth-first sort — matches yarn's apparent behavior but more complex
Option 2: Use alphabetical sort (.sort()) — deterministic but changes snapshot order
I prefer Option 2 for simplicity and reliability, but I'm happy to keep the original order if you prefer. Which do you prefer?

let's use option 2, I think you are right! :)

pyramation · 2025-11-05T01:28:35Z

packages/cli/jest.config.js

+    '^@launchql/logger$': '<rootDir>/../../packages/logger/dist',
+    '^@launchql/(.*)$': '<rootDir>/../../packages/$1/dist',
+    '^pg-env$': '<rootDir>/../../packages/pg-env/dist',
+    '^pg-cache$': '<rootDir>/../../packages/pg-cache/dist',


are these absolutely necessary?
Shouldn't this be picked up by pnpm workspace?

I never had to do this in other packages, it seems that it could be cruft.

I tried workspace:^ and it worked, I will fix those moduleNameMapper issue

pyramation · 2025-11-05T01:30:15Z

packages/introspectron/jest.config.js

+    '^pgsql-test$': '<rootDir>/../../packages/pgsql-test/dist',
+    '^graphile-test$': '<rootDir>/../../packages/graphile-test/dist',
+    '^pg-env$': '<rootDir>/../../packages/pg-env/dist',
+    '^pg-cache$': '<rootDir>/../../packages/pg-cache/dist',


same comment as before, this seems very very unmaintainble.

see reply above about workspace:^

pyramation · 2025-11-05T01:30:55Z

packages/launchql-gen/jest.config.js

+    '^pgsql-test$': '<rootDir>/../../packages/pgsql-test/dist',
+    '^graphile-test$': '<rootDir>/../../packages/graphile-test/dist',
+    '^pg-env$': '<rootDir>/../../packages/pg-env/dist',
+    '^pg-cache$': '<rootDir>/../../packages/pg-cache/dist',


this one also

maybe we use the workspace:* syntax and pnpm resolves this

see reply above about workspace:^

pyramation · 2025-11-05T01:31:17Z

packages/pg-codegen/jest.config.js

+    '^graphile-test$': '<rootDir>/../../packages/graphile-test/dist',
+    '^pg-env$': '<rootDir>/../../packages/pg-env/dist',
+    '^pg-cache$': '<rootDir>/../../packages/pg-cache/dist',
+  },


same here, we should NOT have moduleNameMapper

see reply above about workspace:^

pyramation · 2025-11-05T01:31:59Z

packages/pgsql-test/jest.config.js

+  modulePathIgnorePatterns: ['dist/*'],
+  moduleNameMapper: {
+    '^@launchql/logger$': '<rootDir>/../../packages/logger/dist',
+    '^@launchql/(.*)$': '<rootDir>/../../packages/$1/dist',


this one too, can you explain what this was for?

perhaps, again, let's research if workspace:* syntax solves this instead of referencing the package versions directly.

see reply above about workspace:^

pyramation · 2025-11-05T01:33:12Z

packages/s3-streamer/__tests__/__snapshots__/uploads.test.ts.snap

-      "etag": "2524b9c1caad97610a5149886a7e793f",
-      "sha": "62c8208c3f148ada7e1a1a3b8972ac5898dfbf89",
-      "uuid": "89250a3c-a794-5d26-8ee4-d692dff9fc35",
+      "etag": "9086b336429e90603656255396a5d462",


this snap getting updated is pretty serious. Do you know why these are changing?

This needs to be documented, or likely reverted back.

see reply above about glob.sync()

so the hashes are just getting moved around, but the hashes themselves aren't changing?

pyramation · 2025-11-05T01:35:41Z

packages/templatizer/src/generated/workspace.ts

    "lint": "lerna run lint --parallel",
    "symlink": "symlink-workspace --logLevel error",
-    "postinstall": "yarn symlink"
+    "postinstall": "pnpm run symlink"


we probably do not need to symlink anymore. Assuming we use workspace:* syntax, pnpm does the symlinking to simulate local modules

pnpm workspaces should solve this

yep, symlink is automatically run by pnpm when you do pnpm install anyways, this is unnecessary. It should be handled with workspace:* protocol

"postinstall": "pnpm run symlink" deleted

pyramation · 2025-11-05T01:38:15Z

scripts/symlink-workspace.js

@@ -0,0 +1,41 @@
+const fs = require('fs');
+const path = require('path');
+


this entire script should be deleted

this file has been deleted

- Replace version-based dependencies with workspace:^ protocol in packages/cli/package.json - Remove moduleNameMapper from Jest config as pnpm workspace symlinks handle module resolution - Add .sort() to test file lists for deterministic snapshot ordering This leverages pnpm's native workspace protocol for better dependency resolution and simplifies Jest configuration by relying on pnpm's symlink structure.

…ve Jest moduleNameMapper - Replace workspace dependencies with workspace:^ protocol in: * packages/pgsql-test (dependencies) * packages/launchql-gen (devDependencies) * packages/pg-codegen (dependencies) * packages/introspectron (devDependencies) - Remove moduleNameMapper from Jest configs: * packages/pgsql-test/jest.config.js * packages/launchql-gen/jest.config.js * packages/pg-codegen/jest.config.js * packages/introspectron/jest.config.js This completes the migration to pnpm's native workspace protocol, allowing Jest to resolve workspace packages via pnpm's symlink structure without manual module path mappings.

- Remove symlink and postinstall scripts from root package.json - Remove symlink-workspace dependency from all package.json files - Replace lerna commands with pnpm commands in templates - Delete scripts/symlink-workspace.js (no longer needed) - Update workspace templates to use pnpm native workspace commands pnpm automatically handles workspace package linking via workspace:^ protocol, so manual symlink scripts are unnecessary.

Make the chmod step for lql binary conditional to prevent CI failures when node_modules/.bin/lql doesn't exist. pnpm workspace linking may not always create the binary symlink in node_modules/.bin, but pnpm exec still works correctly for workspace packages

sdqede added 22 commits October 30, 2025 17:27

wip: yarn to pnpm

6acf4ed

chore: migrate boilerplate configs to pnpm

3ad8e3a

chore: enable pnpm workspaces in lerna

eda8d0e

imported pnpm-lock.yaml from yarn.lock

7e2df23

keep yarn.lock for testing

da26bd5

refactor: update build scripts in package.json files to specify tscon…

edc3a2a

…fig.json for TypeScript compilation, and preventing .js and .d.ts files from being generated in the src folder

chore: replace lerna commands with pnpm in root scripts

93b1902

Replace lerna run commands with pnpm -r to avoid NX dependency issues and simplify workspace script execution.

chore: replace symlink-workspace with custom script

2bbe7ea

Replace symlink-workspace package with custom Node.js script to support pnpm workspace bin linking, as symlink-workspace doesn't work with pnpm's isolated node_modules structure.

fix: add explicit tsconfig.json path to build:dev script

e47571d

Ensure TypeScript compiler uses correct config file to prevent output files from being generated in source directories instead of dist folder.

reinstall packages

6d41657

deleted yarn.lock files

0a4bc3d

pyramation reviewed Nov 5, 2025

View reviewed changes

sdqede added 4 commits November 5, 2025 17:18

pyramation changed the base branch from main to feat/pnpm November 5, 2025 09:11

pyramation merged commit 88e58f2 into constructive-io:feat/pnpm Nov 5, 2025
5 checks passed

		@@ -0,0 +1,41 @@
		const fs = require('fs');
		const path = require('path');

yarn -> pnpm #253

yarn -> pnpm #253

Uh oh!

Conversation

sdqede commented Nov 4, 2025

Key Changes

Testing

CI/CD

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sdqede Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sdqede Nov 5, 2025 •

edited

Loading