-
Notifications
You must be signed in to change notification settings - Fork 0
Parallel builds #36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallel builds #36
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❇️ CodePress Review Summary
👋 Hey team,
Overall the changes look solid, but I spotted 5 must-fix issues and left 0 helpful notes inline.
Here's the quick rundown:
✅ Decision: APPROVE
The change is backward-compatible and scoped to the build script. Potential edge cases (invalid --parallel values, error handling) are non-blocking and can be iterated on without risking production runtime.
🚧 Needs a bit of love
The required findings center on unsafe parsing and concurrency management around the --parallel flag. Invalid, missing, or negative values are silently coerced to unlimited parallelism, and an early band-selection step misclassifies CLI flags, causing the script to exit before options are parsed. The concurrency runner lacks robust rejection handling, leading to unhandled promise rejections, leaked tasks, and premature termination. Collectively these issues risk resource exhaustion and CI instability; the code should validate the flag input, fix the flag/band parsing interaction, and ensure the pool always attaches rejection handlers and cleans up in finally.
scripts/build-swc.mjs
Outdated
| break; | ||
| case "--parallel": | ||
| case "-p": | ||
| args.parallel = Number(argv[++i]) || 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 REQUIRED: Silently coercing invalid or missing --parallel values to 0 (unlimited) is dangerous. For example, "-p foo" or a missing value makes Number(...) NaN, which falls back to 0 and triggers unbounded parallelism. Negative values (e.g., "-p -1") also result in unlimited since runWithConcurrency treats <= 0 as unlimited. This can overwhelm local/CI resources and cause instability. Please validate the flag and error out on invalid input instead of defaulting to unlimited.
| args.parallel = Number(argv[++i]) || 0; | |
| case "--parallel": | |
| case "-p": { | |
| const raw = argv[++i]; | |
| const n = Number(raw); | |
| if (raw == null || !Number.isFinite(n) || !Number.isInteger(n) || n < 0) { | |
| console.error(`[codepress] --parallel expects a non-negative integer (0 = unlimited); received: "${raw ?? ""}"`); | |
| process.exit(1); | |
| } | |
| args.parallel = n; // 0 = unlimited; >0 = bounded | |
| break; | |
| } |
scripts/build-swc.mjs
Outdated
| const results = []; | ||
| const executing = new Set(); | ||
| for (const task of tasks) { | ||
| const p = task().then((r) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 REQUIRED: Rejection path is not handled: executing.delete(p) only runs on fulfillment, and no rejection handler is attached to p at creation time. This can lead to:
• Dangling entries in executing on failure, and
• A race where a fast-rejecting task triggers an unhandledRejection (especially when total tasks < concurrency and no Promise.race runs before loop end) in Node, which can terminate the process in newer Node versions.
To make this safe and fail-fast without leaking, attach a handler immediately and delete from executing on both resolve and reject.
| const p = task().then((r) => { | |
| const p = task().finally(() => { | |
| executing.delete(p); | |
| }); |
| `Examples:\n` + | ||
| ` node scripts/build-swc.mjs --next 15.4.0\n` + | ||
| ` node scripts/build-swc.mjs --band v26 --target wasip1\n` + | ||
| ` node scripts/build-swc.mjs --parallel 2 # Build 2 bands at a time\n` + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 REQUIRED: runWithConcurrency lacks rejection handling for in-flight tasks. If any task rejects while throttling, await Promise.race(executing) throws and the function returns early, leaving other executing promises without rejection handlers. This can produce unhandled promise rejections and leave spawned cargo processes running after the script exits (critical stability issue).
Evidence: p is created with task().then(...delete...) only; no .catch/.finally on rejection. The loop awaits Promise.race(executing) without try/catch. On first rejection, execution unwinds before reaching the final Promise.all(results), so other rejecting promises may be unhandled.
Required: ensure each task always has a rejection handler and that the pool drains correctly. Use .finally to delete from the set, and catch the race to allow the loop to continue attaching handlers; the eventual Promise.all will still surface the error.
| ` node scripts/build-swc.mjs --parallel 2 # Build 2 bands at a time\n` + | |
| async function runWithConcurrency(tasks, concurrency) { | |
| if (concurrency <= 0) { | |
| return Promise.all(tasks.map((fn) => fn())); | |
| } | |
| const results = []; | |
| const executing = new Set(); | |
| for (const task of tasks) { | |
| const p = Promise.resolve() | |
| .then(task) | |
| .finally(() => { | |
| executing.delete(p); | |
| }); | |
| results.push(p); | |
| executing.add(p); | |
| if (executing.size >= concurrency) { | |
| try { | |
| await Promise.race(executing); | |
| } catch { | |
| // Swallow here; final Promise.all(results) will reject with the first error. | |
| } | |
| } | |
| } | |
| return Promise.all(results); | |
| } |
scripts/build-swc.mjs
Outdated
| next: undefined, | ||
| band: undefined, | ||
| target: undefined, | ||
| parallel: undefined, // number of concurrent builds, 0 = unlimited |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 REQUIRED: Blocking issues with the new --parallel flag and concurrency handling:
1) CLI flag parsing conflicts with early band selection: the top-of-file band selection treats all argv tokens as band IDs and exits on "unknown band(s)". Running with flags like "--parallel 2" will trigger that early exit before parseArgs runs, breaking the new option. Fix by ignoring argv entries that are flags (start with "-") and/or only considering argv entries that match validIds when computing requestedIds, or move band selection after parseArgs.
2) Unsafe fallback to unlimited concurrency: parseArgs sets args.parallel = Number(argv[++i]) || 0. Any invalid/missing value (NaN) or negative will be interpreted as 0 (unlimited), which can overwhelm machines. Validate explicitly: accept only integers ≥ 0, treat NaN/invalid as an error, and do not coerce invalid values to 0.
3) Concurrency runner lacks rejection cleanup and may cause unhandled rejections: runWithConcurrency deletes from executing only on resolve; on rejection the promise remains in the set and Promise.race will reject immediately, aborting early and potentially leaving other started tasks to reject unobserved. Ensure removal on both resolve and reject (e.g., then(..., ...) or finally) and either use Promise.allSettled or attach per-task catch to prevent late unhandled rejections in both the limited and "unlimited" branches.
These issues can cause immediate CLI breakage and/or uncontrolled resource usage and unstable failure behavior in CI, and should be fixed before merging.
Add parallel band building to SWC build script
This PR introduces a --parallel/-p option to the SWC build script, enabling configurable concurrent builds of bands, including an unlimited option. It updates CLI parsing/help, adds a concurrency runner, and preserves default sequential behavior when not specified.
Key Changes:
--parallel/-pCLI argument and help textrunWithConcurrency(tasks, concurrency)for bounded/unlimited parallel executionReview Notes:
--parallelinputs currently coerce to 0 (unlimited)