Skip to content

feat(cli): exit non-zero when error diagnostics are emitted#113

Merged
emmanuelnk merged 1 commit into
mainfrom
feat/cli-exit-nonzero-on-error-diagnostics
May 13, 2026
Merged

feat(cli): exit non-zero when error diagnostics are emitted#113
emmanuelnk merged 1 commit into
mainfrom
feat/cli-exit-nonzero-on-error-diagnostics

Conversation

@emmanuelnk
Copy link
Copy Markdown
Owner

@emmanuelnk emmanuelnk commented May 13, 2026

Summary

Closes #98.

The build CLI previously always exited with status 0 regardless of diagnostic severity, so pre-commit hooks (which often hide stdout) could silently let broken workflows through. This PR makes the CLI set process.exitCode = 1 when any diagnostic at error or fatal severity is emitted during a build, after configured diagnostics.rules have been applied.

A new diagnostics.failOnError config flag (defaults to true) lets users opt out for the pre-2.6.0 behaviour. Individual codes can still be downgraded via rules or suppressed at call sites via Diagnostics.suppress / suppressWarnings — those existing escape hatches behave exactly as before.

process.exitCode is set rather than calling process.exit(1) so any pending IO flushes cleanly before Node exits.

What a reviewer should look at

  • packages/cli/src/commands/diagnostics.tsConsoleDiagnosticsReporter now tracks a _hasErrors flag (exposed via a hasErrors getter) that flips inside emit() when the effective severity (post-rules) is error or fatal. Suppressed (off) diagnostics never flip it, and rule-downgraded errors don't either.
  • packages/cli/src/commands/build.ts — captures the reporter instance, and at the end of generateWorkflowFiles checks reporter.hasErrors && (config.diagnostics?.failOnError ?? true). Setting process.exitCode lets yargs and async work unwind naturally.
  • packages/cli/src/commands/types/build.ts — adds the failOnError?: boolean field to DiagnosticsConfig with a doc comment describing the default and the escape hatches.
  • packages/cli/src/commands/diagnostics.spec.ts — new hasErrors describe block covers: default-false, sub-error severities, error/fatal flip, rule downgrade keeps it false, rule upgrade flips it, rule off keeps it false, and stickiness (once flipped, stays flipped).
  • packages/cli/src/commands/build.integration.spec.ts — two layers of tests. The first uses __mocks__/{error,warning}-emitting.fixture.ts to drive importWorkflowFile end-to-end and verify the reporter state. The second creates a temp project (with a symlinked node_modules so the workspace lib resolves), process.chdirs into it, runs the real generateWorkflowFiles, and asserts process.exitCode for the error path, the warning-only path, and the failOnError: false opt-out.
  • __mocks__/*.fixture.ts — note the .fixture.ts suffix (not .wac.ts). The CLI's *.wac.ts glob shouldn't pick these fixtures up during the repo's own pre-commit build, otherwise the error-emitting fixture would fail every commit. The integration test that exercises the full CLI flow copies the fixture content into a temp wf.wac.ts so it gets discovered there.
  • docs/docs/guides/configuration.md — corrects the (previously inaccurate) claim that severity "error" "fails the build", adds a failOnError subsection, and links the two together.
  • docs/docs/guides/typed-actions.md — adds a "Build Exit Code" section right after the suppression docs so the workflow author sees exactly what error severity means for CI/pre-commit.

Test Plan

  • pnpm -r test — 302/302 pass (lib 97, cli 85, actions 120)
  • pnpm exec eslint packages/cli/src/commands/ — clean
  • pnpm -r build — clean
  • Pre-commit hook runs full test suite + production build on this branch — succeeds, the new fixtures are correctly skipped by *.wac.ts glob
  • Maintainer to spot-check the docs reads well on the rendered site

Backwards compatibility

This is a behavior change visible to anyone running the CLI: builds that previously succeeded silently with error-severity diagnostics will now exit non-zero. The escape hatch (diagnostics.failOnError: false) is one-line and is documented. Suggest a minor version bump (2.6.0).

Closes #98. The build CLI previously always exited with status 0
regardless of diagnostic severity, so pre-commit hooks that hide
stdout could silently let broken workflows through. The CLI now
sets process.exitCode = 1 when any diagnostic at error or fatal
severity (after applying configured rules) is emitted.

Users can downgrade specific codes via diagnostics.rules,
suppress at call sites via Diagnostics.suppress, or disable the
behaviour globally via the new diagnostics.failOnError config
flag (defaults to true).
@vercel
Copy link
Copy Markdown

vercel Bot commented May 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
github-actions-workflow-ts Ready Ready Preview, Comment May 13, 2026 1:57pm

@emmanuelnk emmanuelnk requested a review from bcheidemann May 13, 2026 13:58
@emmanuelnk emmanuelnk merged commit f53f1f2 into main May 13, 2026
11 checks passed
@emmanuelnk emmanuelnk deleted the feat/cli-exit-nonzero-on-error-diagnostics branch May 13, 2026 14:28
Copy link
Copy Markdown
Collaborator

@bcheidemann bcheidemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I'm a bit late to the party, as I see this has already merged.

I've left a couple comments but overall this looks great :) Nice work!

Lmk if you'd like me to pick up either of the suggested changes.

```json
{
"diagnostics": {
"failOnError": true
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some devs prefer to configure their tools to fail on warnings. I wonder if this should be:

"maxSeverity": "info" // fails on warnings and above

Or

"failOnSeverity": "warn" // fails on warnings and above

Comment on lines +35 to +41
/**
* Whether any diagnostic at `error` or `fatal` severity (after applying
* configured rules) has been emitted through this reporter.
*/
get hasErrors(): boolean {
return this._hasErrors
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this to the Diagnostics.DiagnosticsReporter interface? This would ensure this is implemented if we (or anyone else) ever implements another reporter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Diagnostics with severity of error or higher should cause the CLI to exit with non-zero code

2 participants