feat: include errors for projects when printing JSON graph [UNIFY-1102]

[snyk-tim]

Pull Request Submission Checklist

• Follows CONTRIBUTING guidelines
• Commit messages
  are release-note ready, emphasizing
  what was changed, not how.
• Includes detailed description of changes
• Contains risk assessment (Low | Medium | High)
• Highlights breaking API changes (if applicable)
• Links to automated tests covering new functionality
• Includes manual testing instructions (if necessary)
• Updates relevant GitBook documentation (PR link: ___)
• Includes product update to be announced in the next stable release notes

What does this PR do?

Currently, --print-effective-graph only outputs results for successfully handled project files detected during analysis, and excludes projects that encounter errors during detection. This PR introduces --print-effective-graph-with-errors, which includes those errors in the output.

The core behaviour remains unchanged, and the command continues to emit one line per project file found, whether processing succeeds or results in an error.

This change is required to surface errors to customers using the new Test flow.

Error JSON lines will not include the depGraph field and instead include an error field, shaped as an error catalog error in the JSON API format. Error lines will also omit the target field, which is not required in this context.

Error Catalog Integration

The new getOrCreateErrorCatalogError() helper function will take any Error type and return a ProblemError:

• If the underlying error is already a ProblemError, it is returned as is.
• If the error is a CustomError with an attached errorCatalog, that catalog error is used.
• Otherwise, the error message is wrapped in a GeneralCLIFailureError to return the error message as a general CLI ProblemError.

Why not add the errors to --print-effective-graph?

Several extensions rely on the output produced by the --print-effective-graph argument in this project. The cli-extension-dep-graph extension, in particular, consumes this output and is itself depended on by a number of other extensions. Including errors by default would require all downstream extensions to handle additional workflow.Data objects created to represent those errors, and they would likely fail without corresponding updates. For this reason, exposing errors as an opt-in option is the safer and less disruptive approach.

Where should the reviewer start?

The main change is in src/lib/snyk-test/run-test.ts, where error JSON lines are emitted when failedResults are present and the --print-effective-graph-with-errors argument is supplied.

How should this be manually tested?

Run the following command on an example project which is broken somehow.

snyk test --all-projects --print-effective-graph-with-errors

Compare the output to the current argument.

snyk test --all-projects --print-effective-graph

What's the product update that needs to be communicated to CLI users?

This is used internally by other CLI extensions, primarily cli-extension-dep-graph.

[snyk-tim]

The shouldPrintEffectiveDepGraph() check is used in multiple places and should remain true for both --print-effective-graph and --print-effective-graph-with-errors

[PeterSchafer]

Question: Is there a reason why we do always use this generic error code? Why don't we try to use additional information from the original error instance we received from the plugins. Choosing this hardcoded value here will make it more and more difficult to correctly report on errors in the future. It would even be better if the string would be empty by default, so that the decision of mapping it to generic can be done somewhere else.

I hope this makes sense. If you want we can chat a bit more. Maybe look at the CustomError on how we made it possible to transparently handle error catalog errors if available.

[snyk-tim]

I've updated this so that the error returned will always return a ProblemError.

• If the underlying error is already a ProblemError, it is returned as is.
• If the error is a CustomError with an attached errorCatalog, that catalog error is used.
• Otherwise, the error message is wrapped in a GeneralCLIFailureError to return the error message as a general CLI ProblemError.

[PeterSchafer]

Question: Are we really uncertain about the expected exit code?

[snyk-tim]

For the test we should expect 0. I've fixed this 🙏

[PeterSchafer]

Nice! Thank you!