Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions tools/sbom-diff-and-risk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,10 @@ Offline `stale_package` evaluation is intentionally deferred. When enrichment is

## Output Formats

- `report.json`
- `report.md`
- `report.sarif`
- `report.json`
- `summary.json` when `--summary-json` is provided
- `report.md`
- `report.sarif`

## Install

Expand All @@ -97,11 +98,12 @@ Generate reports from the bundled CycloneDX example inputs:
```bash
sbom-diff-risk compare \
--before examples/cdx_before.json \
--after examples/cdx_after.json \
--format auto \
--out-json outputs/report.json \
--out-md outputs/report.md
```
--after examples/cdx_after.json \
--format auto \
--out-json outputs/report.json \
--summary-json outputs/summary.json \
--out-md outputs/report.md
```

Generate reports from the `requirements.txt` examples:

Expand Down Expand Up @@ -175,6 +177,8 @@ Offline mode remains the default. No network access occurs unless `--enrich-pypi

`--summary-json PATH` writes only the stable `report.json["summary"]` object for compact machine consumption. It uses the same summary schema as the full JSON report.

The checked-in [examples/sample-summary.json](examples/sample-summary.json) artifact is generated from the bundled CycloneDX example with `--summary-json outputs/summary.json` and matches the `summary` object in [examples/sample-report.json](examples/sample-report.json).

## Dependency Provenance Analysis (Opt-in)

This section is about analyzing third-party package provenance signals. It is not about verifying the `sbom-diff-and-risk` tool's own release artifacts.
Expand Down Expand Up @@ -268,6 +272,7 @@ The [examples/](examples/) directory includes:
- provenance-aware policy examples at `examples/policy-provenance-minimal.yml` and `examples/policy-provenance-strict.yml`
- a Scorecard-aware policy example at `examples/policy-scorecard-minimal.yml`
- a sample pass JSON report at [sample-report.json](examples/sample-report.json)
- a sample summary-only JSON artifact at [sample-summary.json](examples/sample-summary.json)
- a sample pass Markdown report at [sample-report.md](examples/sample-report.md)
- sample policy-warn reports at [sample-policy-warn-report.json](examples/sample-policy-warn-report.json) and [sample-policy-warn-report.md](examples/sample-policy-warn-report.md)
- sample policy-fail reports at [sample-policy-fail-report.json](examples/sample-policy-fail-report.json) and [sample-policy-fail-report.md](examples/sample-policy-fail-report.md)
Expand Down
2 changes: 1 addition & 1 deletion tools/sbom-diff-and-risk/docs/report-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ When provenance policy fields are relevant, reports may also include `provenance

## Summary contract

`summary` is the stable, compact entry point for automation that needs counts without walking the full report. The `--summary-json PATH` CLI option writes only this stable `report.json["summary"]` object.
`summary` is the stable, compact entry point for automation that needs counts without walking the full report. The `--summary-json PATH` CLI option writes only this stable `report.json["summary"]` object. The checked-in [../examples/sample-summary.json](../examples/sample-summary.json) artifact is the summary-only output for the default CycloneDX example and matches the `summary` object in [../examples/sample-report.json](../examples/sample-report.json).

Base `summary` fields:

Expand Down
33 changes: 19 additions & 14 deletions tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,27 +28,32 @@ python -m pip install -e .[dev]
Generate the default CycloneDX example reports:

```powershell
sbom-diff-risk compare `
--before examples/cdx_before.json `
--after examples/cdx_after.json `
--format auto `
--out-json outputs/report.json `
--out-md outputs/report.md
```
sbom-diff-risk compare `
--before examples/cdx_before.json `
--after examples/cdx_after.json `
--format auto `
--out-json outputs/report.json `
--summary-json outputs/summary.json `
--out-md outputs/report.md
```

Expected output files:

- `outputs/report.json`
- `outputs/report.md`
- `outputs/report.json`
- `outputs/summary.json`
- `outputs/report.md`

Compare the outputs against the checked-in sample reports:

```powershell
Compare-Object (Get-Content examples/sample-report.json) (Get-Content outputs/report.json)
Compare-Object (Get-Content examples/sample-report.md) (Get-Content outputs/report.md)
```

No differences means the sample path reproduced the committed example output.
Compare-Object (Get-Content examples/sample-report.json) (Get-Content outputs/report.json)
Compare-Object (Get-Content examples/sample-summary.json) (Get-Content outputs/summary.json)
Compare-Object (Get-Content examples/sample-report.md) (Get-Content outputs/report.md)
```

No differences means the sample path reproduced the committed example output.

`examples/sample-summary.json` is the summary-only artifact for the same run and is expected to match `examples/sample-report.json`'s `summary` object.

Generate the strict-policy SARIF sample:

Expand Down
14 changes: 14 additions & 0 deletions tools/sbom-diff-and-risk/examples/sample-summary.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"added": 1,
"removed": 0,
"changed": 1,
"risk_counts": {
"new_package": 1,
"major_upgrade": 0,
"version_change_unclassified": 1,
"unknown_license": 0,
"stale_package": 0,
"suspicious_source": 0,
"not_evaluated": 2
}
}
12 changes: 11 additions & 1 deletion tools/sbom-diff-and-risk/tests/test_reports.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from sbom_diff_risk.policy_models import PolicyConfig
from sbom_diff_risk.policy_parser import build_policy
from sbom_diff_risk.normalize import normalize_input
from sbom_diff_risk.report_json import render_report_json
from sbom_diff_risk.report_json import render_report_json, render_summary_json
from sbom_diff_risk.report_md import render_report_markdown
from sbom_diff_risk.risk import evaluate_risks, summarize_risks

Expand All @@ -32,6 +32,16 @@ def test_report_json_matches_cyclonedx_golden_pass() -> None:
assert rendered == expected


def test_summary_json_matches_cyclonedx_golden_pass() -> None:
report = _build_report("cdx_before.json", "cdx_after.json")

rendered = render_summary_json(report)
expected = _read_example("sample-summary.json")

assert rendered == expected
assert json.loads(rendered) == json.loads(_read_example("sample-report.json"))["summary"]


def test_report_markdown_matches_cyclonedx_golden_pass() -> None:
report = _build_report("cdx_before.json", "cdx_after.json")

Expand Down