Skip to content

Commit fe6eb2d

Browse files
authored
[codex] Add optional policy JSON output
1 parent daf6dbf commit fe6eb2d

7 files changed

Lines changed: 230 additions & 16 deletions

File tree

tools/sbom-diff-and-risk/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,7 @@ Offline `stale_package` evaluation is intentionally deferred. When enrichment is
104104

105105
- `report.json`
106106
- `summary.json` when `--summary-json` is provided
107+
- `policy.json` when `--policy-json` is provided
107108
- `report.md`
108109
- `report.sarif`
109110

@@ -183,6 +184,7 @@ sbom-diff-risk compare \
183184
- `--pyproject-group name`
184185
- `--out-json path`
185186
- `--summary-json path`
187+
- `--policy-json path`
186188
- `--out-md path`
187189
- `--out-sarif path`
188190
- `--policy path`
@@ -203,6 +205,12 @@ The checked-in [examples/sample-summary.json](examples/sample-summary.json) arti
203205

204206
For CI dashboard, job-summary, and local-threshold examples, see [docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md).
205207

208+
`--policy-json PATH` writes only policy-related JSON sections from the full
209+
report. It includes `policy_evaluation`, policy finding lists, `rule_catalog`,
210+
and `summary.policy` when policy evaluation is applied. For CI job-summary
211+
examples, see
212+
[docs/policy-decision-ci-cookbook.md](docs/policy-decision-ci-cookbook.md).
213+
206214
## Dependency Provenance Analysis (Opt-in)
207215

208216
This section is about analyzing third-party package provenance signals. It is not about verifying the `sbom-diff-and-risk` tool's own release artifacts.

tools/sbom-diff-and-risk/docs/policy-decision-ci-cookbook.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# Policy decision CI cookbook
22

3-
This page shows how to consume policy decision explanation fields from
4-
`report.json` in CI without changing the `sbom-diff-risk` analysis model.
3+
This page shows how to consume policy decision explanation fields from the
4+
`--policy-json PATH` sidecar in CI without changing the `sbom-diff-risk`
5+
analysis model.
56

67
Use this when a repository wants a small job summary that explains local policy
78
blocks, warnings, or suppressions in machine-readable terms.
@@ -13,24 +14,25 @@ sbom-diff-risk compare \
1314
--before examples/cdx_before.json \
1415
--after examples/cdx_after.json \
1516
--policy examples/policy-strict.yml \
16-
--out-json outputs/policy-report.json
17+
--out-json outputs/report.json \
18+
--policy-json outputs/policy.json
1719
```
1820

1921
The strict example policy can make the command return a policy failure exit
20-
code. In CI, keep the generated `outputs/policy-report.json` artifact so the
21-
policy decision metadata remains available for review.
22+
code. In CI, keep the generated `outputs/policy.json` artifact so the policy
23+
decision metadata remains available for review.
2224

2325
## Python consumer
2426

25-
This example reads the full JSON report, prints compact policy status, and then
26-
prints the stable explanation fields for blocking and warning findings.
27+
This example reads the policy-only JSON sidecar, prints compact policy status,
28+
and then prints the stable explanation fields for blocking and warning findings.
2729

2830
```python
2931
import json
3032
from pathlib import Path
3133

3234
report = json.loads(
33-
Path("outputs/policy-report.json").read_text(encoding="utf-8")
35+
Path("outputs/policy.json").read_text(encoding="utf-8")
3436
)
3537

3638
policy = report.get("summary", {}).get("policy")
@@ -73,10 +75,10 @@ tool. The snippet does not create a new package safety verdict.
7375
## PowerShell consumer
7476

7577
This example uses `ConvertFrom-Json` to print the same policy status and
76-
explanation fields.
78+
explanation fields from the policy-only sidecar.
7779

7880
```powershell
79-
$report = Get-Content outputs/policy-report.json -Raw | ConvertFrom-Json
81+
$report = Get-Content outputs/policy.json -Raw | ConvertFrom-Json
8082
$policy = $report.summary.policy
8183
8284
if ($null -eq $policy) {

tools/sbom-diff-and-risk/docs/report-schema.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,18 @@ For reviewer-facing examples and interpretation guidance, see
7676
consumer snippets, see
7777
[policy-decision-ci-cookbook.md](policy-decision-ci-cookbook.md).
7878

79+
The `--policy-json PATH` CLI option writes a policy-only JSON sidecar using the
80+
same policy-related sections from the full JSON report:
81+
82+
- `policy_evaluation`
83+
- `blocking_findings`
84+
- `warning_findings`
85+
- `suppressed_findings`
86+
- `rule_catalog`
87+
- `summary.policy` when policy evaluation is applied
88+
- `provenance_policy` and `provenance_policy_impact` when provenance policy
89+
fields are relevant
90+
7991
## Summary contract
8092

8193
`summary` is the stable, compact entry point for automation that needs counts

tools/sbom-diff-and-risk/src/sbom_diff_risk/cli.py

Lines changed: 34 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
from .policy_evaluator import evaluate_policy
1414
from .policy_parser import build_policy
1515
from .presentation import effective_policy_evaluation, summarize_violations_by_rule
16-
from .report_json import render_report_json, render_summary_json
16+
from .report_json import render_policy_json, render_report_json, render_summary_json
1717
from .report_md import render_report_markdown
1818
from .report_sarif import render_report_sarif_output
1919
from .risk import evaluate_risks, summarize_risks
@@ -31,7 +31,10 @@ def build_parser() -> argparse.ArgumentParser:
3131
"compare",
3232
help="Compare two dependency inputs and write JSON and/or Markdown reports.",
3333
description="Compare two local dependency inputs and emit deterministic reports.",
34-
epilog="Exit codes: 0 = success/no blocking violations, 1 = blocking policy violations, 2 = usage/parse/runtime error.",
34+
epilog=(
35+
"Exit codes: 0 = success/no blocking violations, "
36+
"1 = blocking policy violations, 2 = usage/parse/runtime error."
37+
),
3538
)
3639
compare.add_argument("--before", type=Path, required=True, help="Path to the before input.")
3740
compare.add_argument("--after", type=Path, required=True, help="Path to the after input.")
@@ -59,7 +62,18 @@ def build_parser() -> argparse.ArgumentParser:
5962
help="Select a PEP 735 [dependency-groups] group when a compared input is pyproject.toml.",
6063
)
6164
compare.add_argument("--out-json", type=Path, default=None, help="Write a JSON report to this path.")
62-
compare.add_argument("--summary-json", type=Path, default=None, help="Write the stable JSON summary object to this path.")
65+
compare.add_argument(
66+
"--summary-json",
67+
type=Path,
68+
default=None,
69+
help="Write the stable JSON summary object to this path.",
70+
)
71+
compare.add_argument(
72+
"--policy-json",
73+
type=Path,
74+
default=None,
75+
help="Write policy evaluation and policy finding JSON sections to this path.",
76+
)
6377
compare.add_argument("--out-md", type=Path, default=None, help="Write a Markdown report to this path.")
6478
compare.add_argument(
6579
"--out-sarif",
@@ -91,7 +105,10 @@ def build_parser() -> argparse.ArgumentParser:
91105
compare.add_argument(
92106
"--enrich-pypi",
93107
action="store_true",
94-
help="Opt-in PyPI provenance and integrity enrichment. Default behavior remains offline with no network access.",
108+
help=(
109+
"Opt-in PyPI provenance and integrity enrichment. "
110+
"Default behavior remains offline with no network access."
111+
),
95112
)
96113
compare.add_argument(
97114
"--pypi-timeout",
@@ -139,8 +156,17 @@ def run_compare(args: argparse.Namespace) -> int:
139156
scorecard_timeout = getattr(args, "scorecard_timeout", DEFAULT_SCORECARD_TIMEOUT_SECONDS)
140157

141158
summary_json = getattr(args, "summary_json", None)
142-
if args.out_json is None and summary_json is None and args.out_md is None and args.out_sarif is None:
143-
raise ValueError("at least one of --out-json, --summary-json, --out-md, or --out-sarif must be provided")
159+
policy_json = getattr(args, "policy_json", None)
160+
if (
161+
args.out_json is None
162+
and summary_json is None
163+
and policy_json is None
164+
and args.out_md is None
165+
and args.out_sarif is None
166+
):
167+
raise ValueError(
168+
"at least one of --out-json, --summary-json, --policy-json, --out-md, or --out-sarif must be provided"
169+
)
144170
if pypi_timeout <= 0:
145171
raise ValueError("--pypi-timeout must be a positive number of seconds.")
146172
if scorecard_timeout <= 0:
@@ -232,6 +258,8 @@ def run_compare(args: argparse.Namespace) -> int:
232258
_write_text(args.out_json, render_report_json(report))
233259
if summary_json is not None:
234260
_write_text(summary_json, render_summary_json(report))
261+
if policy_json is not None:
262+
_write_text(policy_json, render_policy_json(report))
235263
if args.out_md is not None:
236264
_write_text(args.out_md, render_report_markdown(report))
237265
if args.out_sarif is not None:

tools/sbom-diff-and-risk/src/sbom_diff_risk/report_json.py

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,27 @@ def render_summary_json(report: CompareReport) -> str:
5151
return json.dumps(_summary_to_dict(report), indent=2) + "\n"
5252

5353

54+
def render_policy_json(report: CompareReport) -> str:
55+
policy_sections = build_policy_report_sections(report.metadata.policy_evaluation)
56+
payload: dict[str, object] = {
57+
"policy_evaluation": policy_sections["policy_evaluation"],
58+
"blocking_findings": policy_sections["blocking_findings"],
59+
"warning_findings": policy_sections["warning_findings"],
60+
"suppressed_findings": policy_sections["suppressed_findings"],
61+
"rule_catalog": policy_sections["rule_catalog"],
62+
}
63+
64+
policy_summary = _policy_summary_to_dict(report.metadata.policy_evaluation)
65+
if policy_summary is not None:
66+
payload["summary"] = {"policy": policy_summary}
67+
68+
if policy_sections["provenance_policy"] is not None:
69+
payload["provenance_policy"] = policy_sections["provenance_policy"]
70+
payload["provenance_policy_impact"] = policy_sections["provenance_policy_impact"]
71+
72+
return json.dumps(payload, indent=2) + "\n"
73+
74+
5475
def _summary_to_dict(report: CompareReport) -> dict[str, object]:
5576
summary: dict[str, object] = {
5677
"added": report.summary.added,

tools/sbom-diff-and-risk/tests/test_cli_exit_codes.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ def test_cli_compare_help_mentions_policy_flags_and_exit_codes() -> None:
119119
assert result.returncode == 0
120120
assert "--out-sarif" in result.stdout
121121
assert "--summary-json" in result.stdout
122+
assert "--policy-json" in result.stdout
122123
assert "--pyproject-group" in result.stdout
123124
assert "--policy" in result.stdout
124125
assert "--fail-on" in result.stdout
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
from __future__ import annotations
2+
3+
import json
4+
from pathlib import Path
5+
6+
from sbom_diff_risk import cli
7+
8+
9+
def test_cli_policy_json_writes_policy_only_file(tmp_path: Path) -> None:
10+
project_root = Path(__file__).resolve().parents[1]
11+
policy_path = tmp_path / "policy.json"
12+
13+
exit_code = cli.main(
14+
[
15+
"compare",
16+
"--before",
17+
str(project_root / "examples" / "cdx_before.json"),
18+
"--after",
19+
str(project_root / "examples" / "cdx_after.json"),
20+
"--policy",
21+
str(project_root / "examples" / "policy-strict.yml"),
22+
"--policy-json",
23+
str(policy_path),
24+
]
25+
)
26+
27+
payload = json.loads(policy_path.read_text(encoding="utf-8"))
28+
29+
assert exit_code == 1
30+
assert payload["summary"]["policy"] == {
31+
"status": "fail",
32+
"blocking": 3,
33+
"warning": 1,
34+
"suppressed": 0,
35+
}
36+
assert payload["policy_evaluation"]["applied"] is True
37+
assert payload["policy_evaluation"]["exit_code"] == 1
38+
assert len(payload["blocking_findings"]) == 3
39+
assert len(payload["warning_findings"]) == 1
40+
assert payload["blocking_findings"][0]["decision_reason"] == "added_package_count_exceeded_threshold"
41+
assert payload["blocking_findings"][0]["policy_rule"] == "max_added_packages"
42+
assert "components" not in payload
43+
assert "risks" not in payload
44+
assert policy_path.read_text(encoding="utf-8").endswith("\n")
45+
46+
47+
def test_cli_policy_json_matches_full_report_policy_sections(tmp_path: Path) -> None:
48+
project_root = Path(__file__).resolve().parents[1]
49+
report_path = tmp_path / "report.json"
50+
policy_path = tmp_path / "policy.json"
51+
52+
exit_code = cli.main(
53+
[
54+
"compare",
55+
"--before",
56+
str(project_root / "examples" / "cdx_before.json"),
57+
"--after",
58+
str(project_root / "examples" / "cdx_after.json"),
59+
"--policy",
60+
str(project_root / "examples" / "policy-strict.yml"),
61+
"--out-json",
62+
str(report_path),
63+
"--policy-json",
64+
str(policy_path),
65+
]
66+
)
67+
68+
report_payload = json.loads(report_path.read_text(encoding="utf-8"))
69+
policy_payload = json.loads(policy_path.read_text(encoding="utf-8"))
70+
71+
assert exit_code == 1
72+
assert policy_payload == _policy_sidecar_from_full_report(report_payload)
73+
74+
75+
def test_cli_policy_json_without_policy_records_not_applied(tmp_path: Path) -> None:
76+
project_root = Path(__file__).resolve().parents[1]
77+
policy_path = tmp_path / "policy.json"
78+
79+
exit_code = cli.main(
80+
[
81+
"compare",
82+
"--before",
83+
str(project_root / "examples" / "cdx_before.json"),
84+
"--after",
85+
str(project_root / "examples" / "cdx_after.json"),
86+
"--policy-json",
87+
str(policy_path),
88+
]
89+
)
90+
91+
payload = json.loads(policy_path.read_text(encoding="utf-8"))
92+
93+
assert exit_code == 0
94+
assert payload["policy_evaluation"]["applied"] is False
95+
assert payload["policy_evaluation"]["exit_code"] == 0
96+
assert "summary" not in payload
97+
assert payload["blocking_findings"] == []
98+
assert payload["warning_findings"] == []
99+
assert payload["suppressed_findings"] == []
100+
101+
102+
def test_cli_policy_json_omitted_does_not_write_policy_file(tmp_path: Path) -> None:
103+
project_root = Path(__file__).resolve().parents[1]
104+
report_path = tmp_path / "report.json"
105+
policy_path = tmp_path / "policy.json"
106+
107+
exit_code = cli.main(
108+
[
109+
"compare",
110+
"--before",
111+
str(project_root / "examples" / "cdx_before.json"),
112+
"--after",
113+
str(project_root / "examples" / "cdx_after.json"),
114+
"--out-json",
115+
str(report_path),
116+
]
117+
)
118+
119+
assert exit_code == 0
120+
assert report_path.is_file()
121+
assert not policy_path.exists()
122+
123+
124+
def _policy_sidecar_from_full_report(report_payload: dict[str, object]) -> dict[str, object]:
125+
policy_payload = {
126+
"policy_evaluation": report_payload["policy_evaluation"],
127+
"blocking_findings": report_payload["blocking_findings"],
128+
"warning_findings": report_payload["warning_findings"],
129+
"suppressed_findings": report_payload["suppressed_findings"],
130+
"rule_catalog": report_payload["rule_catalog"],
131+
}
132+
133+
summary = report_payload["summary"]
134+
assert isinstance(summary, dict)
135+
if "policy" in summary:
136+
policy_payload["summary"] = {"policy": summary["policy"]}
137+
138+
if "provenance_policy" in report_payload:
139+
policy_payload["provenance_policy"] = report_payload["provenance_policy"]
140+
policy_payload["provenance_policy_impact"] = report_payload["provenance_policy_impact"]
141+
142+
return policy_payload

0 commit comments

Comments
 (0)