Skip to content

[codex] Add policy decision explanation fields #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stacknil merged 1 commit into from
May 9, 2026
Merged

[codex] Add policy decision explanation fields #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions tools/sbom-diff-and-risk/docs/report-schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,22 @@ JSON reports currently use this top-level structure:

When provenance policy fields are relevant, reports may also include `provenance_policy` and `provenance_policy_impact`. Consumers should treat unrecognized top-level fields as additive report data.

## Policy finding explanation fields

Policy findings in `policy_evaluation.blocking_violations`, `policy_evaluation.warning_violations`, `policy_evaluation.suppressed_violations`, `blocking_findings`, `warning_findings`, `suppressed_findings`, and provenance policy impact sections include stable explainability metadata.

These fields describe why a local policy rule produced a block, warning, or suppression. They are policy-decision metadata only; they are not dependency safety verdicts, CVE results, or proof that a package is safe or unsafe.

| Field | Meaning |
| --- | --- |
| `decision_reason` | Stable reason code for the policy decision, such as `risk_finding_matched_policy_rule`, `added_package_count_exceeded_threshold`, or `scorecard_score_below_threshold`. |
| `policy_rule` | Policy rule id that produced the decision. This mirrors `rule_id` for consumers that group explanation data separately. |
| `severity_source` | Source of the active severity, such as `block_on`, `warn_on`, `default_block`, or `default_warn`; `null` when a policy finding has no active severity. |
| `matched_threshold` | Configured threshold or allowlist value involved in the decision, when applicable. |
| `observed_value` | Observed local value that was compared to the policy rule, when applicable. |

Explanation fields appear only on policy finding objects. Risk findings in `risks` remain the analyzer's local heuristic findings and do not receive policy-decision metadata unless a policy evaluation maps them into policy findings.

## Summary contract

`summary` is the stable, compact entry point for automation that needs counts without walking the full report. The `--summary-json PATH` CLI option writes only this stable `report.json["summary"]` object. The checked-in [../examples/sample-summary.json](../examples/sample-summary.json) artifact is the summary-only output for the default CycloneDX example and matches the `summary` object in [../examples/sample-report.json](../examples/sample-report.json). For CI consumption examples, see [summary-json-ci-cookbook.md](summary-json-ci-cookbook.md).
Expand Down
60 changes: 60 additions & 0 deletions tools/sbom-diff-and-risk/examples/sample-policy-fail-report.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,11 @@
"rule_id": "max_added_packages",
"level": "block",
"message": "Added package count 1 exceeds max_added_packages=0.",
"decision_reason": "added_package_count_exceeded_threshold",
"policy_rule": "max_added_packages",
"severity_source": "block_on",
"matched_threshold": 0,
"observed_value": 1,
"component_key": null,
"component_name": null,
"finding_bucket": null,
Expand All @@ -344,6 +349,11 @@
"rule_id": "stale_package",
"level": "block",
"message": "stale_package was not evaluated because enrichment mode is disabled.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "stale_package",
"severity_source": "block_on",
"matched_threshold": null,
"observed_value": "not_evaluated",
"component_key": "purl:pkg:pypi/requests",
"component_name": "requests",
"finding_bucket": "not_evaluated",
Expand All @@ -353,6 +363,11 @@
"rule_id": "stale_package",
"level": "block",
"message": "stale_package was not evaluated because enrichment mode is disabled.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "stale_package",
"severity_source": "block_on",
"matched_threshold": null,
"observed_value": "not_evaluated",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "not_evaluated",
Expand All @@ -364,6 +379,11 @@
"rule_id": "new_package",
"level": "warn",
"message": "Component was not present in the before input.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "new_package",
"severity_source": "warn_on",
"matched_threshold": null,
"observed_value": "new_package",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "new_package",
Expand All @@ -384,6 +404,11 @@
"rule_id": "max_added_packages",
"level": "block",
"message": "Added package count 1 exceeds max_added_packages=0.",
"decision_reason": "added_package_count_exceeded_threshold",
"policy_rule": "max_added_packages",
"severity_source": "block_on",
"matched_threshold": 0,
"observed_value": 1,
"component_key": null,
"component_name": null,
"finding_bucket": null,
Expand All @@ -393,6 +418,11 @@
"rule_id": "stale_package",
"level": "block",
"message": "stale_package was not evaluated because enrichment mode is disabled.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "stale_package",
"severity_source": "block_on",
"matched_threshold": null,
"observed_value": "not_evaluated",
"component_key": "purl:pkg:pypi/requests",
"component_name": "requests",
"finding_bucket": "not_evaluated",
Expand All @@ -402,6 +432,11 @@
"rule_id": "stale_package",
"level": "block",
"message": "stale_package was not evaluated because enrichment mode is disabled.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "stale_package",
"severity_source": "block_on",
"matched_threshold": null,
"observed_value": "not_evaluated",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "not_evaluated",
Expand All @@ -413,6 +448,11 @@
"rule_id": "new_package",
"level": "warn",
"message": "Component was not present in the before input.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "new_package",
"severity_source": "warn_on",
"matched_threshold": null,
"observed_value": "new_package",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "new_package",
Expand Down Expand Up @@ -596,6 +636,11 @@
"rule_id": "max_added_packages",
"level": "block",
"message": "Added package count 1 exceeds max_added_packages=0.",
"decision_reason": "added_package_count_exceeded_threshold",
"policy_rule": "max_added_packages",
"severity_source": "block_on",
"matched_threshold": 0,
"observed_value": 1,
"component_key": null,
"component_name": null,
"finding_bucket": null,
Expand All @@ -605,6 +650,11 @@
"rule_id": "stale_package",
"level": "block",
"message": "stale_package was not evaluated because enrichment mode is disabled.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "stale_package",
"severity_source": "block_on",
"matched_threshold": null,
"observed_value": "not_evaluated",
"component_key": "purl:pkg:pypi/requests",
"component_name": "requests",
"finding_bucket": "not_evaluated",
Expand All @@ -614,6 +664,11 @@
"rule_id": "stale_package",
"level": "block",
"message": "stale_package was not evaluated because enrichment mode is disabled.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "stale_package",
"severity_source": "block_on",
"matched_threshold": null,
"observed_value": "not_evaluated",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "not_evaluated",
Expand All @@ -625,6 +680,11 @@
"rule_id": "new_package",
"level": "warn",
"message": "Component was not present in the before input.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "new_package",
"severity_source": "warn_on",
"matched_threshold": null,
"observed_value": "new_package",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "new_package",
Expand Down
15 changes: 15 additions & 0 deletions tools/sbom-diff-and-risk/examples/sample-policy-warn-report.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,11 @@
"rule_id": "new_package",
"level": "warn",
"message": "Component was not present in the before input.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "new_package",
"severity_source": "warn_on",
"matched_threshold": null,
"observed_value": "new_package",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "new_package",
Expand All @@ -348,6 +353,11 @@
"rule_id": "new_package",
"level": "warn",
"message": "Component was not present in the before input.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "new_package",
"severity_source": "warn_on",
"matched_threshold": null,
"observed_value": "new_package",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "new_package",
Expand Down Expand Up @@ -523,6 +533,11 @@
"rule_id": "new_package",
"level": "warn",
"message": "Component was not present in the before input.",
"decision_reason": "risk_finding_matched_policy_rule",
"policy_rule": "new_package",
"severity_source": "warn_on",
"matched_threshold": null,
"observed_value": "new_package",
"component_key": "purl:pkg:pypi/urllib3",
"component_name": "urllib3",
"finding_bucket": "new_package",
Expand Down
Loading