Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 21 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ current flagship tool. It compares SBOMs and dependency manifests, produces
JSON, Markdown, and SARIF review artifacts, supports local policy checks, and
can optionally record PyPI provenance and OpenSSF Scorecard evidence.

For a fast reviewer overview, start with the [`sbom-diff-and-risk` reviewer
brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md).
For the clearest reviewer route, start with the
[`sbom-diff-and-risk` reviewer path](tools/sbom-diff-and-risk/docs/reviewer-path.md);
it separates orientation, artifact inspection, local reproduction, and release
evidence.

## Additional Project

Expand Down Expand Up @@ -50,6 +52,7 @@ evidence.
Useful entry points:

- [`sbom-diff-and-risk` README](tools/sbom-diff-and-risk/README.md)
- [Reviewer path](tools/sbom-diff-and-risk/docs/reviewer-path.md)
- [Reviewer brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md)
- [Reviewer evidence pack](tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md)
- [v0.9.0 release notes][release-notes-v090]
Expand Down Expand Up @@ -101,15 +104,22 @@ publishing is intentionally deferred.
attestations, and PyPI Trusted Publishing provenance as interchangeable
evidence.

## Reviewer Quick Path

1. Read the [`sbom-diff-and-risk` reviewer brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md).
2. Skim the [`sbom-diff-and-risk` README](tools/sbom-diff-and-risk/README.md)
for CLI scope and examples.
3. Check the [v0.9.0 release notes][release-notes-v090].
4. Use the [verification guide](tools/sbom-diff-and-risk/docs/verification.md)
to choose the right provenance check.
5. Inspect the [examples](tools/sbom-diff-and-risk/examples/) for sample reports and policy files.
## Reviewer Quick Path

For `sbom-diff-and-risk`, use the
[reviewer path](tools/sbom-diff-and-risk/docs/reviewer-path.md) and first choose
the review question:

1. 30 seconds:
read the [reviewer brief](tools/sbom-diff-and-risk/docs/reviewer-brief.md).
2. 5 minutes:
inspect [sample JSON, summary, policy, Markdown, and SARIF artifacts](tools/sbom-diff-and-risk/examples/).
3. 15 minutes:
run the deterministic example check in
[example artifact regeneration](tools/sbom-diff-and-risk/docs/example-artifact-regeneration.md).
4. Release evidence:
use the [verification guide](tools/sbom-diff-and-risk/docs/verification.md)
and [release provenance docs](tools/sbom-diff-and-risk/docs/release-provenance.md).

## Status

Expand Down
57 changes: 57 additions & 0 deletions docs/reviewer-brief.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Reviewer brief

## Problem

Scientific and security-oriented review work often needs small deterministic tools, not vague platforms. Reviewers need evidence that inputs, outputs, and claims stay inspectable.

## What it does

`scientific-computing-toolkit` is a portfolio repository for scientific-computing infrastructure and supply-chain review work.

The current flagship project is `tools/sbom-diff-and-risk`, a local CLI for comparing two SBOMs or dependency manifests and producing deterministic JSON, Markdown, SARIF, and policy sidecar artifacts.

## Quick run

From the repository root:

```bash
cd tools/sbom-diff-and-risk
python -m pip install -e ".[dev]"
sbom-diff-risk compare --before examples/cdx_before.json --after examples/cdx_after.json --format auto --out-json outputs/report.json --summary-json outputs/summary.json --out-md outputs/report.md
```

## Sample output

The flagship tool can emit:

- `report.json`
- `summary.json`
- `policy.json`
- `report.md`
- `report.sarif`

The checked-in examples and docs cover deterministic local output, optional policy decisions, and opt-in provenance or Scorecard evidence when explicit enrichment flags are enabled.

## What this proves

- deterministic supply-chain review tooling
- reviewer-oriented artifact design instead of black-box scoring
- careful separation between local default behavior and opt-in enrichment
- the ability to package a broader repo around one clear flagship tool

## Safety / boundaries

- local-file analysis is the default
- no hidden network enrichment
- not a vulnerability scanner or package reputation oracle
- production PyPI publishing remains intentionally deferred

## Limitations

- the root repo is currently flagship-led rather than evenly balanced across multiple finished tools
- heuristic risk buckets do not resolve CVEs or exploitability
- provenance and Scorecard evidence are advisory, not proof that a dependency is safe

## Next milestone

Keep strengthening the flagship reviewer route while adding the next finished tool or mini-lab at the same documentation and evidence standard.
11 changes: 8 additions & 3 deletions tools/sbom-diff-and-risk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,16 @@ dry-run story, and keeps production PyPI publishing intentionally deferred.

It uses conservative heuristics for change intelligence. By default it does not resolve CVEs, does not act as a reputation oracle, and does not perform hidden network enrichment.

## Start Here

## Start Here

For the clearest reviewer route, start with
[docs/reviewer-path.md](docs/reviewer-path.md). It gives a 30-second,
5-minute, and 15-minute path through the project, plus separate release
evidence and deep-review routes.

This project has two different provenance stories:

For a concise reviewer-facing overview, start with
For a concise reviewer-facing overview after choosing the review route, use
[docs/reviewer-brief.md](docs/reviewer-brief.md). For reproducible review
evidence and verification commands, use
[docs/reviewer-evidence-pack.md](docs/reviewer-evidence-pack.md). For
Expand Down
26 changes: 15 additions & 11 deletions tools/sbom-diff-and-risk/docs/reviewer-brief.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# Reviewer brief

## Summary
## Summary

`sbom-diff-and-risk` is a local CLI for comparing two SBOMs or dependency manifests and producing deterministic review artifacts: JSON, Markdown, and SARIF. It is built for conservative supply-chain review, not for vulnerability scanning or package reputation scoring.

Current released version: `v0.9.0`.

For the shortest ordered review route, use
[reviewer-path.md](reviewer-path.md).

## Why this project matters

Expand Down Expand Up @@ -37,15 +40,16 @@ Dependency review often needs evidence that is stable enough for code review, CI
| Did Trusted Publishing get exercised safely? | `docs/pypi-trusted-publishing-readiness.md` documents the completed TestPyPI dry-run. |
| Is production PyPI enabled? | `docs/pypi-production-publishing-decision.md` documents that production PyPI is intentionally deferred. |

## Quick verification path

1. Read this brief for the 30-second project shape.
2. Read [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for reproducible commands and evidence paths.
3. Read `README.md` for CLI scope, supported inputs, and examples.
4. Read `docs/verification.md` to choose the right verification path.
5. Use `docs/self-provenance.md` when verifying workflow-built wheel or source distribution artifacts.
6. Use `docs/release-provenance.md` when verifying GitHub Release assets.
7. Use `docs/pypi-production-publishing-decision.md` before making any production PyPI publishing decision.
## Quick verification path

1. Use [reviewer-path.md](reviewer-path.md) as the ordered route.
2. Read this brief for the 30-second project shape.
3. Read [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for reproducible commands and evidence paths.
4. Read `README.md` for CLI scope, supported inputs, and examples.
5. Read `docs/verification.md` to choose the right verification path.
6. Use `docs/self-provenance.md` when verifying workflow-built wheel or source distribution artifacts.
7. Use `docs/release-provenance.md` when verifying GitHub Release assets.
8. Use `docs/pypi-production-publishing-decision.md` before making any production PyPI publishing decision.

## What this project intentionally does not claim

Expand Down
5 changes: 4 additions & 1 deletion tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Reviewer evidence pack

This page is a reproducible evidence checklist for reviewing `sbom-diff-and-risk`. It focuses on what can be verified from the repository, examples, GitHub release assets, and TestPyPI dry-run documentation. It does not introduce new CLI behavior.
This page is a reproducible evidence checklist for reviewing `sbom-diff-and-risk`. It focuses on what can be verified from the repository, examples, GitHub release assets, and TestPyPI dry-run documentation. It does not introduce new CLI behavior.

For the shortest ordered route through these materials, start with
[reviewer-path.md](reviewer-path.md).

## Project Identity

Expand Down
156 changes: 156 additions & 0 deletions tools/sbom-diff-and-risk/docs/reviewer-path.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Reviewer path

Use this page as the ordered review route for `sbom-diff-and-risk`. It is
designed to make one thing clear at every step: what evidence you are checking,
where to find it, and what it does not prove.

## First choose the review question

| Review question | Start here | Good stopping point |
| --- | --- | --- |
| What is this tool? | [30-second orientation](#30-second-orientation) | You can state the tool's scope and non-claims. |
| What artifacts does it produce? | [5-minute artifact review](#5-minute-artifact-review) | You can point to JSON, summary, policy, Markdown, and SARIF examples. |
| Can the examples be reproduced locally? | [15-minute reproduction check](#15-minute-reproduction-check) | `regenerate-example-artifacts.py --check` passes without enrichment. |
| Can the released tool artifacts be verified? | [Release evidence](#release-evidence) | You can choose the correct GitHub release, checksum, or attestation path. |
| Is this enough for a full review? | [Deep review](#deep-review) | You have followed the reproducible checklist in the evidence pack. |

## 30-second orientation

Read:

- [reviewer-brief.md](reviewer-brief.md)
- the first screen of the [tool README](../README.md)

Confirm these claims only:

- local, deterministic SBOM/dependency diff CLI
- supported inputs: CycloneDX JSON, SPDX JSON, `requirements.txt`, and
conservative `pyproject.toml`
- output artifacts: JSON, Markdown, SARIF, `summary.json`, and `policy.json`
- local policy checks with reviewer-facing decision explanation fields
- optional PyPI provenance and OpenSSF Scorecard enrichment only when
explicitly enabled
- production PyPI publishing remains intentionally deferred

Stop here if you only need the project shape for a reviewer, resume, or PR
summary.

Do not infer:

- CVE scanning
- dependency safety verdicts
- package reputation scoring
- hidden network enrichment
- production PyPI availability

## 5-minute artifact review

Inspect the checked-in examples in this order:

| Step | Artifact | What it proves |
| --- | --- | --- |
| 1 | [sample-report.json](../examples/sample-report.json) | Full machine-readable diff, risk, policy, and metadata shape. |
| 2 | [sample-summary.json](../examples/sample-summary.json) | Compact CI-facing `summary` contract. |
| 3 | [sample-policy.json](../examples/sample-policy.json) | Policy-only sidecar for CI consumers. |
| 4 | [sample-report.md](../examples/sample-report.md) | Human-readable reviewer report. |
| 5 | [sample-sarif.sarif](../examples/sample-sarif.sarif) | Conservative code-scanning output for selected high-signal findings. |
| 6 | [github-actions-policy-consumer.yml](../examples/github-actions-policy-consumer.yml) | Copyable consumer path for capturing policy JSON in GitHub Actions. |

Then read:

- [report-schema.md](report-schema.md)
- [policy-decision-explainability.md](policy-decision-explainability.md)
- [github-code-scanning.md](github-code-scanning.md)

Look for these reviewer anchors:

- `summary` is the compact machine-readable entry point
- `summary.policy` appears only when policy evaluation runs
- `summary.enrichment` appears only when enrichment evidence exists
- policy findings explain `decision_reason`, `policy_rule`,
`matched_threshold`, and `observed_value`
- SARIF is intentionally narrow and does not mirror every report finding

Stop here if you need to understand the review outputs without running code.

## 15-minute reproduction check

From `tools/sbom-diff-and-risk`, run the deterministic example checks:

```powershell
python -m pip install -e .[dev]
python scripts/regenerate-example-artifacts.py --check
python scripts/regenerate-example-artifacts.py --check --only requirements
```

Expected result:

- the full checked-in no-network example set is up to date
- the focused requirements example check passes
- JSON, Markdown, summary, policy sidecar, and SARIF examples match the
committed artifacts
- no PyPI, Scorecard, CVE, or advisory network lookup is performed

For the exact regeneration scope, read
[example-artifact-regeneration.md](example-artifact-regeneration.md).

Stop here if you need reproducible local evidence that the examples still match
the code.

## Release evidence

Use this section only when the review question is about the released
`sbom-diff-and-risk` tool artifacts. It is not the path for judging third-party
dependency safety.

| Evidence surface | Use when | Read |
| --- | --- | --- |
| Verification decision guide | You need to choose the right release verification path. | [verification.md](verification.md) |
| GitHub Release assets and checksums | You downloaded wheel or source distribution files from a release. | [release-provenance.md](release-provenance.md) |
| Workflow artifact attestations | You are verifying workflow-built wheel or source distribution artifacts. | [self-provenance.md](self-provenance.md) |
| TestPyPI Trusted Publishing dry-run | You are checking whether the dry-run publisher path worked. | [pypi-trusted-publishing-readiness.md](pypi-trusted-publishing-readiness.md) |
| Production PyPI decision gate | You are deciding whether production PyPI should be enabled later. | [pypi-production-publishing-decision.md](pypi-production-publishing-decision.md) |

Keep the evidence surfaces separate:

- GitHub workflow artifact attestation verifies workflow-built artifacts.
- GitHub Release asset checksums verify downloaded release bytes against the
release checksum manifest.
- GitHub immutable-release verification applies only when the release is
immutable and GitHub has generated release attestations.
- TestPyPI Trusted Publishing proves the dry-run publisher path worked.
- Production PyPI Trusted Publishing is intentionally deferred and does not
exist for this project yet.

Stop here if your review question is release provenance rather than dependency
analysis behavior.

## Deep review

Use [reviewer-evidence-pack.md](reviewer-evidence-pack.md) for the full
reproducible checklist, including:

- local demo commands
- release asset inspection
- checksum verification
- artifact attestation verification
- TestPyPI evidence boundaries
- SARIF/code-scanning boundaries
- non-claims

Use these supporting docs for focused review questions:

- [dependency-risk-heuristics.md](dependency-risk-heuristics.md) for risk bucket
semantics
- [parser-boundaries.md](parser-boundaries.md) for deterministic parser limits
- [policy-schema.md](policy-schema.md) for policy file shape
- [policy-decision-ci-cookbook.md](policy-decision-ci-cookbook.md) for CI policy
consumption
- [summary-json-ci-cookbook.md](summary-json-ci-cookbook.md) for summary-only CI
consumption

## Reviewer bottom line

`sbom-diff-and-risk` is review infrastructure. It makes dependency changes,
policy decisions, and selected supply-chain trust signals easier to inspect.
It does not decide whether a dependency is safe.