Skip to content

Commit 7237692

Browse files
saptarshimandal1saptarshimandal1
authored
Merge pull request #135 from NHSDigital/feature/APM-6390
Added new SBOM config for generation all 3 reports
2 parents 0e263b7 + f842980 commit 7237692

File tree

10 files changed

+2120
-1640
lines changed

10 files changed

+2120
-1640
lines changed

‎.github/README.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# SBOM & Vulnerability Scanning Automation
2+
3+
This repository uses GitHub Actions to automatically generate a Software Bill of Materials (SBOM), scan for vulnerabilities, and produce package inventory reports.
4+
5+
All reports are named with the repository name for easy identification.
6+
7+
## Features
8+
9+
SBOM Generation: Uses Syft to generate an SPDX JSON SBOM.
10+
SBOM Merging: Merges SBOMs for multiple tools if needed.
11+
SBOM to CSV: Converts SBOM JSON to a CSV report.
12+
Vulnerability Scanning: Uses Grype to scan the SBOM for vulnerabilities and outputs a CSV report.
13+
Package Inventory: Extracts a simple package list (name, type, version) as a CSV.
14+
Artifacts: All reports are uploaded as workflow artifacts with the repository name in the filename.
15+
16+
## Workflow Overview
17+
18+
The main workflow is defined in .github/workflows/sbom.yml
19+
20+
## Scripts
21+
22+
scripts/create-sbom.sh
23+
Generates an SBOM for the repo and for specified tools, merging them as needed.
24+
scripts/update-sbom.py
25+
Merges additional SBOMs into the main SBOM.
26+
.github/scripts/sbom_json_to_csv.py
27+
Converts the SBOM JSON to a detailed CSV report.
28+
.github/scripts/grype_json_to_csv.py
29+
Converts Grype’s vulnerability scan JSON output to a CSV report.
30+
Output columns: REPO, NAME, INSTALLED, FIXED-IN, TYPE, VULNERABILITY, SEVERITY
31+
.github/scripts/sbom_packages_to_csv.py
32+
Extracts a simple package inventory from the SBOM.
33+
Output columns: name, type, version
34+
35+
## Example Reports
36+
37+
Vulnerability Report
38+
grype-report-[RepoName].csv
39+
REPO,NAME,INSTALLED,FIXED-IN,TYPE,VULNERABILITY,SEVERITY
40+
my-repo,Flask,2.1.2,,library,CVE-2022-12345,High
41+
...
42+
43+
Package Inventory
44+
sbom-packages-[RepoName].csv
45+
name,type,version
46+
Flask,library,2.1.2
47+
Jinja2,library,3.1.2
48+
...
49+
50+
## Usage
51+
52+
Push to main branch or run the workflow manually.
53+
Download artifacts from the workflow run summary.
54+
55+
## Customization
56+
57+
Add more tools to scripts/create-sbom.sh as needed.
58+
Modify scripts to adjust report formats or add more metadata.

‎.github/scripts/grype_json_to_csv.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import json
2+
import csv
3+
import sys
4+
5+
input_file = sys.argv[1] if len(sys.argv) > 1 else "grype-report.json"
6+
output_file = sys.argv[2] if len(sys.argv) > 2 else "grype-report.csv"
7+
8+
with open(input_file, "r", encoding="utf-8") as f:
9+
data = json.load(f)
10+
11+
columns = ["NAME", "INSTALLED", "FIXED-IN", "TYPE", "VULNERABILITY", "SEVERITY"]
12+
13+
with open(output_file, "w", newline="", encoding="utf-8") as csvfile:
14+
writer = csv.DictWriter(csvfile, fieldnames=columns)
15+
writer.writeheader()
16+
for match in data.get("matches", []):
17+
pkg = match.get("artifact", {})
18+
vuln = match.get("vulnerability", {})
19+
row = {
20+
"NAME": pkg.get("name", ""),
21+
"INSTALLED": pkg.get("version", ""),
22+
"FIXED-IN": vuln.get("fix", {}).get("versions", [""])[0] if vuln.get("fix", {}).get("versions") else "",
23+
"TYPE": pkg.get("type", ""),
24+
"VULNERABILITY": vuln.get("id", ""),
25+
"SEVERITY": vuln.get("severity", ""),
26+
}
27+
writer.writerow(row)
28+
print(f"CSV export complete: {output_file}")

‎.github/scripts/sbom_json_to_csv.py‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
import json
2+
import csv
3+
import sys
4+
# from pathlib import Path
5+
from tabulate import tabulate
6+
7+
input_file = sys.argv[1] if len(sys.argv) > 1 else "sbom.json"
8+
output_file = sys.argv[2] if len(sys.argv) > 2 else "sbom.csv"
9+
10+
with open(input_file, "r", encoding="utf-8") as f:
11+
sbom = json.load(f)
12+
13+
packages = sbom.get("packages", [])
14+
15+
columns = [
16+
"name",
17+
"versionInfo",
18+
"type",
19+
"supplier",
20+
"downloadLocation",
21+
"licenseConcluded",
22+
"licenseDeclared",
23+
"externalRefs"
24+
]
25+
26+
27+
def get_type(pkg):
28+
spdxid = pkg.get("SPDXID", "")
29+
if "-" in spdxid:
30+
parts = spdxid.split("-")
31+
if len(parts) > 2:
32+
return parts[2]
33+
refs = pkg.get("externalRefs", [])
34+
for ref in refs:
35+
if ref.get("referenceType") == "purl":
36+
return ref.get("referenceLocator", "").split("/")[0]
37+
return ""
38+
39+
40+
def get_external_refs(pkg):
41+
refs = pkg.get("externalRefs", [])
42+
return ";".join([ref.get("referenceLocator", "") for ref in refs])
43+
44+
45+
with open(output_file, "w", newline="", encoding="utf-8") as csvfile:
46+
writer = csv.DictWriter(csvfile, fieldnames=columns)
47+
writer.writeheader()
48+
for pkg in packages:
49+
row = {
50+
"name": pkg.get("name", ""),
51+
"versionInfo": pkg.get("versionInfo", ""),
52+
"type": get_type(pkg),
53+
"supplier": pkg.get("supplier", ""),
54+
"downloadLocation": pkg.get("downloadLocation", ""),
55+
"licenseConcluded": pkg.get("licenseConcluded", ""),
56+
"licenseDeclared": pkg.get("licenseDeclared", ""),
57+
"externalRefs": get_external_refs(pkg)
58+
}
59+
writer.writerow(row)
60+
61+
print(f"CSV export complete: {output_file}")
62+
63+
64+
with open("sbom_table.txt", "w", encoding="utf-8") as f:
65+
table = []
66+
for pkg in packages:
67+
row = [
68+
pkg.get("name", ""),
69+
pkg.get("versionInfo", ""),
70+
get_type(pkg),
71+
pkg.get("supplier", ""),
72+
pkg.get("downloadLocation", ""),
73+
pkg.get("licenseConcluded", ""),
74+
pkg.get("licenseDeclared", ""),
75+
get_external_refs(pkg)
76+
]
77+
table.append(row)
78+
f.write(tabulate(table, columns, tablefmt="grid"))

‎.github/scripts/sbom_packages_to_csv.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import json
2+
import csv
3+
import sys
4+
import os
5+
6+
input_file = sys.argv[1] if len(sys.argv) > 1 else "sbom.json"
7+
repo_name = sys.argv[2] if len(sys.argv) > 2 else os.getenv("GITHUB_REPOSITORY", "unknown-repo").split("/")[-1]
8+
output_file = f"sbom-packages-{repo_name}.csv"
9+
10+
with open(input_file, "r", encoding="utf-8") as f:
11+
sbom = json.load(f)
12+
13+
packages = sbom.get("packages", [])
14+
15+
columns = ["name", "type", "version"]
16+
17+
with open(output_file, "w", newline="", encoding="utf-8") as csvfile:
18+
writer = csv.DictWriter(csvfile, fieldnames=columns)
19+
writer.writeheader()
20+
for pkg in packages:
21+
row = {
22+
"name": pkg.get("name", ""),
23+
"type": pkg.get("type", ""),
24+
"version": pkg.get("versionInfo", "")
25+
}
26+
writer.writerow(row)
27+
28+
print(f"Package list CSV generated: {output_file}")

‎.github/workflows/sbom.yml‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
name: SBOM Vulnerability Scanning
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
environment:
7+
description: "Run SBOM check"
8+
required: true
9+
type: choice
10+
options:
11+
- yes
12+
- no
13+
14+
env:
15+
SYFT_VERSION: "1.27.1"
16+
TF_VERSION: "1.12.2"
17+
18+
jobs:
19+
deploy:
20+
name: Software Bill of Materials
21+
runs-on: ubuntu-latest
22+
permissions:
23+
actions: read
24+
contents: write
25+
steps:
26+
- name: Checkout
27+
uses: actions/checkout@v5
28+
29+
- name: Setup Python 3.13
30+
uses: actions/setup-python@v5
31+
with:
32+
python-version: "3.13"
33+
34+
- name: Setup Terraform
35+
uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd
36+
37+
- uses: terraform-linters/setup-tflint@ae78205cfffec9e8d93fd2b3115c7e9d3166d4b6
38+
name: Setup TFLint
39+
40+
- name: Set architecture variable
41+
id: os-arch
42+
run: |
43+
case "${{ runner.arch }}" in
44+
X64) ARCH="amd64" ;;
45+
ARM64) ARCH="arm64" ;;
46+
esac
47+
echo "arch=${ARCH}" >> $GITHUB_OUTPUT
48+
49+
- name: Download and setup Syft
50+
run: |
51+
DOWNLOAD_URL="https://github.com/anchore/syft/releases/download/v${{ env.SYFT_VERSION }}/syft_${{ env.SYFT_VERSION }}_linux_${{ steps.os-arch.outputs.arch }}.tar.gz"
52+
echo "Downloading: ${DOWNLOAD_URL}"
53+
54+
curl -L -o syft.tar.gz "${DOWNLOAD_URL}"
55+
tar -xzf syft.tar.gz
56+
chmod +x syft
57+
58+
# Add to PATH for subsequent steps
59+
echo "$(pwd)" >> $GITHUB_PATH
60+
61+
- name: Create SBOM
62+
run: bash scripts/create-sbom.sh terraform python tflint
63+
64+
- name: Convert SBOM JSON to CSV
65+
run: |
66+
pip install --upgrade pip
67+
pip install tabulate
68+
REPO_NAME=$(basename $GITHUB_REPOSITORY)
69+
python .github/scripts/sbom_json_to_csv.py sbom.json SBOM_${REPO_NAME}.csv
70+
71+
- name: Upload SBOM CSV as artifact
72+
uses: actions/upload-artifact@v4
73+
with:
74+
name: sbom-csv
75+
path: SBOM_${{ github.event.repository.name }}.csv
76+
77+
- name: Install Grype
78+
run: |
79+
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
80+
81+
- name: Scan SBOM for Vulnerabilities (JSON)
82+
run: |
83+
grype sbom:sbom.json -o json > grype-report.json
84+
85+
86+
87+
- name: Convert Grype JSON to CSV
88+
run: |
89+
pip install --upgrade pip
90+
REPO_NAME=$(basename $GITHUB_REPOSITORY)
91+
python .github/scripts/grype_json_to_csv.py grype-report.json grype-report-${REPO_NAME}.csv
92+
93+
94+
- name: Upload Vulnerability Report
95+
uses: actions/upload-artifact@v4
96+
with:
97+
name: grype-report
98+
path: grype-report-${{ github.event.repository.name }}.csv
99+
100+
- name: Generate Package Inventory CSV
101+
run: |
102+
pip install --upgrade pip
103+
REPO_NAME=$(basename $GITHUB_REPOSITORY)
104+
python .github/scripts/sbom_packages_to_csv.py sbom.json $REPO_NAME
105+
106+
- name: Upload Package Inventory CSV
107+
uses: actions/upload-artifact@v4
108+
with:
109+
name: sbom-packages
110+
path: sbom-packages-${{ github.event.repository.name }}.csv

‎azure/azure-build-pipeline.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,28 @@ trigger:
77
tags:
88
include:
99
- v*
10+
trigger:
11+
- main
12+
13+
pool:
14+
vmImage: 'ubuntu-latest'
15+
16+
steps:
17+
- task: UsePythonVersion@0
18+
inputs:
19+
versionSpec: '3.x'
20+
addToPath: true
21+
22+
- script: |
23+
pip install poetry
24+
poetry install
25+
poetry add --dev flake8
26+
displayName: 'Install Poetry and flake8'
27+
28+
- script: |
29+
find . -name '*.py' -not -path '**/.venv/*' -not -path '**/.direnv/*' | xargs poetry run flake8
30+
displayName: 'Run flake8 linting'
31+
1032

1133
pr:
1234
branches:

0 commit comments

Comments
 (0)