Skip to content

Add SARIF reporting support#477

Open
seonghobae wants to merge 2 commits into
usestrix:mainfrom
Seongho-Bae:feature/sarif-support
Open

Add SARIF reporting support#477
seonghobae wants to merge 2 commits into
usestrix:mainfrom
Seongho-Bae:feature/sarif-support

Conversation

@seonghobae
Copy link
Copy Markdown

Summary

  • Add SARIF 2.1.0 report generation for Strix vulnerability findings with GitHub code scanning-compatible locations and rules.
  • Add --sarif / --sarif-output CLI flags and write SARIF before non-interactive exit code 2 so CI uploads can run with if: always().
  • Document a GitHub Actions upload flow and cover unsafe/locationless finding handling with regression tests.

Validation

  • uv run pytest -q → 118 passed, 4 warnings
  • uv run mypy strix/telemetry/sarif.py strix/interface/main.py → success
  • uv run pyright strix/telemetry/sarif.py → 0 errors
  • uv run ruff check strix/telemetry/sarif.py strix/interface/main.py tests/telemetry/test_sarif.py tests/interface/test_sarif_cli.py → all checks passed

Notes

  • Locationless findings and dropped unsafe locations are summarized in SARIF run properties instead of creating invalid GitHub alerts.
  • Unsafe paths, URI-scheme paths, absolute paths, parent traversal, and invalid line ranges are rejected.

greptile-apps[bot]

This comment was marked as off-topic.

@seanturner-zh
Copy link
Copy Markdown
Contributor

Hi @Seongho-Bae — heads up on parallel development.

We independently shipped a SARIF emitter at the same path (strix/telemetry/sarif.py) on a personal fork this week (ours went up 2026-05-07; yours 2026-05-05). Same file, same intent, different specifics. No ego on our side about which lands — yours is cleaner in several places where ours wasn't:

  • explicit --sarif / --sarif-output CLI flags
  • write-before-non-interactive-exit-2 so if: always() upload works in GitHub Actions
  • _sarif_uri path-safety rejection (URI-scheme, absolute, .., invalid line ranges)
  • locationlessFindings summary in run.properties rather than invalid alerts
  • mypy / pyright / ruff clean

I built a combined branch that takes your two commits as the base and layers our additions on top: seanturner83/strix:feat/sarif-combined. Preserves your authorship on the foundation. All your tests pass unchanged except one assertion I had to update for a properties.strix.* namespacing change (see below). Ours adds:

  • CWE normalisation. Strix findings ship CWE as CWE-306, cwe:306, or bare 306 depending on the agent's output variant. Without normalisation the same weakness across runs dedups into separate rules. One helper (_normalise_cwe) collapses them to canonical CWE-NNN form.
  • Rule-level GitHub code-scanning properties. rule.defaultConfiguration.level, rule.properties.security-severity (CVSS→0.0-10.0 string that drives alert ranking), rule.properties.tags (security + CWE-NNN + CVE-…), rule.helpUricwe.mitre.org. Without these, alerts show at GitHub's default severity with no tag filter.
  • PoC namespacing. Moving exploit payloads under result.properties.strix.poc rather than flat on result.properties, so generic SARIF UI consumers don't surface exploit text to triage audiences by default. result.properties.security-severity stays top-level because that's where GitHub reads it. (This is the one breaking change to your test suite — the commit updates test_build_sarif_maps_code_location_and_metadata to match, rationale in the commit body.)
  • Backwards-compat aliases (build_sarif_document, write_sarif(run_dir, reports)) so internal callers can use either entry point.
  • Tracer-sidecar hook in strix/telemetry/tracer.py that writes findings.sarif automatically on every run, orthogonal to your --sarif CLI flag. Failure to emit SARIF never blocks the primary CSV+MD path. Useful for consumers that want SARIF as a default, without needing a flag on every invocation.
  • Three real SARIF example files from SAST / DAST / crystal-box runs against OWASP Juice Shop, committed under docs/sarif-examples/ so reviewers can eyeball actual output + upload to GitHub code-scanning to see the alert rendering.

State on the combined branch: 17 SARIF tests pass (10 from your PR unchanged + 7 new), 35 tracer-adjacent tests pass, ruff clean, mypy clean, 0 new warnings.

Three ways I can help:

  1. I open a PR from seanturner83:feat/sarif-combined → your branch (Seongho-Bae:feature/sarif-support), stacking our additions on your commits. You'd merge that into your branch; Add SARIF reporting support #477 then carries the combined result for maintainers to review. Keeps everything in one thread and preserves your authorship on the foundation.
  2. I paste the diffs here as a comment and you pull in whichever bits you want into Add SARIF reporting support #477 directly. Authorship on the additions stays with me via the commit chain on feat/sarif-combined, but the final merged PR is yours.
  3. I open a separate PR to usestrix:main that explicitly acknowledges yours, credits the foundation, and notes it should be merged after Add SARIF reporting support #477. Less collaborative but independent of your availability.

My preference is (1), falling back to (2) or (3) based on what works for you. No urgency, but if I don't hear back in about a week I'll default to (3) so the work isn't blocked indefinitely — happy to revise on your request after that.

Combined branch for your review: https://github.com/seanturner83/strix/tree/feat/sarif-combined

@seonghobae
Copy link
Copy Markdown
Author

Hi @Seongho-Bae — heads up on parallel development.

We independently shipped a SARIF emitter at the same path (strix/telemetry/sarif.py) on a personal fork this week (ours went up 2026-05-07; yours 2026-05-05). Same file, same intent, different specifics. No ego on our side about which lands — yours is cleaner in several places where ours wasn't:

  • explicit --sarif / --sarif-output CLI flags
  • write-before-non-interactive-exit-2 so if: always() upload works in GitHub Actions
  • _sarif_uri path-safety rejection (URI-scheme, absolute, .., invalid line ranges)
  • locationlessFindings summary in run.properties rather than invalid alerts
  • mypy / pyright / ruff clean

I built a combined branch that takes your two commits as the base and layers our additions on top: seanturner83/strix:feat/sarif-combined. Preserves your authorship on the foundation. All your tests pass unchanged except one assertion I had to update for a properties.strix.* namespacing change (see below). Ours adds:

  • CWE normalisation. Strix findings ship CWE as CWE-306, cwe:306, or bare 306 depending on the agent's output variant. Without normalisation the same weakness across runs dedups into separate rules. One helper (_normalise_cwe) collapses them to canonical CWE-NNN form.
  • Rule-level GitHub code-scanning properties. rule.defaultConfiguration.level, rule.properties.security-severity (CVSS→0.0-10.0 string that drives alert ranking), rule.properties.tags (security + CWE-NNN + CVE-…), rule.helpUricwe.mitre.org. Without these, alerts show at GitHub's default severity with no tag filter.
  • PoC namespacing. Moving exploit payloads under result.properties.strix.poc rather than flat on result.properties, so generic SARIF UI consumers don't surface exploit text to triage audiences by default. result.properties.security-severity stays top-level because that's where GitHub reads it. (This is the one breaking change to your test suite — the commit updates test_build_sarif_maps_code_location_and_metadata to match, rationale in the commit body.)
  • Backwards-compat aliases (build_sarif_document, write_sarif(run_dir, reports)) so internal callers can use either entry point.
  • Tracer-sidecar hook in strix/telemetry/tracer.py that writes findings.sarif automatically on every run, orthogonal to your --sarif CLI flag. Failure to emit SARIF never blocks the primary CSV+MD path. Useful for consumers that want SARIF as a default, without needing a flag on every invocation.
  • Three real SARIF example files from SAST / DAST / crystal-box runs against OWASP Juice Shop, committed under docs/sarif-examples/ so reviewers can eyeball actual output + upload to GitHub code-scanning to see the alert rendering.

State on the combined branch: 17 SARIF tests pass (10 from your PR unchanged + 7 new), 35 tracer-adjacent tests pass, ruff clean, mypy clean, 0 new warnings.

Three ways I can help:

  1. I open a PR from seanturner83:feat/sarif-combined → your branch (Seongho-Bae:feature/sarif-support), stacking our additions on your commits. You'd merge that into your branch; Add SARIF reporting support #477 then carries the combined result for maintainers to review. Keeps everything in one thread and preserves your authorship on the foundation.
  2. I paste the diffs here as a comment and you pull in whichever bits you want into Add SARIF reporting support #477 directly. Authorship on the additions stays with me via the commit chain on feat/sarif-combined, but the final merged PR is yours.
  3. I open a separate PR to usestrix:main that explicitly acknowledges yours, credits the foundation, and notes it should be merged after Add SARIF reporting support #477. Less collaborative but independent of your availability.

My preference is (1), falling back to (2) or (3) based on what works for you. No urgency, but if I don't hear back in about a week I'll default to (3) so the work isn't blocked indefinitely — happy to revise on your request after that.

Combined branch for your review: https://github.com/seanturner83/strix/tree/feat/sarif-combined

Hi, @seanturner-zh !
Please make PR(s) on your preference, I'll happy to support you. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants