Skip to content

Add durable pipeline status reporting#963

Draft
anth-volk wants to merge 7 commits into
mainfrom
feat/pipeline-status-reporting
Draft

Add durable pipeline status reporting#963
anth-volk wants to merge 7 commits into
mainfrom
feat/pipeline-status-reporting

Conversation

@anth-volk
Copy link
Copy Markdown
Collaborator

Fixes #962

Summary

  • add a Modal pipeline status callable and protected HTTP endpoint
  • persist durable redacted pipeline error records with canonical stage/substage IDs
  • expose bounded status payloads with run manifests, stage manifests, missing expected manifests, and traceback context
  • add AI-facing pipeline operations guidance for diagnosing deployed failures

Tests

  • make lint
  • uv run --no-sync --with pytest pytest tests/unit/test_pipeline_status.py tests/unit/test_step_manifest.py tests/unit/test_pipeline.py tests/unit/test_pipeline_source_contracts.py
  • PYTHONPATH=. /opt/miniconda3/bin/pytest tests/unit/test_pipeline.py
  • PYTHONPATH=. /opt/miniconda3/bin/pytest tests/integration/test_modal_pipeline_seams.py (skips without Modal credentials)
  • uv run --no-sync --with pyyaml python scripts/run_quality_guards.py
  • uv run --no-sync --with ruff ruff check modal_app/step_manifests/status.py modal_app/step_manifests/errors.py policyengine_us_data/utils/error_redaction.py policyengine_us_data/utils/step_manifest.py modal_app/pipeline.py tests/unit/test_pipeline_status.py tests/unit/test_pipeline.py tests/integration/test_modal_pipeline_seams.py
  • git diff --check

@anth-volk anth-volk force-pushed the feat/pipeline-status-reporting branch from b8fc8b2 to 990bb7a Compare May 12, 2026 18:10
@anth-volk anth-volk force-pushed the feat/pipeline-status-reporting branch from a9f5152 to f06b026 Compare May 12, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add durable pipeline status and error reporting

1 participant