feat(e2e): add E2E workflow and CPU smoke test#300
feat(e2e): add E2E workflow and CPU smoke test#300runpod-Henrik wants to merge 3 commits intomainfrom
Conversation
Adds a GitHub Actions E2E workflow and a CPU smoke test that runs on every release to verify the full deployment pipeline end-to-end. ## Workflow (.github/workflows/e2e.yml) - Three jobs: Unit+Integration (with coverage), E2E, Summary - Triggered manually via workflow_dispatch (push/PR triggers commented out, ready to enable after merge) - E2E job uses RUNPOD_API_KEY secret; runs pytest e2e/ serially with no per-test timeout - Summary job writes a test results + coverage table to the run's Summary tab ## E2E tests (e2e/) - conftest.py: credential handling (env var or ~/.runpod/config.toml), hard-fail in CI if no key, shared endpoint_id_from_state() and sweep_endpoints() helpers - test_cpu_smoke.py (E2E-002): deploy a minimal CPU worker, invoke it, assert output, undeploy; always sweeps all endpoints on the dedicated e2e account to prevent quota leaks from partial failures Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
runpod-Henrik
left a comment
runpod-Henrik
left a comment
There was a problem hiding this comment.
Review
.github/workflows/e2e.yml
No issues. Trigger, permissions, job structure, zero-test guard, and summary output are all correct. --override-ini="addopts=" correctly suppresses the --cov-fail-under=65 that would otherwise kill the E2E job. The inputs.tests filter expression resolves safely to no flag when empty.
e2e/conftest.py
sweep_endpoints calls client._execute_graphql (private method) — intentional and acceptable for test code, but may surface in linter output.
try: import tomllib / except ImportError: import tomli — project requires Python ≥3.11, so tomllib is always in stdlib and the tomli fallback is dead code. Not harmful, just slightly misleading.
_REAL_API_KEY captured at module import time, before any monkeypatching — correct design, and the comment explains why.
e2e/test_cpu_smoke.py
Fixed in latest commit (3498b29): subprocess.run(..., timeout=60) in the finally block was not catching subprocess.TimeoutExpired, which would bubble out of finally and suppress the sweep_endpoints call. Now wrapped in try/except subprocess.TimeoutExpired.
WORKER_NAME is a module-level f-string evaluated at import time — not a risk given the workflow runs one test at a time with -p no:xdist, but worth knowing.
Overall
Clean and mergeable. No functional issues.
1 participant
Summary
.github/workflows/e2e.ymlwith three jobs: Unit+Integration (with coverage), E2E, and a Summary that writes results to the run's Summary tabe2e/directory with a CPU smoke test that deploys a minimal worker, invokes it, and verifies output — runs every release to catch regressions in the full deploy pipelineworkflow_dispatchonly for now; push/PR triggers are commented in and ready to enable after this mergesTest plan
uv.lockregenerated againstorigin/mainworkflow_dispatchonmainafter merge to confirm smoke test runs clean on the release branchNotes
e2e/is separate fromtests/soci.ymlnever collects these tests (it would deploy real infrastructure sinceRUNPOD_API_KEYis already in the environment)sweep_endpoints()in conftest nukes all endpoints on the dedicated e2e account after each test — prevents quota leaks from partial failures