Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces an AI-driven GitHub Actions workflow to select and run relevant LISA test cases for PR validation, along with a dedicated runbook, and a small tweak to provisioning smoke-test panic labeling.
Changes:
- Added a new GitHub Actions workflow (
ai-test-selection.yml) that calls a selector script to choose test cases and then runs LISA in Azure. - Added an AI test selection helper script (
.github/scripts/ai_select_cases.py) and a new runbook (lisa/microsoft/runbook/ai_selected.yml) intended for running selected cases. - Updated provisioning smoke-test serial-console panic stage label in one failure path.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
lisa/microsoft/testsuites/core/provisioning.py |
Adjusts the SerialConsole.check_panic() stage string used when SSH port readiness fails. |
lisa/microsoft/runbook/ai_selected.yml |
Adds a runbook intended to run testcases by a variable-provided name pattern. |
.github/workflows/ai-test-selection.yml |
Adds a multi-job workflow to select tests via GitHub Models, run selected LISA tests on Azure, and report results back to the PR. |
.github/scripts/ai_select_cases.py |
Implements test enumeration + GitHub Models call + outputs selected case list and a runbook fragment. |
e01ca69 to
21a140b
Compare
921f980 to
ae79977
Compare
ae79977 to
9728e3b
Compare
9728e3b to
86a4845
Compare
| on: | ||
| pull_request: | ||
| branches: | ||
| - main |
There was a problem hiding this comment.
This workflow runs on pull_request and uses Azure secrets/OIDC. For PRs from forks, secrets are not available (leading to failures), and more importantly, you generally don’t want to run privileged Azure jobs for untrusted contributors. Add a guard (job-level if) to only run the Azure portions when the PR is from the same repository (e.g., github.event.pull_request.head.repo.fork == false) and/or when the author is trusted, while still allowing the “select tests” and “report” steps to run safely.
| - name: Azure Login (OIDC) | ||
| uses: azure/login@v2 | ||
| with: | ||
| client-id: ${{ secrets.AZURE_CLIENT_ID }} | ||
| tenant-id: ${{ secrets.AZURE_TENANT_ID }} | ||
| subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} |
There was a problem hiding this comment.
This workflow runs on pull_request and uses Azure secrets/OIDC. For PRs from forks, secrets are not available (leading to failures), and more importantly, you generally don’t want to run privileged Azure jobs for untrusted contributors. Add a guard (job-level if) to only run the Azure portions when the PR is from the same repository (e.g., github.event.pull_request.head.repo.fork == false) and/or when the author is trusted, while still allowing the “select tests” and “report” steps to run safely.
🤖 AI Test SelectionNo test cases were selected for this PR. |
1cf90e0 to
bacd412
Compare
🤖 AI Test SelectionNo test cases were selected for this PR. |
✅ AI Test Selection — PASSED7 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
🤖 AI Test SelectionNo test cases were selected for this PR. |
✅ AI Test Selection — PASSED8 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
❌ AI Test Selection — FAILED9 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
| generic/091 (lisa_0_0) | ❌ FAILED | 0.000 | - output mismatch (see /home/lisatest/lisa_working/tool/xfstests/xfstests-dev/results//xfs/generic/091.out.bad) |
79bb3ae to
d6ecad5
Compare
🤖 AI Test SelectionNo test cases were selected for this PR. |
✅ AI Test Selection — PASSED10 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED10 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED12 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED10 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
d41775f to
f52b934
Compare
✅ AI Test Selection — PASSED10 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED9 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
f52b934 to
64286a2
Compare
✅ AI Test Selection — PASSED12 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED1 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
185618b to
3958c95
Compare
✅ AI Test Selection — PASSED1 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED1 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
✅ AI Test Selection — PASSED3 test case(s) selected (view run) Marketplace image: suse sles-15-sp6-arm64 gen2 latest
Test case details
|
Description
Related Issue
Type of Change
Checklist
Test Validation
Key Test Cases:
Impacted LISA Features:
Tested Azure Marketplace Images:
Test Results