Skip to content

ci: Test geos integration of geosPythonPackages for every new PR #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alexbenedicto wants to merge 11 commits into from
Closed

ci: Test geos integration of geosPythonPackages for every new PR #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b5143fa
Add geos integration test
alexbenedicto Aug 1, 2025
ca53a7f
Add documentation for geos integration testing
alexbenedicto Aug 1, 2025
eca4fe0
Add permissions and avoid silent fail of curl command
alexbenedicto Aug 6, 2025
f316ee8
Move silent fail catch and hardcode GEOS-DEV branch to test against
alexbenedicto Aug 6, 2025
eff6f71
Make check_geos_integration_label test fail when no label
alexbenedicto Aug 6, 2025
a2f66e8
Add more error handling in curl commands
alexbenedicto Aug 6, 2025
ef4304a
Try different triggering approach
alexbenedicto Aug 12, 2025
40ece5e
Back to previous triggering, needs merge in GEOS develop to be tested…
alexbenedicto Aug 14, 2025
18ccf4f
Try more specific branch naming
alexbenedicto Aug 21, 2025
5a13339
Improve verbosity of trigger GEOS integration test workflow, help imp…
alexbenedicto Sep 19, 2025
5570f4e
Add detailed HTTP error reporting
alexbenedicto Sep 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
325 changes: 323 additions & 2 deletions .github/workflows/python-package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ jobs:
- geos-trame
- geos-utils
- geos-xml-tools
- geos-xml-viewer
- hdf5-wrapper
- pygeos-tools
include:
Expand Down Expand Up @@ -98,4 +97,326 @@ jobs:
run:
# python -m pytest ./${{ matrix.package-name }} --doctest-modules --junitxml=junit/test-results.xml --cov-report=xml --cov-report=html |
# wrap pytest to avoid error when no tests in the package
sh -c 'python -m pytest ./${{ matrix.package-name }}; ret=$?; [ $ret = 5 ] && exit 0 || exit $ret'
sh -c 'python -m pytest ./${{ matrix.package-name }}; ret=$?; [ $ret = 5 ] && exit 0 || exit $ret'

# Step 1: Validate that all standard CI tests pass BEFORE checking GEOS integration
validate_standard_ci:
runs-on: ubuntu-latest
needs: [semantic_pull_request, build]
steps:
- name: Check standard CI results
run: |
echo "Checking standard CI results..."
echo "Semantic PR check: ${{ needs.semantic_pull_request.result }}"
echo "Build and test: ${{ needs.build.result }}"

# All standard tests must pass before proceeding
if [[ "${{ needs.semantic_pull_request.result }}" != "success" ]]; then
echo "❌ Semantic PR check failed - fix PR title before proceeding"
exit 1
fi

if [[ "${{ needs.build.result }}" != "success" ]]; then
echo "❌ Build and test failed - fix code issues before proceeding"
exit 1
fi

echo "✅ All standard CI tests passed! Ready for GEOS integration testing."

# Step 2: Only after standard tests pass, check for GEOS integration label
check_geos_integration_label:
runs-on: ubuntu-latest
needs: [validate_standard_ci]
permissions:
pull-requests: read
outputs:
should_test_geos: ${{ steps.check_label.outputs.should_test_geos }}
has_label: ${{ steps.check_label.outputs.has_label }}
steps:
- name: Check for GEOS integration test label
id: check_label
run: |
set -e # Exit immediately if a command exits with a non-zero status.
echo "Standard CI tests have passed. Now checking for GEOS integration requirements..."

# Check if the PR has the 'test-geos-integration' label
# -fsS gives Fails on Error (f), Silent on Success (s), Shows Error Message (S)
pr_json=$(curl -fsS -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.number }})

LABELS=$(echo ${pr_json} | jq -r '[.labels[].name] | join(",")')
echo "PR labels: ${LABELS}"

if [[ "${LABELS}" == *"test-geos-integration"* ]]; then
echo "has_label=true" >> $GITHUB_OUTPUT
echo "should_test_geos=true" >> $GITHUB_OUTPUT
echo "✅ Found 'test-geos-integration' label - will proceed with GEOS integration testing"
else
echo "has_label=false" >> $GITHUB_OUTPUT
echo "should_test_geos=false" >> $GITHUB_OUTPUT
echo "❌ Missing 'test-geos-integration' label"
echo ""
echo "REQUIRED: This PR must have the 'test-geos-integration' label to be merged"
echo "This ensures that changes are tested against GEOS before merging"
echo ""
echo "To add the label:"
echo "1. Go to your PR page"
echo "2. Click on 'Labels' in the right sidebar"
echo "3. Add the 'test-geos-integration' label"
exit 1
fi

# Step 3: Only trigger GEOS integration if label exists AND standard tests passed
trigger_geos_integration_test:
runs-on: ubuntu-latest
needs: [check_geos_integration_label]
if: needs.check_geos_integration_label.outputs.should_test_geos == 'true'
permissions:
pull-requests: read
outputs:
workflow_run_id: ${{ steps.trigger_workflow.outputs.workflow_run_id }}
steps:
- name: Get PR information
id: pr_info
run: |
echo "Triggering GEOS integration test..."
pr_json=$(curl -fsSL -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
"https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.number }}")

LABELS=$(echo "${pr_json}" | jq -r '[.labels[].name] | join(",")')
PR_HEAD_REPO=$(echo ${pr_json} | jq -r '.head.repo.clone_url')
PR_HEAD_BRANCH=$(echo ${pr_json} | jq -r '.head.ref')
PR_HEAD_SHA=$(echo ${pr_json} | jq -r '.head.sha')

echo "pr_repo=${PR_HEAD_REPO}" >> $GITHUB_OUTPUT
echo "pr_branch=${PR_HEAD_BRANCH}" >> $GITHUB_OUTPUT
echo "pr_sha=${PR_HEAD_SHA}" >> $GITHUB_OUTPUT

echo "PR Repository: ${PR_HEAD_REPO}"
echo "PR Branch: ${PR_HEAD_BRANCH}"
echo "PR SHA: ${PR_HEAD_SHA}"

- name: Trigger GEOS integration test workflow
id: trigger_workflow
run: |
set -e
echo "All standard tests passed ✅"
echo "GEOS integration label found ✅"
echo "Now triggering GEOS integration test workflow..."

# Determine which GEOS branch contains the workflow file
# Priority: 1) ci/benedicto/testGeosPythonPackagesIntegration (current dev branch)
# 2) develop (main branch)
# 3) main (fallback)
GEOS_BRANCH="ci/benedicto/testGeosPythonPackagesIntegration"

# Check if workflow exists on the target branch
echo "Checking if workflow exists on branch: ${GEOS_BRANCH}"
workflow_check=$(curl -fsSL \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
https://api.github.com/repos/GEOS-DEV/GEOS/contents/.github/workflows/test_geospythonpackages_integration.yml?ref=${GEOS_BRANCH} 2>/dev/null || echo "not_found")

if [[ "$workflow_check" == "not_found" ]]; then
echo "Workflow not found on ${GEOS_BRANCH}, trying develop branch..."
GEOS_BRANCH="develop"
workflow_check=$(curl -fsSL \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
https://api.github.com/repos/GEOS-DEV/GEOS/contents/.github/workflows/test_geospythonpackages_integration.yml?ref=${GEOS_BRANCH} 2>/dev/null || echo "not_found")

if [[ "$workflow_check" == "not_found" ]]; then
echo "❌ ERROR: Workflow file not found on develop branch either!"
echo "This could indicate a permissions issue with GEOS_INTEGRATION_TOKEN"
echo "Please verify the token has proper repository access"
exit 1
fi
fi

echo "✅ Found workflow on branch: ${GEOS_BRANCH}"

# Test the dispatch endpoint first
echo "Testing workflow dispatch endpoint access..."
dispatch_test=$(curl -fsSL \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
"https://api.github.com/repos/GEOS-DEV/GEOS/actions/workflows/test_geospythonpackages_integration.yml" 2>&1 || echo "endpoint_error")

if [[ "$dispatch_test" == "endpoint_error" ]]; then
echo "❌ ERROR: Cannot access workflow dispatch endpoint"
echo "This suggests the GEOS_INTEGRATION_TOKEN lacks 'actions:write' permission"
exit 1
fi

echo "✅ Workflow dispatch endpoint accessible"

# Trigger the workflow_dispatch event in the GEOS repository
# Note: ref should point to a branch that contains the workflow file
# The python package branch info is passed as inputs
echo "Triggering workflow dispatch on GEOS repository..."
echo "Target branch: ${GEOS_BRANCH}"
echo "Python package repo: ${{ steps.pr_info.outputs.pr_repo }}"
echo "Python package branch: ${{ steps.pr_info.outputs.pr_branch }}"

# Make the request with detailed error reporting
echo "Making workflow dispatch request..."
http_code=$(curl -w "%{http_code}" -o /tmp/dispatch_response.txt \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/GEOS-DEV/GEOS/actions/workflows/test_geospythonpackages_integration.yml/dispatches \
-d "{\"ref\": \"${GEOS_BRANCH}\", \"inputs\": {\"python_package_repo\": \"${{ steps.pr_info.outputs.pr_repo }}\", \"python_package_branch\": \"${{ steps.pr_info.outputs.pr_branch }}\", \"python_package_pr\": \"${{ github.event.number }}\", \"requested_by\": \"${{ github.actor }}\"}}")

echo "HTTP response code: $http_code"
echo "Response body:"
cat /tmp/dispatch_response.txt

# Check the HTTP response code
if [[ "$http_code" -eq 204 ]]; then
echo "✅ GEOS integration test workflow triggered successfully on branch: ${GEOS_BRANCH}"
elif [[ "$http_code" -eq 404 ]]; then
echo "❌ ERROR: 404 Not Found"
echo "Possible causes:"
echo "1. Workflow file does not exist on branch ${GEOS_BRANCH}"
echo "2. GEOS_INTEGRATION_TOKEN lacks access to GEOS-DEV/GEOS repository"
echo "3. Token is invalid or expired"
exit 1
elif [[ "$http_code" -eq 403 ]]; then
echo "❌ ERROR: 403 Forbidden"
echo "GEOS_INTEGRATION_TOKEN lacks 'actions:write' permission"
echo "The token needs both 'repo' and 'actions:write' scopes"
exit 1
elif [[ "$http_code" -eq 422 ]]; then
echo "❌ ERROR: 422 Unprocessable Entity"
echo "Invalid input data or workflow configuration issue"
echo "Check that all required inputs are provided correctly"
exit 1
else
echo "❌ ERROR: Unexpected HTTP response code: $http_code"
echo "Response body:"
cat /tmp/dispatch_response.txt
exit 1
fi

# Wait a moment for the workflow to start, then find the run ID
sleep 10

# Get the latest workflow runs to find our triggered run
runs_response=$(curl -fsS -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
https://api.github.com/repos/GEOS-DEV/GEOS/actions/workflows/test_geospythonpackages_integration.yml/runs?per_page=5)

# Find the most recent run (this assumes it's our triggered run)
workflow_run_id=$(echo "$runs_response" | jq -r '.workflow_runs[0].id')
echo "workflow_run_id=${workflow_run_id}" >> $GITHUB_OUTPUT
echo "Triggered workflow run ID: ${workflow_run_id}"

# Step 4: Wait for GEOS integration results (only if triggered)
wait_for_geos_integration_result:
runs-on: ubuntu-latest
needs: [trigger_geos_integration_test]
if: needs.trigger_geos_integration_test.outputs.workflow_run_id != ''
outputs:
geos_test_result: ${{ steps.wait_for_result.outputs.geos_test_result }}
steps:
- name: Wait for GEOS integration test to complete
id: wait_for_result
run: |
WORKFLOW_RUN_ID="${{ needs.trigger_geos_integration_test.outputs.workflow_run_id }}"
echo "Waiting for GEOS integration test to complete (Run ID: ${WORKFLOW_RUN_ID})..."
echo "This may take 15-30 minutes..."

# Wait for the workflow to complete (with timeout)
timeout=1800 # 30 minutes
elapsed=0
interval=60

while [ $elapsed -lt $timeout ]; do
run_status=$(curl -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
https://api.github.com/repos/GEOS-DEV/GEOS/actions/runs/${WORKFLOW_RUN_ID} \
| jq -r '.status')

conclusion=$(curl -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GEOS_INTEGRATION_TOKEN }}" \
https://api.github.com/repos/GEOS-DEV/GEOS/actions/runs/${WORKFLOW_RUN_ID} \
| jq -r '.conclusion')

echo "Workflow status: ${run_status}, conclusion: ${conclusion} (${elapsed}s elapsed)"

if [[ "${run_status}" == "completed" ]]; then
echo "Workflow completed with conclusion: ${conclusion}"

if [[ "${conclusion}" == "success" ]]; then
echo "✅ GEOS integration test PASSED"
echo "geos_test_result=success" >> $GITHUB_OUTPUT
exit 0
else
echo "❌ GEOS integration test FAILED"
echo "geos_test_result=failure" >> $GITHUB_OUTPUT
exit 1
fi
fi

sleep $interval
elapsed=$((elapsed + interval))
done

echo "❌ TIMEOUT: GEOS integration test did not complete within ${timeout} seconds"
echo "geos_test_result=timeout" >> $GITHUB_OUTPUT
exit 1

# Step 5: Final validation - requires ALL tests to pass
final_validation:
runs-on: ubuntu-latest
needs: [check_geos_integration_label, wait_for_geos_integration_result]
if: always()
steps:
- name: Final merge validation
run: |
echo "=== FINAL MERGE VALIDATION ==="
echo ""
echo "Standard CI Tests: ✅ PASSED (already validated)"
echo "GEOS Integration Label: ${{ needs.check_geos_integration_label.outputs.has_label == 'true' && '✅ PRESENT' || '❌ MISSING' }}"
echo "GEOS Integration Tests: ${{ needs.wait_for_geos_integration_result.outputs.geos_test_result == 'success' && '✅ PASSED' || needs.wait_for_geos_integration_result.result == 'skipped' && '⏭️ SKIPPED (no label)' || '❌ FAILED' }}"
echo ""

# Check label requirement
if [[ "${{ needs.check_geos_integration_label.outputs.has_label }}" != "true" ]]; then
echo "❌ INVALID: Missing 'test-geos-integration' label"
echo ""
echo "This PR cannot be merged without the 'test-geos-integration' label"
echo "Please add the label and wait for GEOS integration tests to pass"
exit 1
fi

# Check GEOS test results (only if they were supposed to run)
if [[ "${{ needs.wait_for_geos_integration_result.result }}" == "failure" || "${{ needs.wait_for_geos_integration_result.outputs.geos_test_result }}" == "failure" ]]; then
echo "❌ INVALID: GEOS integration tests failed"
echo ""
echo "The changes in this PR break GEOS functionality"
echo "Please check the GEOS workflow logs and fix the issues"
echo "GEOS workflow: https://github.com/GEOS-DEV/GEOS/actions/runs/${{ needs.trigger_geos_integration_test.outputs.workflow_run_id }}"
exit 1
fi

if [[ "${{ needs.wait_for_geos_integration_result.outputs.geos_test_result }}" == "timeout" ]]; then
echo "❌ INVALID: GEOS integration tests timed out"
echo ""
echo "Please check the GEOS workflow manually and re-run if needed"
echo "GEOS workflow: https://github.com/GEOS-DEV/GEOS/actions/runs/${{ needs.trigger_geos_integration_test.outputs.workflow_run_id }}"
exit 1
fi

# All validations passed
echo "✅ VALID: All requirements met"
echo ""
echo "This PR is ready for review and merge:"
echo " ✅ Standard CI tests passed"
echo " ✅ 'test-geos-integration' label present"
echo " ✅ GEOS integration tests passed"
echo ""
echo "🎉 Ready for merge!"
Loading
Loading