CI: auto-trigger AITER prebuilt upload when 3rdparty/aiter updates on dev

[VeeraRajasekhar]

Description

This adds CI to build and upload AITER prebuilts to AMD Artifactory when the AITER submodule pointer or contents under 3rdparty/aiter change on dev, so prebuilts stay aligned with the submodule without relying only on manual runs.

Fixes # (issue)

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

Please list the changes introduced in this PR:

• Add .github/workflows/aiter-prebuilt-upload.yml: push to dev with paths: 3rdparty/aiter.

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[Micky774]

What happens if e.g. an aiter prebuilt cache entry was already created from a branch before it gets merged into dev, thus on merge this action would not be needed -- does it still run and overwrite the entry? Does it cancel gracefully?

[ipanfilo]

Pushing to dev is too late. Binaries are expected to be cached when PR is created, otherwise the PR CI will have to rebuild them

[VeeraRajasekhar]

I initially started with an idea to provide a PR comment trigger which does this. Do you think it is better? In this case I might need to provide a way to force-push if the user needs to trigger multiple times while working on the PR.

[ipanfilo]

I wonder if it can be driven by CI labels + filter by specific path modification. I.e. on the first CI run after aiter commit update, it first builds and uploads AITER and then goes further with CI.

[Micky774]

On that note, can we have an upload as a side effect during the build-and-test workflow? That would provide a relatively simple way to implement this.

Specifically, we can do this more-or-less as-is by using the following filtering for a prebuilt cache upload:

  on:                                                                                                                                       
    pull_request: 
      paths:
        - '3rdparty/aiter'
        - '3rdpart/aiter/***'

Not sure which one exactly is needed since aiter is a submodule but one should work. Still, I'd be more interested in conditionally checking whether the AITER submodule was built from source, and then uploading if it was in the build-and-test flow.

[VeeraRajasekhar]

What happens if e.g. an aiter prebuilt cache entry was already created from a branch before it gets merged into dev, thus on merge this action would not be needed -- does it still run and overwrite the entry? Does it cancel gracefully?

It will cancel It won't force push or anything.

[VeeraRajasekhar]

rocm-ci-dispatch.yml

• Job aiter_upload_trigger: paths-filter on 3rdparty/aiter vs the PR base → sets trigger_aiter_upload.
• dispatch calls rocm-ci.yml with test_level + trigger_aiter_upload (same entry path as existing PR CI).

rocm-ci.yml

• select_docker_image: resolves the Docker image once from ci/ci_config.json (shared script: select_te_docker_image_ci_config.sh).
• upload_aiter_prebuilt: runs only when workflow_call / workflow_dispatch and inputs.trigger_aiter_upload — i.e. PR CI (or manual), not on push to dev/release. It uses aiter-prebuilt-upload.yml and passes the resolved image.
• build_and_test: GPU matrix; waits for image resolution and for upload to succeed or be skipped (skipped on post-merge push, where we assume prebuilts were already published from the PR).

aiter-prebuilt-upload.yml

• Reusable workflow: workflow_call takes required docker_image from rocm-ci; workflow_dispatch unchanged for manual runs.

[ipanfilo]

Big rocm-ci refactor is in progress #528, let's postpone this PR till refactoring one is merged

[ipanfilo]

Put the PR on hold not to interfere with other WIP

[ipanfilo]

workflow_dispatch is not PR triggered action. Also no need to specify default in description

[VeeraRajasekhar]

rebasing with the latest changes to rocm-ci file.

[Micky774]

Rather than adding a label to skip the pre-built upload, can we instead have it be opt-in so that we don't burn extra resources on intermediate changes and iteration?

[VeeraRajasekhar]

Rather than adding a label to skip the pre-built upload, can we instead have it be opt-in so that we don't burn extra resources on intermediate changes and iteration?

I will be removing this “skip prebuilt upload” label. The original idea was to handle cases where Artifactory was unreachable from the runner, but that’s no longer the main control: we now have preflight checks that test whether the AITER prebuilt Artifactory is reachable from the current runner. If it isn’t, we skip the upload and still run CI. So we don’t need a separate skip label for that anymore.

As for keeping an Aiter-Upload label, doing this would force an awkward ordering (“only add ci-level-* after upload succeeds”) and extra process overhead. It would also waste effort when people set test labels first and we build but never publish.

Current behavior
Labels stay as they are. Dispatch can wait until the AITER upload step finishes (success or skipped after preflight). Preflight determines reachability: if Artifactory isn’t reachable, we skip upload and continue with tests instead of failing or burning upload work unnecessarily.