OSAC-853: add AAP presubmit e2e-vmaas job by omer-vishlitzky · Pull Request #79365 · openshift/release

omer-vishlitzky · 2026-05-16T12:08:50Z

Summary

Add e2e-vmaas presubmit test for osac-aap PRs, matching the pattern used by fulfillment-service and osac-operator
Build the execution environment (EE) image from the PR source in CI
Override AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's playbooks and EE instead of the pinned versions

Why AAP is different

Unlike other OSAC components (single container image), AAP has three independently-versioned artifacts that must all come from the PR:

Artifact	How it's overridden
EE image (runtime + collections)	`kustomize edit set image` (existing mechanism) + `AAP_EE_IMAGE` sed
Playbooks (git-synced by AAP)	`AAP_PROJECT_GIT_BRANCH` sed to PR head SHA
Config manifests (K8s resources)	`COPY config/` into installer (existing mechanism)

The boot script detects a .aap-source-sha marker file in the installer image. When present, it overrides the git ref and EE image. When absent (all other repos), it's a no-op.

Jira: https://redhat.atlassian.net/browse/OSAC-853

Test plan

Rehearsal jobs pass on this PR
Open a test PR on osac-aap and verify the e2e-vmaas job triggers and the AAP overrides are applied

This PR adds a presubmit e2e-vmaas job for osac-aap in OpenShift CI that builds and runs tests using the PR’s artifacts (execution environment image, playbooks git ref, and installer config) instead of pinned releases.

What changed (practical impact)

Repos/config affected: ci-operator config for osac-project/osac-aap and the osac-project cluster-tool boot step (step-registry). This changes how osac-aap PRs are built and how the cluster-tool boots and refreshes AAP for e2e-vmaas runs.
Effect: CI now builds PR-specific EE and installer images, injects a source-SHA marker into the installer image, and the cluster-tool boot/refresh applies overrides so the cluster uses the PR’s playbooks, EE image, and config manifests during the test run.

Key changes

ci-operator config (ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml)
- Adds base_images (dev-scripts, origin-cli, osac-installer, osac-test-infra) and binary_build_commands that write a source-SHA (/tmp/source-sha) using git rev-parse HEAD^2 with a HEAD fallback.
- Adds two PR-specific image builds:
  - osac-aap-pr: builds the PR execution environment (CentOS Stream 9 base, Python 3.12, runtime/dev deps, pip requirements, Ansible collections/vendor, and copies oc/kubectl from origin-cli).
  - osac-installer-with-pr: builds an installer image from osac-installer, copies config/ into /installer/base/osac-aap/config/, and copies the generated source-sha into /installer/.aap-source-sha.
- Excludes the PR-specific images from promotion to releases.latest and adds a releases.latest.candidate block for OCP nightly 4.20.
- Adds an e2e-vmaas presubmit wired to osac-aap-pr, osac-installer-with-pr, and osac-test-infra, setting COMPONENT_IMAGE_NAME and using the osac-project-cluster-tool-vmaas workflow.
Cluster-tool boot script (ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh)
- Detects the .aap-source-sha marker in the installer image (copied to /installer/.aap-source-sha).
- Builds kustomize/sed overrides to set AAP_PROJECT_GIT_BRANCH to the detected SHA so AAP will git-sync playbooks from the PR head.
- When a component image override is provided, also sets AAP_EE_IMAGE to the PR-built EE image.
- Applies these overrides during the refresh phase so the running cluster uses the PR’s playbooks, EE, and installer-provided config manifests.

Testing / behavior

Rehearsal jobs pass on this PR.
CI will build the PR-specific EE and installer images and supply them to the e2e-vmaas job.
The installer image’s .aap-source-sha marker triggers the boot script to apply the git-ref and EE-image overrides at refresh time, enabling tests to run against the PR’s playbooks, EE, and config.

Notes

Pattern follows existing OSAC components (fulfillment-service, osac-operator) to coordinate three independently-versioned artifacts: EE image, playbooks (git ref), and config manifests.
PR-specific images are intentionally excluded from promotion into releases.

openshift-ci-robot · 2026-05-16T12:08:54Z

@omer-vishlitzky: This pull request references OSAC-853 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Add e2e-vmaas presubmit test for osac-aap PRs, matching the pattern used by fulfillment-service and osac-operator

Build the execution environment (EE) image from the PR source in CI

Override AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's playbooks and EE instead of the pinned versions

Why AAP is different

Unlike other OSAC components (single container image), AAP has three independently-versioned artifacts that must all come from the PR:

Artifact How it's overridden

EE image (runtime + collections) kustomize edit set image (existing mechanism) + AAP_EE_IMAGE sed

Playbooks (git-synced by AAP) AAP_PROJECT_GIT_BRANCH sed to PR head SHA

Config manifests (K8s resources) COPY config/ into installer (existing mechanism)

The boot script detects a .aap-source-sha marker file in the installer image. When present, it overrides the git ref and EE image. When absent (all other repos), it's a no-op.

Jira: https://redhat.atlassian.net/browse/OSAC-853

Test plan

Rehearsal jobs pass on this PR

Open a test PR on osac-aap and verify the e2e-vmaas job triggers and the AAP overrides are applied

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

coderabbitai · 2026-05-16T12:09:15Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 5ee9187e-8944-4038-b4b8-3d313fd8f351

📥 Commits

Reviewing files that changed from the base of the PR and between ed84f7c and 9a04463.

⛔ Files ignored due to path filters (2)

ci-operator/jobs/osac-project/osac-aap/osac-project-osac-aap-main-postsubmits.yaml is excluded by !ci-operator/jobs/**
ci-operator/jobs/osac-project/osac-aap/osac-project-osac-aap-main-presubmits.yaml is excluded by !ci-operator/jobs/**

📒 Files selected for processing (2)

ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml
ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh

🚧 Files skipped from review as they are similar to previous changes (2)

ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh
ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml

Walkthrough

This PR captures a build source SHA and embeds it into two PR images, updates promotion and release candidate settings, adds an e2e-vmaas test that consumes those images, and updates the cluster boot script to detect the embedded SHA and apply it as AAP configuration overrides during refresh.

Changes

AAP Source SHA Tracking and Override Mechanism

Layer / File(s)	Summary
Base images and source SHA generation `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	Base image references for dev-scripts, origin-cli, osac-installer, and osac-test-infra are defined; `binary_build_commands` generates `/tmp/source-sha` from git revision with fallback.
PR image builds with source SHA embedding `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	`osac-aap-pr` dockerfile builds CentOS Stream 9 with Python 3.12, build/runtime dependencies, ansible collections, and oc/kubectl; `osac-installer-with-pr` copies installer config and the generated source-sha file into the installer image.
Promotion rules and release candidate configuration `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	Latest namespace promotion is updated to exclude the new PR images; release candidate targeting OCP nightly stream version 4.20 is added.
E2E test wiring `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	`e2e-vmaas` test entry declares dependencies on `osac-aap-pr`, `osac-installer-with-pr`, and `osac-test-infra`, specifies the workflow `osac-project-cluster-tool-vmaas`, and defines `COMPONENT_IMAGE_NAME` environment variable.
Boot script source SHA detection and AAP overrides `ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh`	Boot script reads `.aap-source-sha` from the installer image and constructs `sed` commands to override `AAP_PROJECT_GIT_BRANCH` to the detected SHA; if a component image override is also specified, additionally overrides `AAP_EE_IMAGE`.
Refresh command with combined overrides `ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh`	Cluster refresh invocation is updated to apply both `COMPONENT_OVERRIDE_CMD` and the newly added `AAP_OVERRIDE_CMD` before executing the refresh script.

Sequence Diagram(s)

sequenceDiagram
  participant CI as CI Builder
  participant Installer as installer image (osac-installer-with-pr)
  participant Boot as cluster boot script
  participant Refresh as scripts/refresh-after-snapshot.sh
  CI->>Installer: build image and embed .aap-source-sha
  Boot->>Installer: read .aap-source-sha from image
  Boot->>Boot: construct AAP_OVERRIDE_CMD (set AAP_PROJECT_GIT_BRANCH[, AAP_EE_IMAGE])
  Boot->>Refresh: run with COMPONENT_OVERRIDE_CMD + AAP_OVERRIDE_CMD
  Refresh->>Installer: apply sed overrides to kustomization.yaml

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

openshift/release#79304: Related modifications to the OSAC cluster-tool vmaas boot path and applied overrides during e2e-vmaas provisioning.

Suggested labels

rehearsals-ack

Suggested reviewers

danmanor
jhernand

🚥 Pre-merge checks | ✅ 12

✅ Passed checks (12 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically identifies the main change: adding an e2e-vmaas presubmit job for osac-aap, which is the primary objective of the PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	No Ginkgo test files present. PR modifies CI config (YAML) and boot script (shell). Tests use pytest, not Ginkgo. Check not applicable.
Test Structure And Quality	✅ Passed	The PR contains no Ginkgo test files. Changes are limited to YAML CI configuration and a bash boot script. The custom check for Ginkgo test quality is not applicable to this PR.
Microshift Test Compatibility	✅ Passed	No new Ginkgo e2e tests added. PR modifies CI/CD configs and shell scripts. The e2e-vmaas test is Python pytest-based, not Go Ginkgo. Check not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PR does not add Ginkgo e2e tests. Modifies only CI YAML config, shell boot scripts, and Go utility files. Actual tests are Python pytest in external container. SNO check not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR modifies CI infrastructure config for image building and test scripts. No deployment manifests, operator code, or scheduling constraints are introduced.
Ote Binary Stdout Contract	✅ Passed	This PR modifies YAML CI config and shell scripts only. The OTE Binary Stdout Contract check applies to Go test code only. No Go test or OTE binary files are modified.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	This PR does not add Ginkgo e2e tests. It adds bash scripts and YAML CI configuration that run pytest tests (not Ginkgo). The custom check only applies to Ginkgo tests and is not applicable here.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-ci · 2026-05-16T12:09:24Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: omer-vishlitzky
Once this PR has been reviewed and has the lgtm label, please assign jhernand for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

ci-operator/config/osac-project/osac-aap/OWNERS
ci-operator/jobs/osac-project/osac-aap/OWNERS
~~ci-operator/step-registry/osac-project/cluster-tool/boot/OWNERS~~ [omer-vishlitzky]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

omer-vishlitzky · 2026-05-16T17:19:38Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-16T17:19:40Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Add e2e-vmaas presubmit test for osac-aap PRs. Unlike other OSAC components, AAP has three independently-versioned artifacts: the execution environment image, playbooks synced from git, and config manifests. The standard single-image swap doesn't cover playbook sync. Build the EE image from the PR source, copy config/ into the installer, and embed the PR head SHA as a marker file. The boot script detects the marker and overrides AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's code for all three artifacts.

openshift-merge-bot · 2026-05-17T04:05:35Z

[REHEARSALNOTIFIER]
@omer-vishlitzky: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-osac-project-osac-operator-main-e2e-vmaas	osac-project/osac-operator	presubmit	Registry content changed
pull-ci-osac-project-osac-installer-main-e2e-vmaas	osac-project/osac-installer	presubmit	Registry content changed
pull-ci-osac-project-osac-test-infra-main-e2e-vmaas	osac-project/osac-test-infra	presubmit	Registry content changed
pull-ci-osac-project-osac-aap-main-e2e-vmaas	osac-project/osac-aap	presubmit	Presubmit changed
pull-ci-osac-project-osac-aap-main-images	osac-project/osac-aap	presubmit	Ci-operator config changed
pull-ci-osac-project-fulfillment-service-main-e2e-vmaas	osac-project/fulfillment-service	presubmit	Registry content changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

omer-vishlitzky · 2026-05-17T10:14:49Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T10:14:51Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T16:53:34Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T16:53:36Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T17:25:00Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T17:25:02Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T23:48:22Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T23:48:25Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-18T10:09:53Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-18T10:09:57Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci · 2026-05-18T10:32:49Z

@omer-vishlitzky: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/osac-project/osac-aap/main/e2e-vmaas	`9a04463`	link	unknown	`/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 16, 2026

openshift-ci Bot requested review from adriengentil and danmanor May 16, 2026 12:09

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch 2 times, most recently from 57679bb to 0557df8 Compare May 16, 2026 17:15

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch from 0557df8 to ed84f7c Compare May 16, 2026 17:53

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch from ed84f7c to 9a04463 Compare May 17, 2026 04:02

Conversation

omer-vishlitzky commented May 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why AAP is different

Test plan

Uh oh!

openshift-ci-robot commented May 16, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why AAP is different

Test plan

Uh oh!

coderabbitai Bot commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Uh oh!

openshift-ci Bot commented May 16, 2026

Uh oh!

omer-vishlitzky commented May 16, 2026

Uh oh!

openshift-merge-bot Bot commented May 16, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 18, 2026

Uh oh!

openshift-merge-bot Bot commented May 18, 2026

Uh oh!

openshift-ci Bot commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

omer-vishlitzky commented May 16, 2026 •

edited by coderabbitai Bot

Loading

openshift-ci-robot commented May 16, 2026 •

edited by atlassian Bot

Loading

coderabbitai Bot commented May 16, 2026 •

edited

Loading