Skip to content

OCPBUGS-74151: Add test for CPMS OnDelete strategy with full master replacement#30760

Open
hasbro17 wants to merge 1 commit intoopenshift:mainfrom
hasbro17:delete-all-scaling-test
Open

OCPBUGS-74151: Add test for CPMS OnDelete strategy with full master replacement#30760
hasbro17 wants to merge 1 commit intoopenshift:mainfrom
hasbro17:delete-all-scaling-test

Conversation

@hasbro17
Copy link
Contributor

@hasbro17 hasbro17 commented Feb 5, 2026

E2E test for openshift/cluster-etcd-operator#1540

Creates a new test case that validates the ControlPlaneMachineSet OnDelete strategy by deleting all three master machines simultaneously and verifying CPMS correctly replaces them while maintaining cluster health.

The test switches CPMS to OnDelete strategy, deletes all master machines, and validates that CPMS creates replacements with proper etcd membership transitions. Verifies that all old etcd members are removed from both the cluster and etcd-endpoints ConfigMap, and new members are properly integrated.

TODO: need to wire up the vertical scaling workflow in the openshift/release repo so that this test runs in its own job/presubmit and gets skipped in the regular etcd scaling.

@openshift-ci-robot
Copy link

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 5, 2026
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 5, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hasbro17

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 5, 2026
@openshift-ci openshift-ci bot requested review from dusk125 and tjungblu February 5, 2026 07:37
@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 5, 2026

/testwith openshift/cluster-etcd-operator/main/e2e-aws-ovn-etcd-scaling openshift/cluster-etcd-operator#1540

@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 5, 2026

Not sure if multipr tests work on presubmits like that but hopefully that goes through.

@openshift-ci-robot
Copy link

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@hasbro17 hasbro17 force-pushed the delete-all-scaling-test branch from 533dead to 2df8f6a Compare February 5, 2026 23:33
@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 5, 2026

/testwith openshift/cluster-etcd-operator/main/e2e-aws-ovn-etcd-scaling openshift/cluster-etcd-operator#1540

@openshift-ci-robot
Copy link

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 6, 2026

@hasbro17: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-microshift 2df8f6a link true /test e2e-aws-ovn-microshift

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt
Copy link

openshift-trt bot commented Feb 6, 2026

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 2df8f6a

  • "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account vmware-vsphere-csi-driver-operator should not create excessive watch requests" [Total: 2, Pass: 2, Fail: 0, Flake: 0]
  • "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account vsphere-problem-detector-operator should not create excessive watch requests" [Total: 2, Pass: 2, Fail: 0, Flake: 0]

@hasbro17 hasbro17 force-pushed the delete-all-scaling-test branch from 2df8f6a to e24dcd0 Compare February 6, 2026 07:05
@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

/testwith openshift/cluster-etcd-operator/main/e2e-aws-ovn-etcd-scaling openshift/cluster-etcd-operator#1540

@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

/retitle OCPBUGS-74151: Add test for CPMS OnDelete strategy with full master replacement

@openshift-ci openshift-ci bot changed the title WIP: Add test for CPMS OnDelete strategy with full master replacement OCPBUGS-74151: Add test for CPMS OnDelete strategy with full master replacement Feb 6, 2026
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 6, 2026
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Feb 6, 2026
@openshift-ci-robot
Copy link

@hasbro17: This pull request references Jira Issue OCPBUGS-74151, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @geliu2016

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

E2E test for openshift/cluster-etcd-operator#1540

Creates a new test case that validates the ControlPlaneMachineSet OnDelete strategy by deleting all three master machines simultaneously and verifying CPMS correctly replaces them while maintaining cluster health.

The test switches CPMS to OnDelete strategy, deletes all master machines, and validates that CPMS creates replacements with proper etcd membership transitions. Verifies that all old etcd members are removed from both the cluster and etcd-endpoints ConfigMap, and new members are properly integrated.

TODO: need to wire up the vertical scaling workflow in the openshift/release repo so that this test runs in its own job/presubmit and gets skipped in the regular etcd scaling.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from geliu2016 February 6, 2026 07:13
@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

/cc @jubittajohn

@openshift-ci openshift-ci bot requested a review from jubittajohn February 6, 2026 07:13
@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

This test will fail as is without openshift/cluster-etcd-operator#1540
You can see the multiPR test result with that change in
#30760 (comment)
https://github.com/openshift/origin/pull/30760/checks

Once this is in, we'll run the scaling presubmit on openshift/cluster-etcd-operator#1540 to verify that change.

Creates a new test case that validates the ControlPlaneMachineSet OnDelete
strategy by deleting all three master machines simultaneously and verifying
CPMS correctly replaces them while maintaining cluster health.

The test switches CPMS to OnDelete strategy, deletes all master machines,
and validates that CPMS creates replacements with proper etcd membership
transitions. Verifies that all old etcd members are removed from both the
cluster and etcd-endpoints ConfigMap, and new members are properly integrated.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@hasbro17 hasbro17 force-pushed the delete-all-scaling-test branch from e24dcd0 to b5686a4 Compare February 6, 2026 07:19
@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

/testwith openshift/cluster-etcd-operator/main/e2e-aws-ovn-etcd-scaling openshift/cluster-etcd-operator#1540

@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

Updating the OWNERS while we're here.

@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

/cherry-pick release-4.21 release-4.20 release-4.19 release-4.18

@openshift-cherrypick-robot

@hasbro17: once the present PR merges, I will cherry-pick it on top of release-4.21 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick release-4.21 release-4.20 release-4.19 release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot
Copy link

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-trt
Copy link

openshift-trt bot commented Feb 6, 2026

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: b5686a4

  • "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account vmware-vsphere-csi-driver-operator should not create excessive watch requests" [Total: 2, Pass: 2, Fail: 0, Flake: 0]
  • "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account vsphere-problem-detector-operator should not create excessive watch requests" [Total: 2, Pass: 2, Fail: 0, Flake: 0]

@hasbro17
Copy link
Contributor Author

hasbro17 commented Feb 6, 2026

/testwith openshift/cluster-etcd-operator/main/e2e-aws-ovn-etcd-scaling openshift/cluster-etcd-operator#1540

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants