Skip to content

Drop fallback release queries from 3 to 1.#3553

Open
dgoodwin wants to merge 1 commit into
openshift:mainfrom
dgoodwin:reduce-fallback-releases
Open

Drop fallback release queries from 3 to 1.#3553
dgoodwin wants to merge 1 commit into
openshift:mainfrom
dgoodwin:reduce-fallback-releases

Conversation

@dgoodwin
Copy link
Copy Markdown
Contributor

@dgoodwin dgoodwin commented May 22, 2026

This change stops checking 4 total prior releases for basis, and instead
checks only 2. Based on the data I can see and observed behavior, we are
not benefiting much from being this sensitive. Flakes are not at true
consistent rates, if there's a problem, we will typically see it at some
point in time and now have many people monitoring for problems.

Data shows that less 2-4% of regressions matched beyond the prior two
base releases, and this does not account for those that also would have
matched against a newer one, we just found a better pass percentage by
going further. (I do not have this data)

Reducing to just check our base release plus one fallback will save
considerable query costs, and still catch over 96% of regressions we
catch today, factoring in those that would still match we're probably
around 98-99%.

Testing is preserved for 3 so we can keep the functionality if we want it again.

Data from regression tracker on what release we matched against:

sippy_openshift=> select base_release, count(base_release) from test_regressions where release='4.21' and base_release != '4.21' group by base_release order by base_release;
base_release | count
--------------+-------
4.17 | 170
4.18 | 100
4.19 | 481
4.20 | 5764
(4 rows)

sippy_openshift=> select base_release, count(base_release) from test_regressions where release='4.22' and base_release != '4.22' group by base_release order by base_release;
base_release | count
--------------+-------
4.18 | 26
4.19 | 131
4.20 | 649
4.21 | 2923
(4 rows)

sippy_openshift=> select base_release, count(base_release) from test_regressions where release='5.0' and base_release != '5.0' group by base_release order by base_release;
base_release | count
--------------+-------
4.19 | 11
4.20 | 20
4.21 | 529
4.22 | 795
(4 rows)

This change stops checking 4 total prior releases for basis, and instead
checks only 2. Based on the data I can see and observed behavior, we are
not benefiting much from being this sensitive. Flakes are not at true
consistent rates, if there's a problem, we will typically see it at some
point in time and now have many people monitoring for problems.

Data shows that less 2-4% of regressions matched beyond the prior two
base releases, and this does not account for those that also would have
matched against a newer one, we just found a better pass percentage by
going further. (I do not have this data)

Reducing to just check our base release plus one fallback will save
considerable query costs, and still catch over 96% of regressions we
catch today, factoring in those that would still match we're probably
around 98-99%.
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 22, 2026

Walkthrough

The PR parameterizes the fallback release count in the component readiness middleware. A new constant controls the default fallback limit, a wrapper function is introduced, and the underlying calculateFallbackReleases function now accepts a maxReleases parameter instead of using a hardcoded value. The test is updated to validate the parameterized behavior.

Changes

Fallback Release Parameterization

Layer / File(s) Summary
Parameterized fallback release selection
pkg/api/componentreadiness/middleware/releasefallback/releasefallback.go
A new defaultFallbackReleases constant is introduced to control the fallback count. The calculateFallbackReleases function is updated to accept a maxReleases parameter and iterate up to that limit. A new calculateDefaultFallbackReleases wrapper function passes the constant to the updated function, and the call site in getTestFallbackReleases switches to use this wrapper.
Test parameter validation
pkg/api/componentreadiness/middleware/releasefallback/releasefallback_test.go
TestCalculateFallbackReleases is updated to pass an explicit limit parameter (3) to calculateFallbackReleases, validating that the function correctly constrains results to the specified max release count.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 15 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Coverage For New Features ⚠️ Warning New function calculateDefaultFallbackReleases is untested. This pure function wraps calculateFallbackReleases with defaultFallbackReleases constant and is directly called from production code. Add unit test for calculateDefaultFallbackReleases verifying it returns exactly 1 fallback release when maxReleases=1 per the new requirement.
✅ Passed checks (15 passed)
Check name Status Explanation
Title check ✅ Passed The title directly and accurately summarizes the main change: reducing fallback release queries from 3 to 1, which is the core modification across both files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Go Error Handling ✅ Passed New code properly checks errors from PreviousRelease(). No panic calls or unhandled nil dereferences. Pre-existing json.Marshal error ignore unchanged.
Sql Injection Prevention ✅ Passed PR changes control fallback release count logic without constructing SQL. Queries are delegated to DataProvider using parameterized placeholders.
Excessive Css In React Should Use Styles ✅ Passed This PR modifies Go backend middleware, not React components. The custom check for excessive inline CSS in React is not applicable to Go code.
Single Responsibility And Clear Naming ✅ Passed PR introduces clear names for constant and functions, improving clarity through better naming and extracting hardcoded values into well-named constants.
Stable And Deterministic Test Names ✅ Passed No Ginkgo tests found in the PR. Modified files use standard Go testing with stable, descriptive test names only.
Test Structure And Quality ✅ Passed Test file uses standard Go testing framework (testing.T), not Ginkgo. Custom check targets Ginkgo test code only (BeforeEach/AfterEach, It blocks), which is not applicable here.
Microshift Test Compatibility ✅ Passed No Ginkgo e2e tests are added in this PR. Changes are limited to middleware refactoring and standard Go unit tests, so MicroShift compatibility check does not apply.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests added. PR modifies backend middleware and its unit tests only, using Go's standard testing package, not Ginkgo.
Topology-Aware Scheduling Compatibility ✅ Passed This PR modifies Sippy (a CI analysis tool) test result query logic, not Kubernetes deployments, operators, or scheduling constraints. No topology-aware scheduling check required.
Ote Binary Stdout Contract ✅ Passed Code uses logrus library (not stdout), no fmt.Print/Println/Printf calls, and all logging occurs within methods/goroutines, not in process-level code like main(), init(), TestMain(), or BeforeSuite().
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests are added. PR modifies application middleware and unit tests using standard Go testing.T, not Ginkgo framework.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from smg247 and sosiouxme May 22, 2026 17:58
@dgoodwin
Copy link
Copy Markdown
Contributor Author

/hold

Checking what this would do to 4.22 and 5.0 today

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 22, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgoodwin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels May 22, 2026
@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label May 22, 2026
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@dgoodwin
Copy link
Copy Markdown
Contributor Author

/hold cancel

No change to regression counts for 4.22 or 5.0 on the active boards when reducing from 3 to 1.

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 22, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 22, 2026

@dgoodwin: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant