[codex] Fix compatibility regression testing

[xiangfu0]

Summary

• Harden compatibility verifier Maven builds by isolating each build in its own local repository and forcing dependency updates on retries.
• Preserve caller-provided Maven CLI options through PINOT_MAVEN_OPTS instead of dropping them when the verifier adds generated settings.
• Resolve refs through local refs, origin/*, and tags after fetching tags, then fail fast if a requested ref is unknown.
• Pre-install pinot-bom after version rewriting so compatibility builds against master can resolve pinot-bom:<version>-compat-* from the same local repo.

Root Cause

The release-1.5.0 compatibility job failed before verifier tests ran because the current-tree Maven build reused the restored ~/.m2 and repeatedly hit cached remote-id / transient Maven Central resolution errors. The later master matrix jobs also failed after versions:set because the root POM imports pinot-bom:${project.version}, but pinot-bom:1.6.0-compat-* had not been installed into the rewritten build local repo.

User Manual

To rerun the same path locally or in CI, set OLD_COMMIT, WORKING_DIR, and TEST_SUITE, then run .github/workflows/scripts/.pinot_compatibility_verifier.sh. Maven CLI flags should go in PINOT_MAVEN_OPTS; JVM flags should stay in MAVEN_OPTS.

Sample Config / Queries

N/A: this change only updates CI workflow and compatibility verifier build plumbing. Existing sample suites under compatibility-verifier/sample-test-suite and compatibility-verifier/multi-stage-query-engine-test-suite are unchanged.

Validation

• bash -n .github/workflows/scripts/.pinot_compatibility_verifier.sh compatibility-verifier/checkoutAndBuild.sh
• YAML parse for pinot_compatibility_checks.yml, pinot_compatibility_tests.yml, and pinot_multi_stage_query_engine_compatibility_tests.yml
• git diff --check
• Maven option split sanity check with mvn -v

[yashmayya]

Took a close look since the Compatibility Regression against master jobs are red on this PR — they fail with the same pinot-bom:...-compat-... resolution error this change is meant to fix. Looks like an ordering issue with versions:commit; left a note inline.

[yashmayya]

versions:commit here re-reads the rewritten root pom, which imports pinot-bom:${project.version} — now ...-compat-${buildId}. But the bom isn't bumped + installed at that version until the block right below (122-126), so commit fails with Non-resolvable import POM ... pinot-bom:...-compat-.... That's exactly the error the Compatibility Regression against master job hits on this PR, so the new installPinotBom path never actually runs.

The bom versions:set + installPinotBom need to move above this commit. Reordering to set+install the bom right after the root versions:set, then commit, fixes it (reproduced + verified locally). Alternatively, drop versions:commit and pass -DgenerateBackupPoms=false on the root versions:set.

[yashmayya]

Re-reviewed the latest push. The ordering fix is clean — dropping versions:commit and setting -DgenerateBackupPoms=false on the root versions:set means the bom gets rewritten + installed before anything reads the rewritten root model, so there's no more unresolvable pinot-bom:...-compat-... import. Reproduced the new sequence locally and it resolves fine.

CI backs it up too: the Compatibility Regression against master (sample) and Multi-Stage against release-1.5.0 jobs that were red before are green now. The MSE-vs-master job is still running, but it's the same build path as the already-green sample-vs-master. LGTM 👍

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.26%. Comparing base (887dd4d) to head (cef4831).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18597      +/-   ##
============================================
- Coverage     64.28%   64.26%   -0.02%     
  Complexity     1137     1137              
============================================
  Files          3335     3335              
  Lines        205901   206042     +141     
  Branches      32129    32142      +13     
============================================
+ Hits         132358   132420      +62     
- Misses        62905    62977      +72     
- Partials      10638    10645       +7     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-21	64.26% <ø> (-0.02%)	⬇️
temurin	64.26% <ø> (-0.02%)	⬇️
unittests	64.26% <ø> (-0.02%)	⬇️
unittests1	56.78% <ø> (+0.01%)	⬆️
unittests2	36.79% <ø> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.