Skip to content

Add auto-complete flag to LineageEntry#18584

Open
krishan1390 wants to merge 1 commit into
apache:masterfrom
krishan1390:lineage-entry-auto-complete-flag
Open

Add auto-complete flag to LineageEntry#18584
krishan1390 wants to merge 1 commit into
apache:masterfrom
krishan1390:lineage-entry-auto-complete-flag

Conversation

@krishan1390
Copy link
Copy Markdown
Contributor

Summary

  • Adds a new boolean autoCompleteLineageEntry to LineageEntry. Default false preserves the existing client-driven completion semantics (endReplaceSegments).
  • When set to true, the entry signals that an observer (e.g. a controller-side periodic task) is permitted to mark the entry completed automatically once all segmentsTo are ONLINE in the external view.
  • SegmentLineage serialization is updated to be tolerant on read (accepts both 4-tuple legacy and 5-tuple new format) and conservative on write (emits a 5th element only when the flag is true), so rolling upgrades and mixed-version controllers can co-exist.
  • PinotHelixResourceManager's lineage state-transition sites (endReplaceSegments, two branches of revertReplaceSegments) preserve the flag when constructing the updated entry.
  • No consumer of the flag is added in this PR — a follow-up will introduce the observer that uses it.

Test plan

  • Extended SegmentLineageTest to round-trip a LineageEntry with autoCompleteLineageEntry=true through ZNRecord.
  • Added a test that simulates a legacy 4-tuple ZNRecord written by an older controller and asserts it parses with the flag defaulting to false.
  • Asserted that two entries differing only in the new flag are not equal.
  • Confirmed default-false entries still serialize as 4-tuples (no wire-format churn for existing users).

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 26, 2026

Codecov Report

❌ Patch coverage is 76.47059% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.27%. Comparing base (0a8ac02) to head (c36f3aa).

Files with missing lines Patch % Lines
.../org/apache/pinot/common/lineage/LineageEntry.java 75.00% 1 Missing and 1 partial ⚠️
...rg/apache/pinot/common/lineage/SegmentLineage.java 60.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             master   #18584   +/-   ##
=========================================
  Coverage     64.27%   64.27%           
  Complexity     1137     1137           
=========================================
  Files          3335     3335           
  Lines        205898   205905    +7     
  Branches      32129    32131    +2     
=========================================
+ Hits         132333   132343   +10     
+ Misses        62931    62912   -19     
- Partials      10634    10650   +16     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-21 64.27% <76.47%> (+<0.01%) ⬆️
temurin 64.27% <76.47%> (+<0.01%) ⬆️
unittests 64.27% <76.47%> (+<0.01%) ⬆️
unittests1 56.75% <69.23%> (-0.01%) ⬇️
unittests2 36.83% <64.70%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants