Controller cleanups: SegmentDeletionManager fallback, LLC filter, retention strategy extraction, mock util#18582
Open
krishan1390 wants to merge 5 commits into
Open
Controller cleanups: SegmentDeletionManager fallback, LLC filter, retention strategy extraction, mock util#18582krishan1390 wants to merge 5 commits into
krishan1390 wants to merge 5 commits into
Conversation
…znode for any reason
…tZKMetadata list Avoids a redundant ZK fetch when the caller already holds the list of SegmentZKMetadata for the table (e.g. periodic controller tasks that scan all segments of a table). The existing one-arg form is preserved as a thin delegator, so behavior for current callers is unchanged.
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## master #18582 +/- ##
============================================
- Coverage 64.27% 56.77% -7.50%
+ Complexity 1137 7 -1130
============================================
Files 3335 2567 -768
Lines 205898 148958 -56940
Branches 32129 24099 -8030
============================================
- Hits 132333 84568 -47765
+ Misses 62931 57203 -5728
+ Partials 10634 7187 -3447
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four small, independent controller-side improvements grouped into one PR. Each lands as its own commit so reviewers can read them concern-by-concern and each is independently revertable. Please do not squash on merge — preserve per-commit granularity.
SegmentDeletionManager: fall back to recursive remove when batch ZK remove leaves a non-empty znode. The batch ZK remove API is non-recursive, so it silently fails to delete segment znodes that have accumulated children. On batch-remove failure, fall back to the single-path_propertyStore.remove(...)API, which handles non-empty nodes by deleting recursively. Skip when the znode is already absent.PinotHelixResourceManager#getLastLLCCompletedSegments: skip non-LLC-named segments. A realtime table can contain uploaded (non-LLC-named) segments inDONEstate alongside the LLC ones.LLCSegmentName.of(name)returnsnullfor those, causing an NPE on the next field access. Filter them out. Includes a regression test (PinotHelixResourceManagerLastLLCSegmentsTest) that reproduces the NPE without the fix.Extract retention strategy construction into
TableConfigRetentionUtils. Pull theTimeRetentionStrategyconstruction (with its parse / invalid-config handling) out ofRetentionManagerinto a static helper. Returningnullfor absent or malformed config preserves the existing "skip retention" behavior, and the warn-log shape is preserved (single message for both null/empty and parse-error paths).Add a partition-aware
SegmentMetadataMockUtils.mockSegmentMetadataoverload. Additive test helper that returns a mock backed by aMurmurPartitionFunction. Purely test-ergonomics.Test plan
./mvnw -pl pinot-controller -am test -Dtest=SegmentDeletionManagerTest,RetentionManagerTest,PinotHelixResourceManagerLastLLCSegmentsTest./mvnw spotless:apply checkstyle:check license:check -pl pinot-controllerDONEsegment into a realtime table and confirm retention no longer NPEs.🤖 Generated with Claude Code