Add t-digest 3.2 vs 3.3 comparison and merge-order reproducer#18166
Open
xiangfu0 wants to merge 1 commit intoapache:claude/crazy-wilburfrom
Open
Add t-digest 3.2 vs 3.3 comparison and merge-order reproducer#18166xiangfu0 wants to merge 1 commit intoapache:claude/crazy-wilburfrom
xiangfu0 wants to merge 1 commit intoapache:claude/crazy-wilburfrom
Conversation
9 tasks
36b7322 to
2da64c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
com.tdunning:t-digest:3.2into the test target so both versions can be exercised side-by-side in the same Surefire JVMValidation
./mvnw -pl pinot-segment-local -Dtest=TDigestVersionComparisonTest,TDigestMergeOrderReproducerTest -Dsurefire.failIfNoSpecifiedTests=false test10/10)./mvnw -pl pinot-segment-local spotless:apply./mvnw -pl pinot-segment-local checkstyle:check./mvnw -pl pinot-segment-local license:format./mvnw -pl pinot-segment-local license:checkComparison Signal
On the minimized exact-quantile comparison scenario, the deterministic dataset shows a clear gap:
3.2 @ compression 100: about0.000074max normalized error with121centroids3.3 @ compression 100: about0.005094max normalized error with55centroids3.3 @ compression 150: about0.000049max normalized error with79centroidsContext
This PR is intentionally split out from #18103 so the reproducer and the direct 3.2 vs 3.3 comparison can be reviewed independently from the production compression change.