Push limit to leaf stage by default for DISTINCT / no-aggregate GROUP BY#18598
Open
yashmayya wants to merge 1 commit into
Open
Push limit to leaf stage by default for DISTINCT / no-aggregate GROUP BY#18598yashmayya wants to merge 1 commit into
yashmayya wants to merge 1 commit into
Conversation
yashmayya
added
backward-incompat
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #18598 +/- ##
============================================
+ Coverage 64.26% 64.29% +0.02%
Complexity 1137 1137
============================================
Files 3335 3335
Lines 206042 206044 +2
Branches 32142 32143 +1
============================================
+ Hits 132420 132483 +63
+ Misses 62977 62913 -64
- Partials 10645 10648 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
yashmayya
force-pushed
the
default-leaf-limit-pushdown-distinct
branch
from
896b567 to
3048529
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For
SELECT DISTINCT col ... LIMIT nandGROUP BY col ... LIMIT nwithout aggregate functions, the multi-stage engine currently ships every distinct group key from each server to the intermediate stage before applying the limit. This pushes the limit (and order-by-on-key) down to the leaf aggregate by default, so each server emits at mostlimitgroups.This is safe for the no-aggregate case: each leaf produces complete group keys (no partial aggregation), so leaf-level trimming is exact for ordered queries and a valid subset for unordered ones. Queries with aggregate functions are unchanged — they remain gated behind the existing
is_enable_group_trimhint/config. Limited to a single group set, soROLLUP/CUBE/GROUPING SETSare excluded. Opt out per query with/*+ aggOptions(is_enable_group_trim='false') */.Note: for an unordered
... LIMIT(noORDER BY), the specific rows returned may differ from before — this is already unspecified in SQL and was non-deterministic previously.Covered by new planner plan tests (DISTINCT/GROUP BY + LIMIT, ORDER BY on key, HAVING, OFFSET, opt-out hint, and a multi-group-set negative case) and
GroupByOptionsTestintegration tests for paginated DISTINCT/GROUP BY.