Skip to content

HIVE-29473: calculate UDTF stats without inheriting parent SELECT stats #6478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
konstantinb wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

HIVE-29473: calculate UDTF stats without inheriting parent SELECT stats #6478

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
119b0e6
HIVE-29473: calculate UDTF stats without inheriting parent SELECT stats
konstantinb May 12, 2026
448d910
HIVE-29473: SQ feedback + moved unit tests to the existing file + reg…
konstantinb May 14, 2026
adc46bc
Merge remote-tracking branch 'origin/master' into HIVE-29473-udtf-don…
konstantinb May 14, 2026
680e0f0
HIVE-29473: refining UDTF stat objects to reduce execution plan changes
konstantinb May 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
245 changes: 98 additions & 147 deletions iceberg/iceberg-handler/src/test/results/positive/dynamic_partition_writes.q.out
View file
Open in desktop

Large diffs are not rendered by default.

136 changes: 68 additions & 68 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_insert_into_partition.q.out
View file
Open in desktop

Large diffs are not rendered by default.

108 changes: 54 additions & 54 deletions .../iceberg-handler/src/test/results/positive/iceberg_insert_into_partition_transforms.q.out
View file
Open in desktop

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions ...berg-handler/src/test/results/positive/iceberg_insert_into_partition_with_evolution.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,15 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
UDTF Operator
Statistics: Num rows: 1 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
function name: inline
Select Operator
expressions: col1 (type: int), 'rtyuiy' (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.iceberg.mr.hive.HiveIcebergInputFormat
output format: org.apache.iceberg.mr.hive.HiveIcebergOutputFormat
Expand All @@ -87,7 +87,7 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: int), _col1 (type: string)
outputColumnNames: a, b
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(a), max(a), count(1), count(a), compute_bit_vector_hll(a), max(length(b)), avg(COALESCE(length(b),0)), count(b), compute_bit_vector_hll(b)
keys: iceberg_truncate(b, 2) (type: string)
Expand Down
40 changes: 20 additions & 20 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_insert_overwrite_partition.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,15 +374,15 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
UDTF Operator
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
function name: inline
Select Operator
expressions: col1 (type: string), col2 (type: int), 'USA' (type: string), 'CA' (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 1 Data size: 173 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 361 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 173 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 361 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.iceberg.mr.hive.HiveIcebergInputFormat
output format: org.apache.iceberg.mr.hive.HiveIcebergOutputFormat
Expand All @@ -391,7 +391,7 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: string), _col1 (type: int), _col2 (type: string), _col3 (type: string)
outputColumnNames: name, age, country, state
Statistics: Num rows: 1 Data size: 173 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 361 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(length(name)), avg(COALESCE(length(name),0)), count(1), count(name), compute_bit_vector_hll(name), min(age), max(age), count(age), compute_bit_vector_hll(age), max(length(country)), avg(COALESCE(length(country),0)), count(country), compute_bit_vector_hll(country), max(length(state)), avg(COALESCE(length(state),0)), count(state), compute_bit_vector_hll(state)
keys: country (type: string), state (type: string)
Expand Down Expand Up @@ -1390,15 +1390,15 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
UDTF Operator
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
function name: inline
Select Operator
expressions: col1 (type: bigint), col2 (type: int), DATE'1999-12-31' (type: date)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.iceberg.mr.hive.HiveIcebergInputFormat
output format: org.apache.iceberg.mr.hive.HiveIcebergOutputFormat
Expand All @@ -1407,7 +1407,7 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: bigint), _col1 (type: int), _col2 (type: date)
outputColumnNames: bigintcol, intcol, pcol
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(bigintcol), max(bigintcol), count(1), count(bigintcol), compute_bit_vector_hll(bigintcol), min(intcol), max(intcol), count(intcol), compute_bit_vector_hll(intcol), min(pcol), max(pcol), count(pcol), compute_bit_vector_hll(pcol)
keys: pcol (type: date)
Expand Down Expand Up @@ -1858,15 +1858,15 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
UDTF Operator
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
function name: inline
Select Operator
expressions: CAST( col1 AS DATE) (type: date), col2 (type: int), 34567890123456787L (type: bigint)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.iceberg.mr.hive.HiveIcebergInputFormat
output format: org.apache.iceberg.mr.hive.HiveIcebergOutputFormat
Expand All @@ -1875,7 +1875,7 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: date), _col1 (type: int), _col2 (type: bigint)
outputColumnNames: datecol, intcol, pcol
Statistics: Num rows: 1 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(datecol), max(datecol), count(1), count(datecol), compute_bit_vector_hll(datecol), min(intcol), max(intcol), count(intcol), compute_bit_vector_hll(intcol), min(pcol), max(pcol), count(pcol), compute_bit_vector_hll(pcol)
keys: pcol (type: bigint)
Expand Down Expand Up @@ -2326,15 +2326,15 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
UDTF Operator
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
function name: inline
Select Operator
expressions: CAST( col1 AS DATE) (type: date), col2 (type: int), 3.14786D (type: double)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.iceberg.mr.hive.HiveIcebergInputFormat
output format: org.apache.iceberg.mr.hive.HiveIcebergOutputFormat
Expand All @@ -2343,7 +2343,7 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: date), _col1 (type: int), _col2 (type: double)
outputColumnNames: datecol, intcol, pcol
Statistics: Num rows: 1 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 68 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(datecol), max(datecol), count(1), count(datecol), compute_bit_vector_hll(datecol), min(intcol), max(intcol), count(intcol), compute_bit_vector_hll(intcol), min(pcol), max(pcol), count(pcol), compute_bit_vector_hll(pcol)
keys: pcol (type: double)
Expand Down Expand Up @@ -2794,15 +2794,15 @@ STAGE PLANS:
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
UDTF Operator
Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
function name: inline
Select Operator
expressions: CAST( col1 AS DATE) (type: date), col2 (type: int), 3.14786 (type: decimal(10,6))
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 168 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 168 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.iceberg.mr.hive.HiveIcebergInputFormat
output format: org.apache.iceberg.mr.hive.HiveIcebergOutputFormat
Expand All @@ -2811,7 +2811,7 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: date), _col1 (type: int), _col2 (type: decimal(10,6))
outputColumnNames: datecol, intcol, pcol
Statistics: Num rows: 1 Data size: 168 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(datecol), max(datecol), count(1), count(datecol), compute_bit_vector_hll(datecol), min(intcol), max(intcol), count(intcol), compute_bit_vector_hll(intcol), min(pcol), max(pcol), count(pcol), compute_bit_vector_hll(pcol)
keys: pcol (type: decimal(10,6))
Expand Down
Loading
Loading