Skip to content

HIVE-29598: Fix vectorized outer join wrong results due to stale scratch column values #6486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ryukobayashi wants to merge 1 commit into
base: master
Choose a base branch
from
+24 −0
Open

HIVE-29598: Fix vectorized outer join wrong results due to stale scratch column values #6486

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions ...g/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinOuterGenerateResultOperator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@
import org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainer;
import org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector;
import org.apache.hadoop.hive.ql.exec.vector.ColumnVector;
import org.apache.hadoop.hive.ql.exec.vector.DoubleColumnVector;
import org.apache.hadoop.hive.ql.exec.vector.IntervalDayTimeColumnVector;
import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
import org.apache.hadoop.hive.ql.exec.vector.TimestampColumnVector;
import org.apache.hadoop.hive.ql.exec.vector.VectorizationContext;
import org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch;
import org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression;
Expand Down Expand Up @@ -590,13 +593,15 @@ protected void generateOuterNulls(VectorizedRowBatch batch, int[] noMatchs,
ColumnVector colVector = batch.cols[column];
colVector.noNulls = false;
colVector.isNull[batchIndex] = true;
clearVectorValue(colVector, batchIndex);
}

// Small table values are set to null.
for (int column : smallTableValueColumnMap) {
ColumnVector colVector = batch.cols[column];
colVector.noNulls = false;
colVector.isNull[batchIndex] = true;
clearVectorValue(colVector, batchIndex);
}
}
}
Expand Down Expand Up @@ -749,13 +754,32 @@ protected void generateOuterNullsRepeatedAll(VectorizedRowBatch batch) throws Hi
colVector.noNulls = false;
colVector.isNull[0] = true;
colVector.isRepeating = true;
clearVectorValue(colVector, 0);
}

for (int column : smallTableValueColumnMap) {
ColumnVector colVector = batch.cols[column];
colVector.noNulls = false;
colVector.isNull[0] = true;
colVector.isRepeating = true;
clearVectorValue(colVector, 0);
}
}

private static void clearVectorValue(ColumnVector colVector, int index) {
if (colVector instanceof LongColumnVector) {
((LongColumnVector) colVector).vector[index] = 0L;
} else if (colVector instanceof DoubleColumnVector) {
((DoubleColumnVector) colVector).vector[index] = 0.0;
} else if (colVector instanceof BytesColumnVector) {
BytesColumnVector bcv = (BytesColumnVector) colVector;
bcv.vector[index] = null;
bcv.start[index] = 0;
bcv.length[index] = 0;
} else if (colVector instanceof TimestampColumnVector) {
((TimestampColumnVector) colVector).setNullValue(index);
} else if (colVector instanceof IntervalDayTimeColumnVector) {
((IntervalDayTimeColumnVector) colVector).setNullValue(index);
Comment on lines +769 to +782
Copy link
Copy Markdown
Contributor

@soumyakanti3578 soumyakanti3578 May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please confirm that these are the only ColumnVectors that can appear in smallTableValueColumnMap and outerSmallTableKeyColumnMap?
Maybe we should do the same for all ColumnVector types, and it would be better to do this in the individual classes instead of handling it here.

}
}

Expand Down
Loading