Skip to content

[Correctness] ExistenceJoin/LeftAnti + BuildLeft produces incorrect result when probe side is empty #1807

@xumingming

Description

@xumingming

The following test fails with Auron enabled:

https://github.com/apache/spark/blob/b45f071cd1b653731ae5b1c97c77d1f47560b177/sql/core/src/test/scala/org/apache/spark/sql/execution/joins/ExistenceJoinSuite.scala#L256-L263

stacktrace:

ScalaTestFailureLocation: org.apache.spark.sql.execution.SparkPlanTest at (SparkPlanTest.scala:95)
org.scalatest.exceptions.TestFailedException: 
 Results do not match for Spark plan:
 BroadcastNestedLoopJoin BuildLeft, LeftAnti
:- BroadcastExchange IdentityBroadcastMode, [plan_id=3179]
:  +- Scan ExistingRDD[a#15,b#16]
+- LocalTableScan <empty>

 
 == Results ==
 !== Expected Answer - 8 ==   == Actual Answer - 0 ==
![1,2.0]                     
![1,2.0]                     
![2,1.0]                     
![2,1.0]                     
![3,3.0]                     
![6,null]                    
![null,5.0]                  
![null,null]                 
      

Metadata

Metadata

Assignees

No one assigned

    Labels

    correctnessa query provides different output from spark

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions