Skip to content

[opt](nereids) adjust Left join cost factor:probeShortcutFactor #60183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
englefly wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

[opt](nereids) adjust Left join cost factor:probeShortcutFactor #60183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0b86d9b
adjust probeShortcutFactor
englefly Jan 23, 2026
10988d2
add-case
englefly Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostModel.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,8 @@ public Cost visitPhysicalHashJoin(
);
}
double probeShortcutFactor = 1.0;
if (ConnectContext.get() != null && ConnectContext.get().getStatementContext() != null
if (rightRowCount < 10 * leftRowCount
&& ConnectContext.get() != null && ConnectContext.get().getStatementContext() != null
&& !ConnectContext.get().getStatementContext().isHasUnknownColStats()
&& physicalHashJoin.getJoinType().isLeftSemiOrAntiJoin()
&& physicalHashJoin.getOtherJoinConjuncts().isEmpty()
Expand Down
12 changes: 6 additions & 6 deletions regression-test/data/shape_check/tpcds_sf100/constraints/query23.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,29 +53,29 @@ PhysicalCteAnchor ( cteId=CTEId#0 )
--------------hashAgg[LOCAL]
----------------PhysicalUnion
------------------PhysicalProject
--------------------hashJoin[LEFT_SEMI_JOIN shuffle] hashCondition=((catalog_sales.cs_item_sk = frequent_ss_items.item_sk)) otherCondition=() build RFs:RF5 item_sk->[cs_item_sk]
--------------------hashJoin[RIGHT_SEMI_JOIN shuffle] hashCondition=((catalog_sales.cs_item_sk = frequent_ss_items.item_sk)) otherCondition=() build RFs:RF5 cs_item_sk->[item_sk]
----------------------PhysicalCteConsumer ( cteId=CTEId#0 ) apply RFs: RF5
----------------------PhysicalProject
------------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((catalog_sales.cs_bill_customer_sk = best_ss_customer.c_customer_sk)) otherCondition=() build RFs:RF4 c_customer_sk->[cs_bill_customer_sk]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF3 d_date_sk->[cs_sold_date_sk]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF3 RF4 RF5
--------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF3 RF4
------------------------------PhysicalProject
--------------------------------filter((date_dim.d_moy = 5) and (date_dim.d_year = 2000))
----------------------------------PhysicalOlapScan[date_dim]
--------------------------PhysicalCteConsumer ( cteId=CTEId#2 )
----------------------PhysicalCteConsumer ( cteId=CTEId#0 )
------------------PhysicalProject
--------------------hashJoin[LEFT_SEMI_JOIN shuffle] hashCondition=((web_sales.ws_item_sk = frequent_ss_items.item_sk)) otherCondition=() build RFs:RF8 item_sk->[ws_item_sk]
--------------------hashJoin[RIGHT_SEMI_JOIN shuffle] hashCondition=((web_sales.ws_item_sk = frequent_ss_items.item_sk)) otherCondition=() build RFs:RF8 ws_item_sk->[item_sk]
----------------------PhysicalCteConsumer ( cteId=CTEId#0 ) apply RFs: RF8
----------------------PhysicalProject
------------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((web_sales.ws_bill_customer_sk = best_ss_customer.c_customer_sk)) otherCondition=() build RFs:RF7 c_customer_sk->[ws_bill_customer_sk]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF6 d_date_sk->[ws_sold_date_sk]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[web_sales] apply RFs: RF6 RF7 RF8
--------------------------------PhysicalOlapScan[web_sales] apply RFs: RF6 RF7
------------------------------PhysicalProject
--------------------------------filter((date_dim.d_moy = 5) and (date_dim.d_year = 2000))
----------------------------------PhysicalOlapScan[date_dim]
--------------------------PhysicalCteConsumer ( cteId=CTEId#2 )
----------------------PhysicalCteConsumer ( cteId=CTEId#0 )

34 changes: 17 additions & 17 deletions regression-test/data/shape_check/tpcds_sf100/rf_prune/query33.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,62 +9,65 @@ PhysicalResultSink
------------hashAgg[LOCAL]
--------------PhysicalUnion
----------------PhysicalProject
------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((item.i_manufact_id = item.i_manufact_id)) otherCondition=() build RFs:RF3 i_manufact_id->[i_manufact_id]
------------------hashJoin[RIGHT_SEMI_JOIN shuffleBucket] hashCondition=((item.i_manufact_id = item.i_manufact_id)) otherCondition=() build RFs:RF3 i_manufact_id->[i_manufact_id]
--------------------PhysicalProject
----------------------filter((item.i_category = 'Home'))
------------------------PhysicalOlapScan[item] apply RFs: RF3
--------------------hashAgg[GLOBAL]
----------------------PhysicalDistribute[DistributionSpecHash]
------------------------hashAgg[LOCAL]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((store_sales.ss_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF2 i_item_sk->[ss_item_sk]
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((store_sales.ss_item_sk = item.i_item_sk)) otherCondition=()
------------------------------PhysicalProject
--------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF1 ca_address_sk->[ss_addr_sk]
----------------------------------PhysicalProject
------------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[ss_sold_date_sk]
--------------------------------------PhysicalProject
----------------------------------------PhysicalOlapScan[store_sales] apply RFs: RF0 RF1 RF2
----------------------------------------PhysicalOlapScan[store_sales] apply RFs: RF0 RF1
--------------------------------------PhysicalProject
----------------------------------------filter((date_dim.d_moy = 1) and (date_dim.d_year = 2002))
------------------------------------------PhysicalOlapScan[date_dim]
----------------------------------PhysicalProject
------------------------------------filter((customer_address.ca_gmt_offset = -5.00))
--------------------------------------PhysicalOlapScan[customer_address]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[item] apply RFs: RF3
--------------------------------PhysicalOlapScan[item]
----------------PhysicalProject
------------------hashJoin[RIGHT_SEMI_JOIN shuffleBucket] hashCondition=((item.i_manufact_id = item.i_manufact_id)) otherCondition=() build RFs:RF7 i_manufact_id->[i_manufact_id]
--------------------PhysicalProject
----------------------filter((item.i_category = 'Home'))
------------------------PhysicalOlapScan[item]
----------------PhysicalProject
------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((item.i_manufact_id = item.i_manufact_id)) otherCondition=() build RFs:RF7 i_manufact_id->[i_manufact_id]
------------------------PhysicalOlapScan[item] apply RFs: RF7
--------------------hashAgg[GLOBAL]
----------------------PhysicalDistribute[DistributionSpecHash]
------------------------hashAgg[LOCAL]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((catalog_sales.cs_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF6 i_item_sk->[cs_item_sk]
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((catalog_sales.cs_item_sk = item.i_item_sk)) otherCondition=()
------------------------------PhysicalProject
--------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_bill_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF5 ca_address_sk->[cs_bill_addr_sk]
----------------------------------PhysicalProject
------------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF4 d_date_sk->[cs_sold_date_sk]
--------------------------------------PhysicalProject
----------------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF4 RF5 RF6
----------------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF4 RF5
--------------------------------------PhysicalProject
----------------------------------------filter((date_dim.d_moy = 1) and (date_dim.d_year = 2002))
------------------------------------------PhysicalOlapScan[date_dim]
----------------------------------PhysicalProject
------------------------------------filter((customer_address.ca_gmt_offset = -5.00))
--------------------------------------PhysicalOlapScan[customer_address]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[item] apply RFs: RF7
--------------------------------PhysicalOlapScan[item]
----------------PhysicalProject
------------------hashJoin[RIGHT_SEMI_JOIN shuffleBucket] hashCondition=((item.i_manufact_id = item.i_manufact_id)) otherCondition=() build RFs:RF11 i_manufact_id->[i_manufact_id]
--------------------PhysicalProject
----------------------filter((item.i_category = 'Home'))
------------------------PhysicalOlapScan[item]
----------------PhysicalProject
------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((item.i_manufact_id = item.i_manufact_id)) otherCondition=() build RFs:RF11 i_manufact_id->[i_manufact_id]
------------------------PhysicalOlapScan[item] apply RFs: RF11
--------------------hashAgg[GLOBAL]
----------------------PhysicalDistribute[DistributionSpecHash]
------------------------hashAgg[LOCAL]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((web_sales.ws_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF10 ws_item_sk->[i_item_sk]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[item] apply RFs: RF10 RF11
--------------------------------PhysicalOlapScan[item] apply RFs: RF10
------------------------------PhysicalProject
--------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_bill_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF9 ca_address_sk->[ws_bill_addr_sk]
----------------------------------PhysicalProject
Expand All @@ -77,7 +80,4 @@ PhysicalResultSink
----------------------------------PhysicalProject
------------------------------------filter((customer_address.ca_gmt_offset = -5.00))
--------------------------------------PhysicalOlapScan[customer_address]
--------------------PhysicalProject
----------------------filter((item.i_category = 'Home'))
------------------------PhysicalOlapScan[item]

58 changes: 29 additions & 29 deletions regression-test/data/shape_check/tpcds_sf100/rf_prune/query35.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,38 +10,38 @@ PhysicalResultSink
--------------hashAgg[LOCAL]
----------------PhysicalProject
------------------filter(OR[ifnull($c$1, FALSE),ifnull($c$2, FALSE)])
--------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((c.c_customer_sk = catalog_sales.cs_ship_customer_sk)) otherCondition=()
--------------------hashJoin[RIGHT_SEMI_JOIN shuffleBucket] hashCondition=((c.c_customer_sk = catalog_sales.cs_ship_customer_sk)) otherCondition=()
----------------------PhysicalProject
------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((customer_demographics.cd_demo_sk = c.c_current_cdemo_sk)) otherCondition=()
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF5 d_date_sk->[cs_sold_date_sk]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((c.c_current_addr_sk = ca.ca_address_sk)) otherCondition=()
------------------------------hashJoin[LEFT_SEMI_JOIN bucketShuffle] hashCondition=((c.c_customer_sk = store_sales.ss_customer_sk)) otherCondition=() build RFs:RF3 ss_customer_sk->[c_customer_sk,ws_bill_customer_sk]
--------------------------------hashJoin[LEFT_SEMI_JOIN broadcast] hashCondition=((c.c_customer_sk = web_sales.ws_bill_customer_sk)) otherCondition=()
----------------------------------PhysicalProject
------------------------------------PhysicalOlapScan[customer(c)] apply RFs: RF3
----------------------------------PhysicalProject
------------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF2 d_date_sk->[ws_sold_date_sk]
--------------------------------------PhysicalProject
----------------------------------------PhysicalOlapScan[web_sales] apply RFs: RF2 RF3
--------------------------------------PhysicalProject
----------------------------------------filter((date_dim.d_qoy < 4) and (date_dim.d_year = 2001))
------------------------------------------PhysicalOlapScan[date_dim]
--------------------------------PhysicalProject
----------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF1 d_date_sk->[ss_sold_date_sk]
------------------------------------PhysicalProject
--------------------------------------PhysicalOlapScan[store_sales] apply RFs: RF1
------------------------------------PhysicalProject
--------------------------------------filter((date_dim.d_qoy < 4) and (date_dim.d_year = 2001))
----------------------------------------PhysicalOlapScan[date_dim]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[customer_address(ca)]
--------------------------PhysicalProject
----------------------------PhysicalOlapScan[customer_demographics]
----------------------PhysicalProject
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[cs_sold_date_sk]
--------------------------PhysicalProject
----------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF0
----------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF5
--------------------------PhysicalProject
----------------------------filter((date_dim.d_qoy < 4) and (date_dim.d_year = 2001))
------------------------------PhysicalOlapScan[date_dim]
----------------------hashJoin[LEFT_SEMI_JOIN bucketShuffle] hashCondition=((c.c_customer_sk = web_sales.ws_bill_customer_sk)) otherCondition=()
------------------------hashJoin[RIGHT_SEMI_JOIN shuffle] hashCondition=((c.c_customer_sk = store_sales.ss_customer_sk)) otherCondition=()
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF3 d_date_sk->[ss_sold_date_sk]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[store_sales] apply RFs: RF3
------------------------------PhysicalProject
--------------------------------filter((date_dim.d_qoy < 4) and (date_dim.d_year = 2001))
----------------------------------PhysicalOlapScan[date_dim]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((customer_demographics.cd_demo_sk = c.c_current_cdemo_sk)) otherCondition=()
------------------------------PhysicalProject
--------------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((c.c_current_addr_sk = ca.ca_address_sk)) otherCondition=()
----------------------------------PhysicalProject
------------------------------------PhysicalOlapScan[customer(c)]
----------------------------------PhysicalProject
------------------------------------PhysicalOlapScan[customer_address(ca)]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[customer_demographics]
------------------------PhysicalProject
--------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[ws_sold_date_sk]
----------------------------PhysicalProject
------------------------------PhysicalOlapScan[web_sales] apply RFs: RF0
----------------------------PhysicalProject
------------------------------filter((date_dim.d_qoy < 4) and (date_dim.d_year = 2001))
--------------------------------PhysicalOlapScan[date_dim]

Loading