Skip to content

Conversation

@englefly
Copy link
Contributor

What problem does this PR solve?

probeShortcutFactor is not reasonable when right row count is much more than left row count.
tpcds affected queries(1T)
before:
query33.sql,3.252,0.490,0.454,0.412,0.412
query35.sql,12.897,1.059,1.054,1.039,1.039
query64.sql,139.298,2.788,20.533,8.234,2.788
query69.sql,3.004,0.835,0.773,0.772,0.772

after:
query33.sql,1.114,0.512,0.540,0.468,0.468
query35.sql,1.398,1.146,1.194,1.180,1.146
query64.sql,3.562,1.939,2.054,1.985,1.939
query69.sql,0.826,0.848,0.806,0.804,0.804

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 30075 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10988d2c2213e879fd4cc6777cf530a2313e4313, data reload: false

------ Round 1 ----------------------------------
q1	17604	4711	4608	4608
q2	2030	335	197	197
q3	10215	1276	714	714
q4	10223	873	311	311
q5	7509	2074	1812	1812
q6	198	174	140	140
q7	867	723	590	590
q8	9254	1292	1134	1134
q9	4949	4560	4535	4535
q10	6789	1643	1270	1270
q11	518	294	268	268
q12	337	366	219	219
q13	17770	3817	3044	3044
q14	221	232	214	214
q15	601	510	524	510
q16	653	631	595	595
q17	645	753	549	549
q18	6595	6450	6340	6340
q19	1306	965	616	616
q20	389	340	229	229
q21	2615	2003	1904	1904
q22	361	331	276	276
Total cold run time: 101649 ms
Total hot run time: 30075 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4779	4744	4759	4744
q2	311	410	317	317
q3	2152	2718	2234	2234
q4	1319	1682	1286	1286
q5	4099	3980	4049	3980
q6	215	176	131	131
q7	1892	1889	1719	1719
q8	2461	2771	2392	2392
q9	7484	7209	7164	7164
q10	2519	2764	2353	2353
q11	557	477	461	461
q12	731	739	631	631
q13	3628	4166	3445	3445
q14	313	327	298	298
q15	567	540	510	510
q16	669	699	656	656
q17	1188	1355	1410	1355
q18	8084	7995	7639	7639
q19	843	826	881	826
q20	2015	2046	1916	1916
q21	4759	4222	4120	4120
q22	572	570	524	524
Total cold run time: 51157 ms
Total hot run time: 48701 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172057 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10988d2c2213e879fd4cc6777cf530a2313e4313, data reload: false

query5	4373	626	509	509
query6	323	214	192	192
query7	4212	455	259	259
query8	323	242	228	228
query9	8655	2867	2877	2867
query10	456	332	282	282
query11	15242	15184	15057	15057
query12	177	122	120	120
query13	1272	468	363	363
query14	6273	3033	2797	2797
query14_1	2692	2632	2656	2632
query15	205	201	171	171
query16	962	499	475	475
query17	1090	671	576	576
query18	2455	430	334	334
query19	206	183	151	151
query20	125	117	114	114
query21	222	147	121	121
query22	3827	3946	4065	3946
query23	16065	15597	15360	15360
query23_1	15339	15296	15426	15296
query24	7118	1546	1156	1156
query24_1	1160	1141	1173	1141
query25	550	456	406	406
query26	1245	264	156	156
query27	2803	439	277	277
query28	4486	2167	2155	2155
query29	807	557	447	447
query30	310	243	208	208
query31	827	638	543	543
query32	83	78	78	78
query33	516	333	282	282
query34	883	884	536	536
query35	618	674	598	598
query36	873	936	832	832
query37	136	96	89	89
query38	2700	2751	2633	2633
query39	773	748	748	748
query39_1	723	716	715	715
query40	223	140	120	120
query41	72	69	68	68
query42	95	92	94	92
query43	462	464	387	387
query44	1361	759	737	737
query45	188	190	180	180
query46	839	944	612	612
query47	1383	1377	1314	1314
query48	309	314	238	238
query49	598	425	341	341
query50	656	263	201	201
query51	3764	3733	3759	3733
query52	94	93	81	81
query53	207	218	171	171
query54	280	255	260	255
query55	81	87	76	76
query56	301	293	308	293
query57	1002	1040	940	940
query58	262	253	252	252
query59	2145	2154	1949	1949
query60	324	320	310	310
query61	147	143	142	142
query62	404	369	311	311
query63	189	161	160	160
query64	4935	1128	817	817
query65	3788	3716	3753	3716
query66	1444	419	310	310
query67	15521	15645	15416	15416
query68	2378	1054	722	722
query69	427	312	284	284
query70	1005	959	956	956
query71	304	283	267	267
query72	5349	3058	3252	3058
query73	591	726	319	319
query74	8705	8710	8495	8495
query75	2284	2332	1861	1861
query76	2271	1038	660	660
query77	360	387	315	315
query78	9653	9771	9120	9120
query79	2395	902	594	594
query80	1695	509	444	444
query81	540	264	230	230
query82	995	151	118	118
query83	321	258	247	247
query84	258	124	97	97
query85	860	467	397	397
query86	409	311	279	279
query87	2887	2846	2710	2710
query88	3531	2594	2557	2557
query89	304	255	229	229
query90	1944	171	168	168
query91	167	157	130	130
query92	72	76	68	68
query93	1215	993	642	642
query94	621	311	295	295
query95	558	328	384	328
query96	650	506	230	230
query97	2330	2351	2304	2304
query98	213	205	200	200
query99	640	571	538	538
Total cold run time: 247117 ms
Total hot run time: 172057 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10988d2c2213e879fd4cc6777cf530a2313e4313, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.60	0.11	0.11
query5	0.28	0.27	0.25
query6	1.14	0.66	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.56	0.50	0.50
query10	0.56	0.55	0.54
query11	0.14	0.10	0.10
query12	0.16	0.11	0.11
query13	0.60	0.59	0.60
query14	0.96	0.95	0.94
query15	0.80	0.78	0.78
query16	0.41	0.39	0.39
query17	1.08	1.05	1.06
query18	0.23	0.22	0.22
query19	1.84	1.78	1.84
query20	0.02	0.01	0.02
query21	15.44	0.26	0.14
query22	5.25	0.05	0.05
query23	15.94	0.27	0.10
query24	1.72	0.64	0.29
query25	0.09	0.11	0.05
query26	0.14	0.13	0.14
query27	0.08	0.06	0.05
query28	4.01	1.10	0.88
query29	12.53	3.94	3.18
query30	0.28	0.14	0.12
query31	2.81	0.65	0.39
query32	3.23	0.56	0.46
query33	2.98	3.07	3.06
query34	16.16	5.03	4.44
query35	4.43	4.45	4.43
query36	0.66	0.51	0.50
query37	0.11	0.07	0.07
query38	0.06	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.14	0.13
query41	0.10	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 97.2 s
Total hot run time: 26.89 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants