Skip to content

Conversation

@924060929
Copy link
Contributor

What problem does this PR solve?

fix insert failed when concurrent schema change because the lock scope is small, introduced by #45045.

2025-12-22 18:31:27,916 WARN (mysql-nio-pool-20|395) [StmtExecutor.execute():546] Analyze failed. stmt[223, 6daf515a262f4708-959e9e3ef4590797]
org.apache.doris.common.NereidsException: errCode = 2, detailMessage = insert into cols should be corresponding to the query output
	at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:706)
	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:541)
	at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:500)
	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:485)
	at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:311)
	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:198)
	at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:231)
	at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:259)
	at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:403)
	at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = insert into cols should be corresponding to the query output
	... 13 more
Caused by: org.apache.doris.nereids.exceptions.AnalysisException: insert into cols should be corresponding to the query output
	at org.apache.doris.nereids.rules.analysis.BindSink.bindOlapTableSink(BindSink.java:194)
	at org.apache.doris.nereids.pattern.PatternMatcher$1.transform(PatternMatcher.java:96)
	at org.apache.doris.nereids.jobs.rewrite.TopDownVisitorRewriteJob.doRewrite(TopDownVisitorRewriteJob.java:114)
	at org.apache.doris.nereids.jobs.rewrite.TopDownVisitorRewriteJob.rewrite(TopDownVisitorRewriteJob.java:80)
	at org.apache.doris.nereids.jobs.rewrite.TopDownVisitorRewriteJob.execute(TopDownVisitorRewriteJob.java:56)
	at org.apache.doris.nereids.jobs.rewrite.AdaptiveTopDownRewriteJob.execute(AdaptiveTopDownRewriteJob.java:41)
	at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:167)
	at org.apache.doris.nereids.jobs.executor.Analyzer.lambda$execute$0(Analyzer.java:98)
	at org.apache.doris.nereids.util.MoreFieldsThread.keepFunctionSignature(MoreFieldsThread.java:127)
	at org.apache.doris.nereids.jobs.executor.Analyzer.execute(Analyzer.java:97)
	at org.apache.doris.nereids.jobs.executor.Analyzer.analyze(Analyzer.java:92)
	at org.apache.doris.nereids.NereidsPlanner.lambda$analyze$4(NereidsPlanner.java:414)
	at org.apache.doris.nereids.NereidsPlanner.keepOrShowPlanProcess(NereidsPlanner.java:1109)
	at org.apache.doris.nereids.NereidsPlanner.analyze(NereidsPlanner.java:414)
	at org.apache.doris.nereids.trees.plans.commands.insert.FastInsertIntoValuesPlanner.analyze(FastInsertIntoValuesPlanner.java:63)
	at org.apache.doris.nereids.NereidsPlanner.planWithoutLock(NereidsPlanner.java:288)
	at org.apache.doris.nereids.NereidsPlanner.planWithLock(NereidsPlanner.java:263)
	at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:162)
	at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.planInsertExecutor(InsertIntoTableCommand.java:540)
	at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.initPlanOnce(InsertIntoTableCommand.java:353)
	at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.initPlan(InsertIntoTableCommand.java:259)
	at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.initPlan(InsertIntoTableCommand.java:226)
	at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.runInternal(InsertIntoTableCommand.java:569)
	at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.run(InsertIntoTableCommand.java:210)
	at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:678)
	... 12 more

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 23, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31145 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 162fad60efd3eac7441f8942314cb4875f1092fb, data reload: false

------ Round 1 ----------------------------------
q1	17653	4770	4536	4536
q2	2030	302	208	208
q3	10225	1259	722	722
q4	10237	875	303	303
q5	7535	2058	1833	1833
q6	186	167	137	137
q7	864	679	596	596
q8	9262	1330	1066	1066
q9	4817	4544	4472	4472
q10	6767	1662	1269	1269
q11	499	297	272	272
q12	334	375	226	226
q13	17800	3804	3190	3190
q14	230	228	211	211
q15	605	516	519	516
q16	631	633	601	601
q17	642	735	533	533
q18	6471	6385	6648	6385
q19	1240	1046	733	733
q20	456	358	247	247
q21	3050	2211	2055	2055
q22	1120	1090	1034	1034
Total cold run time: 102654 ms
Total hot run time: 31145 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5135	5010	5020	5010
q2	345	443	318	318
q3	2432	2894	2426	2426
q4	1424	1844	1442	1442
q5	4506	4596	4249	4249
q6	219	187	131	131
q7	2049	1969	1808	1808
q8	2557	2426	2416	2416
q9	7134	7050	7069	7050
q10	2702	2658	2271	2271
q11	594	496	474	474
q12	700	795	616	616
q13	3491	3778	3046	3046
q14	269	285	257	257
q15	534	498	485	485
q16	613	675	626	626
q17	1090	1242	1253	1242
q18	7260	7426	7075	7075
q19	831	780	802	780
q20	1876	1966	1857	1857
q21	4631	4271	4178	4178
q22	1053	1057	977	977
Total cold run time: 51445 ms
Total hot run time: 48734 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171575 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 162fad60efd3eac7441f8942314cb4875f1092fb, data reload: false

query5	4432	642	498	498
query6	330	228	202	202
query7	4218	467	264	264
query8	341	252	251	251
query9	8681	2822	2848	2822
query10	495	333	283	283
query11	15180	14997	14744	14744
query12	178	118	118	118
query13	1247	456	362	362
query14	6111	3036	2789	2789
query14_1	2880	2720	2690	2690
query15	199	188	172	172
query16	971	486	379	379
query17	1090	673	573	573
query18	2488	434	329	329
query19	195	178	161	161
query20	126	117	118	117
query21	215	140	122	122
query22	3903	3927	4048	3927
query23	15889	15481	15263	15263
query23_1	15301	15454	15331	15331
query24	7203	1583	1166	1166
query24_1	1170	1159	1167	1159
query25	544	453	415	415
query26	1258	274	155	155
query27	2747	437	281	281
query28	4568	2154	2148	2148
query29	806	555	447	447
query30	313	238	205	205
query31	793	619	544	544
query32	81	73	76	73
query33	535	363	306	306
query34	904	885	538	538
query35	742	771	716	716
query36	877	899	848	848
query37	138	99	89	89
query38	2773	2748	2676	2676
query39	770	740	725	725
query39_1	712	710	699	699
query40	219	152	123	123
query41	72	69	66	66
query42	97	93	93	93
query43	471	438	408	408
query44	1347	739	748	739
query45	188	182	176	176
query46	830	942	610	610
query47	1435	1467	1383	1383
query48	308	316	238	238
query49	599	423	336	336
query50	684	264	199	199
query51	3836	3830	3639	3639
query52	94	89	82	82
query53	218	216	171	171
query54	275	264	248	248
query55	77	79	72	72
query56	288	296	281	281
query57	1016	998	874	874
query58	287	257	255	255
query59	2035	2146	2062	2062
query60	324	322	310	310
query61	148	149	146	146
query62	393	335	329	329
query63	198	158	166	158
query64	4950	1123	846	846
query65	3778	3683	3750	3683
query66	1430	417	314	314
query67	15434	15442	15389	15389
query68	2440	1061	725	725
query69	419	324	270	270
query70	1002	990	881	881
query71	298	281	271	271
query72	5227	3108	3205	3108
query73	606	712	319	319
query74	8690	8680	8530	8530
query75	2310	2329	1868	1868
query76	2260	1058	646	646
query77	350	366	304	304
query78	9648	9658	9140	9140
query79	2363	893	620	620
query80	1732	527	440	440
query81	538	261	228	228
query82	955	147	112	112
query83	316	251	250	250
query84	261	111	101	101
query85	866	465	407	407
query86	410	299	285	285
query87	2896	2852	2753	2753
query88	3539	2600	2570	2570
query89	305	255	245	245
query90	2007	168	164	164
query91	166	158	130	130
query92	73	73	69	69
query93	1220	1010	651	651
query94	648	306	277	277
query95	568	378	297	297
query96	643	511	229	229
query97	2358	2348	2281	2281
query98	224	202	197	197
query99	591	578	492	492
Total cold run time: 247574 ms
Total hot run time: 171575 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 162fad60efd3eac7441f8942314cb4875f1092fb, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.04	0.04
query3	0.26	0.08	0.09
query4	1.61	0.12	0.11
query5	0.27	0.26	0.26
query6	1.16	0.67	0.65
query7	0.04	0.02	0.03
query8	0.05	0.04	0.04
query9	0.57	0.49	0.49
query10	0.55	0.57	0.54
query11	0.14	0.09	0.10
query12	0.15	0.11	0.11
query13	0.61	0.60	0.59
query14	0.95	0.93	0.94
query15	0.79	0.77	0.79
query16	0.38	0.38	0.39
query17	1.04	1.07	1.07
query18	0.22	0.21	0.20
query19	1.88	1.89	1.80
query20	0.04	0.01	0.02
query21	15.43	0.25	0.14
query22	5.12	0.05	0.04
query23	15.89	0.27	0.10
query24	0.91	0.77	0.31
query25	0.13	0.05	0.07
query26	0.14	0.13	0.14
query27	0.07	0.06	0.05
query28	4.78	1.07	0.89
query29	12.52	3.92	3.15
query30	0.28	0.13	0.12
query31	2.81	0.63	0.40
query32	3.24	0.57	0.46
query33	3.00	3.04	3.03
query34	16.01	5.12	4.40
query35	4.50	4.42	4.45
query36	0.66	0.50	0.49
query37	0.10	0.07	0.06
query38	0.07	0.04	0.03
query39	0.05	0.03	0.03
query40	0.17	0.14	0.12
query41	0.09	0.03	0.02
query42	0.05	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 96.92 s
Total hot run time: 26.74 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants