Skip to content

branch-3.0: [fix](cloud-mow) schema change should retry when encouter TXN_CONFILCT in cloud mode #46748 #46955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zhannngchen merged 1 commit into from
Jan 17, 2025
Merged

branch-3.0: [fix](cloud-mow) schema change should retry when encouter TXN_CONFILCT in cloud mode #46748 #46955

zhannngchen merged 1 commit into from
Jan 17, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 14, 2025

Cherry-picked from #46748

@hust-hhb
[fix](cloud-mow) schema change should retry when encouter TXN_CONFILC…
b37b8be 
…T in cloud mode (#46748)

For mow table, shcema change may encouter TXN_CONFILCT beacause of tow
tablet trying to modify delete bitmap lock in the same time, which may
lead to shcema change failed, so should add retry in fe.
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jan 14, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 14, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 14, 2025

TPC-H: Total hot run time: 40927 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b37b8be9bfd68b0a111e6c39e463b3a0bdf5ed3b, data reload: false

------ Round 1 ----------------------------------
q1	17581	7364	7314	7314
q2	2407	170	172	170
q3	11883	1161	1198	1161
q4	10698	801	738	738
q5	8164	2871	2836	2836
q6	236	151	152	151
q7	1005	621	599	599
q8	9442	1939	2061	1939
q9	6790	6435	6413	6413
q10	6938	2254	2302	2254
q11	463	263	265	263
q12	403	214	213	213
q13	17796	3002	2988	2988
q14	231	218	218	218
q15	568	521	524	521
q16	684	616	616	616
q17	967	527	576	527
q18	7245	6685	6748	6685
q19	1378	1122	1046	1046
q20	486	208	200	200
q21	3981	3172	3077	3077
q22	1100	1018	998	998
Total cold run time: 110446 ms
Total hot run time: 40927 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7590	7185	7220	7185
q2	339	233	237	233
q3	3163	2996	2924	2924
q4	1974	1756	1743	1743
q5	5652	5760	5668	5668
q6	215	138	141	138
q7	2198	1778	1768	1768
q8	3342	3511	3486	3486
q9	8790	8889	8797	8797
q10	3611	3580	3558	3558
q11	600	500	502	500
q12	819	602	577	577
q13	5836	3101	3136	3101
q14	301	271	284	271
q15	568	519	517	517
q16	700	678	663	663
q17	1794	1559	1539	1539
q18	7875	7442	7465	7442
q19	1629	1496	1491	1491
q20	2033	1790	1815	1790
q21	5339	5145	5283	5145
q22	1108	1026	998	998
Total cold run time: 65476 ms
Total hot run time: 59534 ms

@doris-robot
Copy link

doris-robot commented Jan 14, 2025

TPC-DS: Total hot run time: 191359 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b37b8be9bfd68b0a111e6c39e463b3a0bdf5ed3b, data reload: false

query1	983	366	370	366
query2	6516	2094	2023	2023
query3	6704	219	218	218
query4	34122	23500	23637	23500
query5	4293	453	429	429
query6	270	164	163	163
query7	4629	304	310	304
query8	287	218	219	218
query9	9663	2665	2661	2661
query10	481	256	258	256
query11	18045	15141	15160	15141
query12	154	105	103	103
query13	1636	418	424	418
query14	10752	6568	7076	6568
query15	254	172	195	172
query16	8088	434	449	434
query17	1627	581	568	568
query18	2095	317	316	316
query19	296	163	154	154
query20	114	109	106	106
query21	210	102	105	102
query22	4533	4435	4361	4361
query23	34636	33836	33795	33795
query24	11247	2849	2816	2816
query25	674	391	392	391
query26	1398	170	168	168
query27	2838	334	338	334
query28	7794	2424	2423	2423
query29	885	427	429	427
query30	336	159	158	158
query31	1000	805	803	803
query32	92	56	57	56
query33	770	292	285	285
query34	970	497	521	497
query35	893	715	700	700
query36	1114	961	956	956
query37	139	76	71	71
query38	4042	3920	3886	3886
query39	1495	1436	1473	1436
query40	290	99	99	99
query41	52	50	50	50
query42	113	100	103	100
query43	539	479	479	479
query44	1277	814	806	806
query45	178	168	165	165
query46	1147	719	702	702
query47	1948	1844	1837	1837
query48	466	374	371	371
query49	1200	396	393	393
query50	810	427	413	413
query51	7260	7149	7024	7024
query52	101	88	93	88
query53	256	184	188	184
query54	1295	467	460	460
query55	77	76	78	76
query56	262	260	261	260
query57	1246	1123	1105	1105
query58	239	205	217	205
query59	3250	2879	2922	2879
query60	302	241	252	241
query61	108	110	121	110
query62	912	700	696	696
query63	219	195	192	192
query64	5402	736	736	736
query65	3294	3220	3203	3203
query66	1454	327	331	327
query67	15813	15495	15481	15481
query68	4375	579	584	579
query69	436	297	269	269
query70	1149	1113	1133	1113
query71	334	261	268	261
query72	6611	4137	4158	4137
query73	753	354	346	346
query74	10096	9034	9086	9034
query75	3363	2646	2632	2632
query76	2765	1048	1107	1048
query77	386	272	271	271
query78	10398	9692	9688	9688
query79	2305	599	618	599
query80	1123	440	427	427
query81	550	238	236	236
query82	898	119	120	119
query83	222	147	146	146
query84	238	81	78	78
query85	1361	307	293	293
query86	451	299	295	295
query87	4369	4270	4251	4251
query88	4165	2362	2353	2353
query89	397	294	295	294
query90	2025	186	186	186
query91	185	153	152	152
query92	68	53	52	52
query93	1628	575	566	566
query94	982	310	288	288
query95	361	260	263	260
query96	625	278	289	278
query97	3318	3150	3188	3150
query98	211	213	196	196
query99	1522	1330	1297	1297
Total cold run time: 303255 ms
Total hot run time: 191359 ms

@doris-robot
Copy link

doris-robot commented Jan 14, 2025

ClickBench: Total hot run time: 33.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b37b8be9bfd68b0a111e6c39e463b3a0bdf5ed3b, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.04
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.50	0.52	0.51
query6	1.13	0.73	0.72
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.50
query10	0.56	0.56	0.54
query11	0.14	0.11	0.11
query12	0.16	0.12	0.11
query13	0.61	0.60	0.60
query14	2.92	2.92	3.06
query15	0.91	0.82	0.83
query16	0.39	0.38	0.38
query17	1.05	1.01	1.03
query18	0.24	0.22	0.22
query19	1.90	1.89	2.08
query20	0.01	0.01	0.01
query21	15.36	0.61	0.58
query22	2.93	2.18	2.05
query23	16.90	1.03	0.84
query24	3.40	1.18	1.15
query25	0.38	0.14	0.19
query26	0.31	0.15	0.13
query27	0.04	0.04	0.05
query28	10.10	1.11	1.08
query29	12.73	3.24	3.22
query30	0.25	0.06	0.06
query31	2.87	0.39	0.40
query32	3.23	0.46	0.47
query33	3.00	2.97	3.01
query34	16.97	4.50	4.55
query35	4.56	4.55	4.53
query36	0.66	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 107.31 s
Total hot run time: 33.54 s

@zhannngchen zhannngchen merged commit 3fb8dbd into branch-3.0 Jan 17, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-46748-branch-3.0 branch January 17, 2025 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen @doris-robot @zhannngchen @dataroaring @hust-hhb