Skip to content

[Fix](case) Fix some docker case #54227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Aug 3, 2025
Merged

[Fix](case) Fix some docker case #54227

dataroaring merged 1 commit into from
Aug 3, 2025

Conversation

@deardeng
Copy link
Contributor

@deardeng deardeng commented Aug 1, 2025

What problem does this PR solve?

Fix

  1. cloud_decommission increased it robustness
  2. test_cloud_full_compaction_multi_segments, test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task fix its not set fe debug point
  3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work, due to [opt](rowset) Remote fetch rowsets to avoid -230 error when capturing rowsets (#52995) #52440,

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 1, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@deardeng
Copy link
Contributor Author

deardeng commented Aug 1, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-H: Total hot run time: 33923 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d370e65175841206d9d1cd82dd8c4362c9f4b5a0, data reload: false

------ Round 1 ----------------------------------
q1	17620	5698	5574	5574
q2	1955	310	197	197
q3	10283	1397	711	711
q4	10224	983	535	535
q5	7478	2303	2295	2295
q6	176	164	133	133
q7	876	742	606	606
q8	9308	1303	1071	1071
q9	6662	4930	5086	4930
q10	6894	2331	1999	1999
q11	476	271	276	271
q12	349	367	224	224
q13	17774	3501	2981	2981
q14	236	229	233	229
q15	546	472	469	469
q16	412	420	372	372
q17	554	826	366	366
q18	7228	7028	6863	6863
q19	3155	960	537	537
q20	324	308	216	216
q21	2740	2991	2341	2341
q22	1002	1077	1003	1003
Total cold run time: 106272 ms
Total hot run time: 33923 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5988	5733	5685	5685
q2	234	310	209	209
q3	2038	2535	2165	2165
q4	1291	1693	1303	1303
q5	4107	4448	4346	4346
q6	220	179	138	138
q7	2018	1835	1847	1835
q8	2515	2479	2396	2396
q9	7532	7255	7539	7255
q10	3181	3397	2906	2906
q11	536	498	501	498
q12	740	756	661	661
q13	3532	3620	3113	3113
q14	293	336	307	307
q15	496	471	465	465
q16	470	505	464	464
q17	1314	1409	1357	1357
q18	7737	7872	7792	7792
q19	12249	855	879	855
q20	2962	1967	2096	1967
q21	14327	4329	4398	4329
q22	1074	1047	1005	1005
Total cold run time: 74854 ms
Total hot run time: 51051 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-DS: Total hot run time: 171628 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d370e65175841206d9d1cd82dd8c4362c9f4b5a0, data reload: false

============================================
query1	981	384	418	384
query2	6516	1846	1668	1668
query3	6758	228	220	220
query4	27085	23371	22971	22971
query5	4418	632	494	494
query6	310	229	213	213
query7	4638	516	294	294
query8	284	248	230	230
query9	8588	3130	3138	3130
query10	495	341	292	292
query11	15928	15764	14789	14789
query12	189	143	133	133
query13	1656	554	442	442
query14	9662	7688	7739	7688
query15	224	212	164	164
query16	7129	640	458	458
query17	968	755	646	646
query18	2011	458	339	339
query19	228	223	197	197
query20	158	150	133	133
query21	228	130	118	118
query22	3971	3853	3825	3825
query23	34349	34113	34119	34113
query24	5480	2438	2413	2413
query25	492	500	435	435
query26	713	286	207	207
query27	2198	492	348	348
query28	3128	2335	2322	2322
query29	651	594	499	499
query30	289	236	205	205
query31	879	797	716	716
query32	88	80	75	75
query33	514	406	382	382
query34	788	814	530	530
query35	831	820	753	753
query36	1021	1047	943	943
query37	126	115	95	95
query38	3964	3964	3960	3960
query39	1453	1376	1363	1363
query40	240	145	139	139
query41	62	60	55	55
query42	139	120	131	120
query43	535	521	489	489
query44	1379	862	871	862
query45	204	188	174	174
query46	938	1042	677	677
query47	1770	1845	1717	1717
query48	398	432	319	319
query49	670	500	423	423
query50	642	685	426	426
query51	4164	4182	4082	4082
query52	126	130	123	123
query53	270	301	215	215
query54	653	646	552	552
query55	91	87	92	87
query56	372	361	351	351
query57	1198	1208	1111	1111
query58	344	347	330	330
query59	2593	2601	2681	2601
query60	400	391	388	388
query61	126	123	121	121
query62	764	730	660	660
query63	252	215	218	215
query64	2348	1079	795	795
query65	4210	4127	4137	4127
query66	1029	437	341	341
query67	query68	17439	874	845	845
query69	1115	283	298	283
query70	1442	1091	1163	1091
query71	736	316	323	316
query72	9238	2284	2166	2166
query73	3575	699	356	356
query74	9080	8939	8573	8573
query75	7511	3124	2708	2708
query76	8892	1230	797	797
query77	1150	426	339	339
query78	9497	11459	9251	9251
query79	15143	657	580	580
query80	1610	634	474	474
query81	585	281	245	245
query82	510	154	121	121
query83	415	297	285	285
query84	293	97	75	75
query85	958	372	326	326
query86	360	307	316	307
query87	4355	4242	4194	4194
query88	5715	2253	2201	2201
query89	501	377	327	327
query90	2603	228	231	228
query91	142	140	112	112
query92	92	72	67	67
query93	6317	980	666	666
query94	1161	397	274	274
query95	427	322	319	319
query96	497	586	279	279
query97	2699	2691	2608	2608
query98	256	227	226	226
query99	1461	1358	1302	1302
Total cold run time: 301081 ms
Total hot run time: 171628 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

ClickBench: Total hot run time: 32.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d370e65175841206d9d1cd82dd8c4362c9f4b5a0, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.24	0.06	0.07
query4	1.67	0.11	0.11
query5	0.44	0.42	0.43
query6	1.18	0.67	0.66
query7	0.03	0.02	0.01
query8	0.05	0.04	0.03
query9	0.56	0.47	0.47
query10	0.54	0.52	0.54
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.64	0.67	0.64
query14	0.94	1.05	1.22
query15	0.90	0.88	0.89
query16	0.40	0.40	0.39
query17	1.09	1.03	1.05
query18	0.21	0.20	0.20
query19	1.89	1.77	1.80
query20	0.01	0.02	0.01
query21	15.38	0.84	0.56
query22	0.79	1.06	0.71
query23	15.00	1.15	0.61
query24	6.34	2.13	0.81
query25	0.58	0.20	0.07
query26	0.63	0.15	0.14
query27	0.07	0.05	0.05
query28	10.09	0.87	0.44
query29	12.65	3.76	3.32
query30	3.06	2.95	2.95
query31	2.82	0.57	0.39
query32	3.27	0.57	0.50
query33	3.04	3.13	3.22
query34	16.14	5.32	4.89
query35	4.93	5.02	4.94
query36	0.69	0.52	0.51
query37	0.09	0.08	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.14	0.13
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 107.2 s
Total hot run time: 32.91 s

@dataroaring dataroaring closed this Aug 3, 2025
@dataroaring
Copy link
Contributor

dataroaring commented Aug 3, 2025

run buildall

@dataroaring dataroaring reopened this Aug 3, 2025
@dataroaring
Copy link
Contributor

dataroaring commented Aug 3, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 3, 2025

TPC-H: Total hot run time: 33363 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a5ca76d3cf0e6389ad22525ccf15ff2fa73a1379, data reload: false

------ Round 1 ----------------------------------
q1	17586	5279	5208	5208
q2	1921	285	183	183
q3	10325	1389	679	679
q4	10244	1056	500	500
q5	7490	2371	2245	2245
q6	177	166	133	133
q7	880	761	630	630
q8	9300	1249	975	975
q9	6852	5016	5211	5016
q10	6895	2357	1960	1960
q11	462	276	267	267
q12	346	358	230	230
q13	17783	3477	2991	2991
q14	229	247	219	219
q15	551	470	473	470
q16	418	439	366	366
q17	566	804	336	336
q18	7422	7047	6964	6964
q19	2538	972	510	510
q20	317	316	216	216
q21	3359	2980	2262	2262
q22	1015	1051	1003	1003
Total cold run time: 106676 ms
Total hot run time: 33363 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5453	5297	5246	5246
q2	228	312	218	218
q3	2062	2577	2203	2203
q4	1284	1676	1324	1324
q5	4144	4400	4545	4400
q6	229	183	134	134
q7	1876	2086	1836	1836
q8	2541	2422	2376	2376
q9	7084	7127	7152	7127
q10	3227	3460	2883	2883
q11	535	538	480	480
q12	686	780	695	695
q13	3644	3735	3377	3377
q14	331	339	321	321
q15	509	481	450	450
q16	463	509	456	456
q17	1166	1535	1367	1367
q18	8126	7714	7541	7541
q19	13738	977	1071	977
q20	2979	2071	1865	1865
q21	14886	4266	4168	4168
q22	1035	1033	997	997
Total cold run time: 76226 ms
Total hot run time: 50441 ms

@doris-robot
Copy link

doris-robot commented Aug 3, 2025

TPC-DS: Total hot run time: 185495 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a5ca76d3cf0e6389ad22525ccf15ff2fa73a1379, data reload: false

query1	1013	377	403	377
query2	6534	1899	1689	1689
query3	6741	223	217	217
query4	26968	23170	23096	23096
query5	4338	600	505	505
query6	316	226	215	215
query7	4632	516	303	303
query8	285	246	227	227
query9	8579	2952	2936	2936
query10	465	348	288	288
query11	15832	14897	14698	14698
query12	181	138	129	129
query13	1647	533	424	424
query14	8658	5872	5901	5872
query15	211	200	165	165
query16	7124	670	462	462
query17	954	765	599	599
query18	1982	451	319	319
query19	212	204	190	190
query20	165	155	136	136
query21	216	125	107	107
query22	4231	4168	4131	4131
query23	34324	34211	34453	34211
query24	5398	2393	2429	2393
query25	552	537	459	459
query26	752	293	170	170
query27	2251	507	350	350
query28	3052	2358	2340	2340
query29	611	600	487	487
query30	288	258	201	201
query31	833	795	711	711
query32	92	77	79	77
query33	453	415	388	388
query34	804	828	516	516
query35	813	826	733	733
query36	997	1035	941	941
query37	129	107	95	95
query38	4039	4115	4017	4017
query39	1429	1381	1369	1369
query40	243	147	135	135
query41	65	56	55	55
query42	147	130	127	127
query43	518	506	483	483
query44	1387	862	857	857
query45	207	214	186	186
query46	948	1056	663	663
query47	1815	1858	1750	1750
query48	386	428	327	327
query49	676	499	421	421
query50	646	687	429	429
query51	4212	4179	4158	4158
query52	137	130	115	115
query53	257	290	207	207
query54	654	627	565	565
query55	89	85	85	85
query56	354	360	344	344
query57	1197	1221	1173	1173
query58	337	337	333	333
query59	2622	2678	2566	2566
query60	431	395	388	388
query61	125	126	122	122
query62	777	762	640	640
query63	248	220	216	216
query64	2334	1085	759	759
query65	4181	4184	4136	4136
query66	733	449	332	332
query67	16051	16344	15485	15485
query68	16687	593	597	593
query69	1006	308	288	288
query70	1308	1081	1134	1081
query71	705	336	319	319
query72	9142	2303	2338	2303
query73	3395	621	352	352
query74	9101	9040	8480	8480
query75	7595	3150	2759	2759
query76	8442	1190	771	771
query77	1133	428	352	352
query78	9484	9864	9389	9389
query79	12643	598	572	572
query80	939	565	492	492
query81	560	267	238	238
query82	409	153	118	118
query83	340	292	272	272
query84	298	107	94	94
query85	871	504	338	338
query86	345	300	296	296
query87	4316	4228	4178	4178
query88	3986	2235	2244	2235
query89	478	358	316	316
query90	2633	222	228	222
query91	142	136	109	109
query92	90	70	66	66
query93	4637	963	661	661
query94	908	397	271	271
query95	396	332	318	318
query96	493	587	288	288
query97	2703	2747	2621	2621
query98	242	234	226	226
query99	1489	1521	1277	1277
Total cold run time: 306794 ms
Total hot run time: 185495 ms

@doris-robot
Copy link

doris-robot commented Aug 3, 2025

ClickBench: Total hot run time: 33.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a5ca76d3cf0e6389ad22525ccf15ff2fa73a1379, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.24	0.06	0.07
query4	1.62	0.10	0.10
query5	0.40	0.40	0.43
query6	1.16	0.68	0.67
query7	0.03	0.02	0.01
query8	0.04	0.03	0.04
query9	0.57	0.47	0.46
query10	0.54	0.54	0.52
query11	0.16	0.10	0.11
query12	0.15	0.11	0.12
query13	0.63	0.66	0.62
query14	0.89	1.13	1.09
query15	0.92	0.88	0.87
query16	0.40	0.40	0.39
query17	1.08	1.04	1.07
query18	0.22	0.20	0.21
query19	1.95	1.81	1.94
query20	0.02	0.01	0.01
query21	15.43	0.84	0.54
query22	0.77	1.24	0.77
query23	14.83	1.18	0.63
query24	6.91	1.51	0.68
query25	0.49	0.29	0.07
query26	0.60	0.17	0.14
query27	0.07	0.06	0.06
query28	9.90	0.83	0.44
query29	12.56	3.86	3.35
query30	3.02	2.97	2.96
query31	2.81	0.56	0.40
query32	3.25	0.57	0.50
query33	2.98	3.24	3.21
query34	16.03	5.35	4.93
query35	4.86	5.00	4.99
query36	0.69	0.53	0.52
query37	0.10	0.07	0.08
query38	0.06	0.05	0.05
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 106.87 s
Total hot run time: 33.1 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 3, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.98% (16153/27860)
Line Coverage 46.75% (145375/310931)
Region Coverage 35.89% (109070/303912)
Branch Coverage 38.58% (48194/124935)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 3, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (5/5) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.35% (22249/27349)
Line Coverage 74.09% (230110/310571)
Region Coverage 61.64% (192086/311624)
Branch Coverage 65.47% (82691/126307)

dataroaring
dataroaring approved these changes Aug 3, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 3, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 3, 2025

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit e5da8ec into apache:master Aug 3, 2025
29 of 32 checks passed
deardeng added a commit to deardeng/incubator-doris that referenced this pull request Aug 4, 2025
@deardeng
[Fix](case) Fix some docker case (apache#54227)
45d988d 
Fix
1. cloud_decommission increased it robustness
2. test_cloud_full_compaction_multi_segments,
test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task
fix its not set fe debug point
3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work,
due to apache#52440,
@deardeng deardeng mentioned this pull request Aug 4, 2025
Merged
deardeng added a commit to deardeng/incubator-doris that referenced this pull request Aug 4, 2025
@deardeng
[Fix](case) Fix some docker case (apache#54227)
ebf3a6a 
Fix
1. cloud_decommission increased it robustness
2. test_cloud_full_compaction_multi_segments,
test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task
fix its not set fe debug point
3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work,
due to apache#52440,
@deardeng deardeng mentioned this pull request Aug 4, 2025
Merged
morrySnow pushed a commit that referenced this pull request Aug 4, 2025
@deardeng
branch-3.1: [Fix](case) Fix some docker case #54227 (#54252)
86f4bf7 
cherry pick from #54227
dataroaring pushed a commit that referenced this pull request Aug 4, 2025
@deardeng
branch-3.0: [Fix](case) Fix some docker case #54227 (#54251)
d6ac40a 
cherry pick from #54227
BiteTheDDDDt pushed a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Sep 1, 2025
@deardeng
[Fix](case) Fix some docker case (apache#54227) (apache#4263)
47b10db 
Fix
1. cloud_decommission increased it robustness
2. test_cloud_full_compaction_multi_segments,
test_cloud_mow_retry_txn_interleave, test_cloud_concurrent_calc_dbm_task
fix its not set fe debug point
3. fix CloudTablet.capture_rs_readers.return.e-230 debug point not work,
due to apache#52440,

pick from apache#54227
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner

@w41ter w41ter Awaiting requested review from w41ter w41ter is a code owner

+1 more reviewer

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.7-merged dev/3.1.0-merged p0_test reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@deardeng @hello-stephen @doris-robot @dataroaring @Yukang-Lian @morrySnow