Skip to content

Conversation

@bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Jul 1, 2025

What problem does this PR solve?

Currently, DeleteBitmap's wrong move assignment operator can cause correctness problem on mow table if SC's alter process produce delete bitmaps when calculating delete bitmaps for incremental rowsets after the modification in CloudSchemaChangeJob::_process_delete_bitmap of #52440 because origin_dbm and delete_bitmap refer to the same DeleteBitmap object.
image

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jul 1, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch from 3f35215 to f2b5fd6 Compare July 1, 2025 07:14
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jul 1, 2025

run buildall

@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch 3 times, most recently from ad7580a to 44b81ca Compare July 1, 2025 07:42
@bobhan1 bobhan1 changed the title [Fix](mow) Fix DeleteBitmap's assignment operator [Fix](mow) Fix DeleteBitmap's assignment operator and constructor Jul 1, 2025
@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch from 44b81ca to 03cb0db Compare July 1, 2025 07:49
@doris-robot
Copy link

TPC-H: Total hot run time: 33290 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f2b5fd603f8e951c019ff40db680f4b4a717caaa, data reload: false

------ Round 1 ----------------------------------
q1	17587	5152	4990	4990
q2	1957	277	175	175
q3	10322	1241	685	685
q4	10214	1002	496	496
q5	7521	2314	2313	2313
q6	179	156	126	126
q7	883	740	595	595
q8	9313	1260	1038	1038
q9	6709	5055	5103	5055
q10	6875	2354	1943	1943
q11	459	279	265	265
q12	338	365	204	204
q13	17761	3626	3015	3015
q14	215	228	206	206
q15	557	491	481	481
q16	408	412	372	372
q17	574	833	351	351
q18	7486	7264	6997	6997
q19	1215	939	532	532
q20	334	324	220	220
q21	3617	2513	2269	2269
q22	1051	1006	962	962
Total cold run time: 105575 ms
Total hot run time: 33290 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5058	5114	5018	5018
q2	236	314	218	218
q3	2145	2672	2291	2291
q4	1352	1724	1331	1331
q5	4149	4118	4375	4118
q6	207	161	126	126
q7	2006	1934	1873	1873
q8	2584	2452	2403	2403
q9	7326	7342	7192	7192
q10	3150	3388	2877	2877
q11	588	507	485	485
q12	654	798	652	652
q13	3520	3662	3054	3054
q14	257	285	249	249
q15	509	464	463	463
q16	414	465	412	412
q17	1072	1499	1296	1296
q18	7243	7120	7171	7120
q19	742	722	810	722
q20	1904	1958	1822	1822
q21	4585	4217	4192	4192
q22	1046	1022	996	996
Total cold run time: 50747 ms
Total hot run time: 48910 ms

@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch 2 times, most recently from 98cb439 to 7e54575 Compare July 1, 2025 08:01
@doris-robot
Copy link

TPC-DS: Total hot run time: 185012 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f2b5fd603f8e951c019ff40db680f4b4a717caaa, data reload: false

query1	1006	390	392	390
query2	6530	1724	1660	1660
query3	6765	210	207	207
query4	26530	23708	23554	23554
query5	4357	581	436	436
query6	305	222	193	193
query7	4619	509	285	285
query8	258	198	195	195
query9	8593	2585	2602	2585
query10	461	308	277	277
query11	15457	15151	14804	14804
query12	153	105	110	105
query13	1645	521	384	384
query14	8969	5674	5671	5671
query15	195	208	169	169
query16	7538	634	466	466
query17	1205	744	594	594
query18	2006	419	316	316
query19	224	198	161	161
query20	117	117	111	111
query21	217	130	109	109
query22	4305	4162	3936	3936
query23	34043	32992	32973	32973
query24	8472	2361	2376	2361
query25	575	521	387	387
query26	1223	266	144	144
query27	2751	537	338	338
query28	4301	2084	2065	2065
query29	737	555	421	421
query30	282	223	188	188
query31	909	830	764	764
query32	72	64	60	60
query33	563	374	355	355
query34	791	875	512	512
query35	780	799	734	734
query36	963	972	935	935
query37	119	99	76	76
query38	4095	4091	4027	4027
query39	1471	1420	1426	1420
query40	211	111	107	107
query41	57	56	54	54
query42	121	104	105	104
query43	483	519	481	481
query44	1308	793	798	793
query45	176	170	157	157
query46	863	1074	624	624
query47	1762	1798	1711	1711
query48	372	414	302	302
query49	726	461	377	377
query50	655	717	414	414
query51	4102	4225	4166	4166
query52	110	100	98	98
query53	222	252	183	183
query54	569	578	503	503
query55	83	76	78	76
query56	300	293	287	287
query57	1201	1203	1124	1124
query58	265	256	257	256
query59	2620	2729	2649	2649
query60	308	327	289	289
query61	121	118	115	115
query62	809	717	636	636
query63	222	186	191	186
query64	4277	1036	635	635
query65	4217	4132	4163	4132
query66	1096	411	296	296
query67	15826	15677	15485	15485
query68	8078	938	509	509
query69	476	303	265	265
query70	1182	1143	1046	1046
query71	458	316	290	290
query72	5756	4767	4793	4767
query73	730	628	342	342
query74	9157	9004	8987	8987
query75	3776	3213	2683	2683
query76	3618	1241	717	717
query77	784	372	291	291
query78	10125	10203	9451	9451
query79	2827	842	585	585
query80	655	514	444	444
query81	490	253	222	222
query82	477	132	99	99
query83	275	250	244	244
query84	294	104	81	81
query85	859	376	327	327
query86	388	303	291	291
query87	4534	4432	4242	4242
query88	3452	2280	2244	2244
query89	412	321	278	278
query90	1842	212	204	204
query91	138	137	109	109
query92	73	69	59	59
query93	1886	955	578	578
query94	665	429	311	311
query95	375	297	284	284
query96	495	624	280	280
query97	2746	2718	2695	2695
query98	238	205	201	201
query99	1471	1419	1278	1278
Total cold run time: 275416 ms
Total hot run time: 185012 ms

@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch from 7e54575 to 60db2a4 Compare July 1, 2025 08:06
@doris-robot
Copy link

ClickBench: Total hot run time: 29.6 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f2b5fd603f8e951c019ff40db680f4b4a717caaa, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.44	0.45	0.40
query6	1.18	0.65	0.66
query7	0.02	0.01	0.01
query8	0.05	0.04	0.03
query9	0.61	0.51	0.50
query10	0.58	0.57	0.58
query11	0.15	0.12	0.11
query12	0.16	0.11	0.12
query13	0.62	0.61	0.60
query14	0.78	0.81	0.82
query15	0.90	0.87	0.85
query16	0.37	0.38	0.39
query17	1.07	1.06	1.03
query18	0.22	0.21	0.20
query19	1.89	1.80	1.81
query20	0.01	0.01	0.02
query21	15.41	0.90	0.54
query22	0.75	1.14	0.87
query23	14.72	1.42	0.61
query24	6.80	2.08	0.82
query25	0.48	0.28	0.07
query26	0.58	0.17	0.15
query27	0.07	0.06	0.05
query28	9.53	0.94	0.44
query29	12.54	4.00	3.31
query30	0.27	0.11	0.07
query31	2.86	0.59	0.38
query32	3.24	0.55	0.47
query33	3.02	3.08	3.11
query34	16.08	5.39	4.82
query35	4.83	4.81	4.82
query36	0.67	0.50	0.49
query37	0.09	0.07	0.06
query38	0.06	0.04	0.03
query39	0.04	0.02	0.03
query40	0.18	0.14	0.13
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.4 s
Total hot run time: 29.6 s

@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch from 60db2a4 to 54c8ec9 Compare July 1, 2025 08:11
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jul 1, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 35579 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 54c8ec9a65544e2f238f14ecd7df5744300ed564, data reload: false

------ Round 1 ----------------------------------
q1	17592	5435	5197	5197
q2	1947	298	199	199
q3	10511	1341	762	762
q4	10321	1046	545	545
q5	8815	2393	2414	2393
q6	222	172	132	132
q7	928	752	644	644
q8	9339	1344	1125	1125
q9	7476	5173	5122	5122
q10	6939	2381	2001	2001
q11	508	298	296	296
q12	377	366	249	249
q13	17783	3813	3198	3198
q14	239	234	213	213
q15	575	483	500	483
q16	460	442	400	400
q17	672	888	392	392
q18	7729	7178	7178	7178
q19	1352	954	577	577
q20	375	369	243	243
q21	4275	3800	3235	3235
q22	1128	1065	995	995
Total cold run time: 109563 ms
Total hot run time: 35579 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5219	5182	5183	5182
q2	265	334	222	222
q3	2208	2665	2400	2400
q4	1415	1869	1547	1547
q5	4597	4478	4462	4462
q6	225	167	156	156
q7	2006	1905	1761	1761
q8	2667	2711	2546	2546
q9	7518	7369	7506	7369
q10	3098	3287	2744	2744
q11	575	513	485	485
q12	665	730	566	566
q13	3435	3731	3174	3174
q14	263	283	274	274
q15	508	472	458	458
q16	436	468	445	445
q17	1170	1560	1348	1348
q18	7473	7083	7165	7083
q19	840	929	1087	929
q20	1956	1975	1884	1884
q21	4908	4459	4472	4459
q22	1122	1051	983	983
Total cold run time: 52569 ms
Total hot run time: 50477 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189317 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 54c8ec9a65544e2f238f14ecd7df5744300ed564, data reload: false

query1	1008	396	393	393
query2	6548	1811	1847	1811
query3	6751	231	225	225
query4	26450	23773	23009	23009
query5	4364	622	481	481
query6	325	228	212	212
query7	4630	526	319	319
query8	291	248	229	229
query9	8636	2960	2955	2955
query10	500	333	301	301
query11	15582	15191	15012	15012
query12	174	114	106	106
query13	1661	586	447	447
query14	9618	6094	6043	6043
query15	212	199	181	181
query16	7668	673	510	510
query17	1207	750	634	634
query18	2037	430	362	362
query19	206	202	178	178
query20	127	122	124	122
query21	216	131	109	109
query22	4259	4319	4085	4085
query23	34122	33234	33165	33165
query24	8481	2492	2478	2478
query25	575	478	407	407
query26	1215	279	159	159
query27	2693	561	373	373
query28	4294	2381	2345	2345
query29	749	586	461	461
query30	292	228	199	199
query31	960	857	763	763
query32	74	71	66	66
query33	601	381	329	329
query34	867	910	583	583
query35	812	820	737	737
query36	1011	1028	962	962
query37	131	109	85	85
query38	4209	4389	4142	4142
query39	1525	1420	1433	1420
query40	220	122	113	113
query41	60	58	56	56
query42	143	120	116	116
query43	540	558	516	516
query44	1453	904	895	895
query45	178	174	164	164
query46	911	1071	666	666
query47	1754	1816	1744	1744
query48	426	455	336	336
query49	763	516	440	440
query50	695	748	451	451
query51	4253	4226	4222	4222
query52	123	117	111	111
query53	252	285	204	204
query54	649	658	551	551
query55	88	89	88	88
query56	351	307	337	307
query57	1173	1211	1109	1109
query58	284	268	283	268
query59	2730	2786	2687	2687
query60	364	349	337	337
query61	137	155	122	122
query62	821	726	657	657
query63	246	210	206	206
query64	4267	1060	702	702
query65	4292	4232	4249	4232
query66	1062	429	326	326
query67	16269	16068	15725	15725
query68	7890	986	587	587
query69	482	347	291	291
query70	1280	1209	1205	1205
query71	480	355	346	346
query72	5316	4810	4765	4765
query73	688	638	386	386
query74	9325	9289	8928	8928
query75	3799	3235	2843	2843
query76	3591	1201	774	774
query77	791	430	311	311
query78	10066	10222	9306	9306
query79	1992	907	643	643
query80	632	537	492	492
query81	471	264	235	235
query82	433	134	105	105
query83	283	288	264	264
query84	281	117	92	92
query85	805	384	426	384
query86	352	314	325	314
query87	4505	4578	4403	4403
query88	3237	2482	2466	2466
query89	403	330	303	303
query90	1935	223	217	217
query91	152	150	129	129
query92	75	66	57	57
query93	1147	1037	648	648
query94	691	415	321	321
query95	403	317	307	307
query96	565	630	316	316
query97	2748	2773	2639	2639
query98	237	220	217	217
query99	1456	1419	1274	1274
Total cold run time: 276096 ms
Total hot run time: 189317 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 54c8ec9a65544e2f238f14ecd7df5744300ed564, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.05	0.04
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.44	0.44	0.42
query6	1.18	0.67	0.67
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.61	0.54	0.52
query10	0.60	0.58	0.58
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.64	0.62	0.62
query14	0.83	0.84	0.82
query15	0.93	0.89	0.91
query16	0.39	0.41	0.41
query17	1.13	1.06	1.10
query18	0.25	0.23	0.23
query19	2.04	1.87	1.95
query20	0.02	0.01	0.02
query21	15.36	0.96	0.58
query22	0.75	1.26	0.68
query23	14.86	1.45	0.68
query24	7.52	1.10	0.59
query25	0.47	0.19	0.07
query26	0.60	0.18	0.16
query27	0.08	0.06	0.05
query28	9.18	1.00	0.47
query29	12.71	4.26	3.48
query30	0.25	0.09	0.08
query31	2.82	0.63	0.41
query32	3.24	0.57	0.47
query33	3.21	3.16	3.17
query34	16.07	5.40	4.81
query35	4.88	4.86	4.85
query36	0.69	0.54	0.49
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.02	0.02
query42	0.03	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.65 s
Total hot run time: 30.05 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 9.52% (2/21) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.07% (15358/26911)
Line Coverage 46.13% (139432/302249)
Region Coverage 45.43% (70663/155533)
Branch Coverage 40.21% (37315/92802)

zhannngchen
zhannngchen previously approved these changes Jul 1, 2025
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jul 2, 2025

run cloud_p0

@doris-robot
Copy link

TPC-DS: Total hot run time: 185161 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3501a83e838abb7e28b8847d8a49516a7f7f4fd3, data reload: false

query1	996	391	388	388
query2	6537	1720	1720	1720
query3	6745	207	209	207
query4	26162	23378	23410	23378
query5	4363	582	437	437
query6	323	207	206	206
query7	4635	485	285	285
query8	280	220	221	220
query9	8607	2613	2598	2598
query10	452	319	267	267
query11	15704	15484	14736	14736
query12	149	104	100	100
query13	1650	520	413	413
query14	9060	5554	5646	5554
query15	218	193	177	177
query16	7248	642	480	480
query17	1208	734	595	595
query18	1992	428	319	319
query19	194	193	167	167
query20	127	122	115	115
query21	216	126	116	116
query22	4093	4127	3998	3998
query23	33770	33096	32978	32978
query24	8459	2415	2369	2369
query25	578	518	421	421
query26	1228	275	157	157
query27	2748	523	338	338
query28	4297	2161	2125	2125
query29	809	583	505	505
query30	275	220	192	192
query31	909	849	742	742
query32	71	62	61	61
query33	538	346	302	302
query34	797	833	520	520
query35	788	792	727	727
query36	974	981	903	903
query37	112	105	73	73
query38	4146	4193	4006	4006
query39	1473	1429	1418	1418
query40	215	115	100	100
query41	55	53	51	51
query42	123	109	111	109
query43	496	505	494	494
query44	1323	813	825	813
query45	180	164	164	164
query46	829	1020	629	629
query47	1754	1802	1705	1705
query48	396	409	323	323
query49	736	515	400	400
query50	649	684	414	414
query51	4109	4321	4152	4152
query52	105	105	97	97
query53	226	250	182	182
query54	581	570	501	501
query55	82	81	85	81
query56	298	293	290	290
query57	1178	1181	1123	1123
query58	275	255	256	255
query59	2633	2760	2589	2589
query60	325	318	325	318
query61	125	124	122	122
query62	779	736	650	650
query63	221	185	189	185
query64	4327	990	704	704
query65	4250	4205	4139	4139
query66	1153	441	311	311
query67	15846	15421	15339	15339
query68	8053	899	528	528
query69	463	310	268	268
query70	1164	1111	1106	1106
query71	454	324	289	289
query72	5608	4802	4891	4802
query73	724	680	358	358
query74	9242	9108	8979	8979
query75	3795	3176	2676	2676
query76	3665	1184	728	728
query77	793	410	309	309
query78	10155	10189	9406	9406
query79	2498	818	588	588
query80	621	514	462	462
query81	480	263	218	218
query82	487	125	103	103
query83	294	259	239	239
query84	293	104	82	82
query85	794	357	310	310
query86	384	290	285	285
query87	4539	4511	4302	4302
query88	3486	2297	2272	2272
query89	392	319	287	287
query90	1917	211	209	209
query91	141	136	108	108
query92	79	59	57	57
query93	1827	960	595	595
query94	656	422	316	316
query95	367	298	292	292
query96	489	578	282	282
query97	2722	2744	2669	2669
query98	264	202	222	202
query99	1450	1373	1281	1281
Total cold run time: 274496 ms
Total hot run time: 185161 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3501a83e838abb7e28b8847d8a49516a7f7f4fd3, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.08	0.08
query4	1.60	0.10	0.10
query5	0.43	0.44	0.41
query6	1.14	0.66	0.66
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.61	0.51	0.54
query10	0.58	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.63	0.63	0.61
query14	0.81	0.85	0.79
query15	0.90	0.89	0.85
query16	0.41	0.39	0.39
query17	1.03	1.06	1.04
query18	0.22	0.21	0.21
query19	1.97	1.80	1.84
query20	0.01	0.01	0.01
query21	15.39	0.88	0.54
query22	0.75	1.21	0.78
query23	14.79	1.37	0.61
query24	7.73	1.54	0.76
query25	0.50	0.29	0.20
query26	0.62	0.16	0.15
query27	0.08	0.05	0.05
query28	9.70	0.87	0.46
query29	12.57	3.90	3.24
query30	0.25	0.08	0.06
query31	2.86	0.62	0.39
query32	3.23	0.55	0.48
query33	3.10	3.08	3.09
query34	16.04	5.34	4.78
query35	4.80	4.85	4.86
query36	0.72	0.50	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.15	0.15
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.7 s
Total hot run time: 29.67 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 8.70% (2/23) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.30% (15491/27034)
Line Coverage 46.28% (140777/304192)
Region Coverage 45.54% (71219/156375)
Branch Coverage 40.25% (37518/93206)

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jul 8, 2025

run cloud_p0

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jul 8, 2025

run external

@bobhan1 bobhan1 force-pushed the fix-delete-bitmap branch from 925f1a8 to b250027 Compare July 8, 2025 05:05
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jul 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33149 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b2500277c508a71cbf3940d5c7e32973d5714d4f, data reload: false

------ Round 1 ----------------------------------
q1	17570	5403	5032	5032
q2	1945	310	193	193
q3	10394	1262	710	710
q4	10276	1029	507	507
q5	8118	2408	2338	2338
q6	177	161	125	125
q7	909	749	595	595
q8	9316	1302	1034	1034
q9	6711	5030	5092	5030
q10	6867	2383	1971	1971
q11	472	291	259	259
q12	347	350	209	209
q13	17761	3669	3084	3084
q14	237	232	215	215
q15	576	482	475	475
q16	425	423	381	381
q17	581	866	352	352
q18	7805	7271	7240	7240
q19	1242	965	547	547
q20	318	352	213	213
q21	3646	3129	2338	2338
q22	364	322	301	301
Total cold run time: 106057 ms
Total hot run time: 33149 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5072	5062	5070	5062
q2	234	321	220	220
q3	2184	2689	2318	2318
q4	1388	1727	1350	1350
q5	4240	4157	4552	4157
q6	211	170	120	120
q7	2116	1962	1829	1829
q8	2866	2589	2510	2510
q9	7289	7270	7356	7270
q10	3150	3322	2825	2825
q11	565	558	493	493
q12	939	766	634	634
q13	3490	4094	3388	3388
q14	290	314	305	305
q15	530	464	468	464
q16	467	486	444	444
q17	1194	1563	1391	1391
q18	8136	7783	7670	7670
q19	806	779	918	779
q20	1924	1996	1829	1829
q21	4810	4432	4243	4243
q22	621	586	544	544
Total cold run time: 52522 ms
Total hot run time: 49845 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185763 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b2500277c508a71cbf3940d5c7e32973d5714d4f, data reload: false

query1	1029	389	410	389
query2	6519	1725	1741	1725
query3	6737	213	214	213
query4	26459	23723	23006	23006
query5	4339	577	439	439
query6	300	214	201	201
query7	4637	494	292	292
query8	288	225	223	223
query9	8641	2598	2601	2598
query10	455	316	264	264
query11	15802	14953	15362	14953
query12	170	106	99	99
query13	1656	541	381	381
query14	9357	6095	5822	5822
query15	210	189	168	168
query16	7511	449	276	276
query17	1296	704	572	572
query18	2006	390	307	307
query19	192	186	158	158
query20	122	116	109	109
query21	212	126	104	104
query22	3985	4025	3930	3930
query23	33985	33081	33047	33047
query24	8426	2399	2371	2371
query25	536	471	388	388
query26	1273	267	143	143
query27	2763	513	342	342
query28	4299	2133	2101	2101
query29	710	544	431	431
query30	290	224	199	199
query31	926	833	752	752
query32	70	61	59	59
query33	544	347	281	281
query34	793	846	516	516
query35	589	647	545	545
query36	926	969	874	874
query37	108	95	80	80
query38	4224	4183	4127	4127
query39	1515	1428	1402	1402
query40	216	119	107	107
query41	56	54	58	54
query42	130	117	113	113
query43	555	519	489	489
query44	1338	829	817	817
query45	170	169	165	165
query46	836	1016	637	637
query47	1741	1810	1732	1732
query48	385	411	319	319
query49	771	485	365	365
query50	623	697	406	406
query51	4107	4251	4088	4088
query52	110	105	105	105
query53	224	253	194	194
query54	591	575	496	496
query55	84	77	78	77
query56	283	303	285	285
query57	1210	1174	1134	1134
query58	267	244	267	244
query59	2616	2691	2565	2565
query60	320	310	300	300
query61	138	119	120	119
query62	866	687	644	644
query63	224	181	189	181
query64	4344	1167	820	820
query65	4266	4157	4182	4157
query66	1083	409	337	337
query67	16018	15565	15625	15565
query68	5036	896	543	543
query69	508	309	262	262
query70	1182	1138	1125	1125
query71	413	310	291	291
query72	5584	4749	4748	4748
query73	649	600	347	347
query74	8873	8970	8873	8873
query75	3217	3203	2711	2711
query76	3215	1162	722	722
query77	460	378	294	294
query78	11082	10930	10250	10250
query79	2086	837	582	582
query80	1607	508	489	489
query81	571	268	229	229
query82	405	124	99	99
query83	355	258	231	231
query84	263	109	85	85
query85	881	432	416	416
query86	460	303	295	295
query87	4410	4504	4302	4302
query88	2892	2375	2272	2272
query89	384	321	281	281
query90	1726	211	210	210
query91	209	138	113	113
query92	63	61	55	55
query93	1304	976	596	596
query94	684	312	201	201
query95	373	290	289	289
query96	489	563	279	279
query97	2662	2763	2665	2665
query98	230	218	207	207
query99	1345	1414	1299	1299
Total cold run time: 271090 ms
Total hot run time: 185763 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b2500277c508a71cbf3940d5c7e32973d5714d4f, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.25	0.07	0.07
query4	1.63	0.11	0.11
query5	0.43	0.44	0.41
query6	1.18	0.65	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.03
query9	0.59	0.50	0.52
query10	0.57	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.62	0.62	0.60
query14	0.80	0.84	0.80
query15	0.89	0.86	0.86
query16	0.39	0.39	0.38
query17	1.04	1.06	1.05
query18	0.24	0.21	0.21
query19	1.94	1.89	1.88
query20	0.02	0.01	0.01
query21	15.40	0.91	0.53
query22	0.77	1.24	0.60
query23	14.92	1.34	0.59
query24	7.50	0.79	0.96
query25	0.47	0.15	0.05
query26	0.58	0.18	0.13
query27	0.06	0.05	0.05
query28	9.51	0.89	0.44
query29	12.54	3.88	3.25
query30	0.25	0.09	0.06
query31	2.84	0.58	0.39
query32	3.23	0.56	0.48
query33	3.02	3.13	3.12
query34	16.05	5.36	4.74
query35	4.74	4.87	4.82
query36	0.66	0.50	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.15	0.14
query41	0.08	0.02	0.02
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 104.12 s
Total hot run time: 29.31 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 8.70% (2/23) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.32% (15499/27038)
Line Coverage 46.29% (140855/304257)
Region Coverage 45.55% (71225/156363)
Branch Coverage 40.27% (37538/93206)

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit e3b5bf4 into apache:master Jul 9, 2025
23 of 25 checks passed
github-actions bot pushed a commit that referenced this pull request Jul 9, 2025
…52582)

### What problem does this PR solve?

Currently, `DeleteBitmap`'s wrong move assignment operator can cause
correctness problem on mow table if SC's alter process produce delete
bitmaps when calculating delete bitmaps for incremental rowsets after
the modification in `CloudSchemaChangeJob::_process_delete_bitmap` of
#52440 because `origin_dbm` and
`delete_bitmap` refer to the same `DeleteBitmap` object.

![image](https://github.com/user-attachments/assets/254d975a-594a-4b47-b028-64e47a05cb72)
github-actions bot pushed a commit that referenced this pull request Jul 9, 2025
…52582)

### What problem does this PR solve?

Currently, `DeleteBitmap`'s wrong move assignment operator can cause
correctness problem on mow table if SC's alter process produce delete
bitmaps when calculating delete bitmaps for incremental rowsets after
the modification in `CloudSchemaChangeJob::_process_delete_bitmap` of
#52440 because `origin_dbm` and
`delete_bitmap` refer to the same `DeleteBitmap` object.

![image](https://github.com/user-attachments/assets/254d975a-594a-4b47-b028-64e47a05cb72)
morrySnow pushed a commit that referenced this pull request Jul 9, 2025
…structor #52582 (#52975)

Cherry-picked from #52582

Co-authored-by: bobhan1 <baohan@selectdb.com>
dataroaring pushed a commit that referenced this pull request Jul 9, 2025
…onstructor #52582 (#52974)

Cherry-picked from #52582

Co-authored-by: bobhan1 <baohan@selectdb.com>
Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Jul 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.7-merged dev/3.1.0-merged p0_w reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants