Skip to content

Conversation

@Hastyshell
Copy link
Collaborator

@Hastyshell Hastyshell commented Jan 18, 2024

Proposed changes

#27607

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Hastyshell
Copy link
Collaborator Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 50246 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d62e4942a80278e08f4ddd0a9f53f6b612d3044, data reload: false

------ Round 1 ----------------------------------
q1	17573	4457	4362	4362
q2	2046	150	141	141
q3	10344	1888	1951	1888
q4	10104	1264	1348	1264
q5	8765	4064	4015	4015
q6	226	121	125	121
q7	2020	1624	1605	1605
q8	9361	2744	2746	2744
q9	11119	10658	10819	10658
q10	8664	3555	3532	3532
q11	414	240	243	240
q12	466	291	300	291
q13	18354	3923	4069	3923
q14	353	321	319	319
q15	502	452	458	452
q16	695	603	578	578
q17	1137	982	934	934
q18	7432	6876	6876	6876
q19	1702	1594	1576	1576
q20	543	278	285	278
q21	4497	4127	4068	4068
q22	499	381	389	381
Total cold run time: 116816 ms
Total hot run time: 50246 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4380	4285	4298	4285
q2	320	229	219	219
q3	4167	4149	4134	4134
q4	2771	2753	2754	2753
q5	7297	7220	7188	7188
q6	236	119	119	119
q7	3239	2849	2848	2848
q8	4361	4444	4493	4444
q9	17644	17222	17294	17222
q10	4252	4248	4262	4248
q11	745	658	669	658
q12	1039	848	843	843
q13	7262	3767	3744	3744
q14	438	428	416	416
q15	492	460	458	458
q16	747	715	722	715
q17	3891	3881	3859	3859
q18	8748	8839	8731	8731
q19	1732	1702	1656	1656
q20	2412	2161	2122	2122
q21	8524	8538	8529	8529
q22	1072	978	943	943
Total cold run time: 85769 ms
Total hot run time: 80134 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 238771 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d62e4942a80278e08f4ddd0a9f53f6b612d3044, data reload: false

query1	929	387	376	376
query2	6529	2236	2002	2002
query3	6919	196	196	196
query4	20296	18128	17990	17990
query5	19726	6273	6231	6231
query6	281	215	212	212
query7	4142	292	300	292
query8	244	234	223	223
query9	3125	2699	2613	2613
query10	418	305	294	294
query11	11339	10628	10680	10628
query12	112	73	67	67
query13	5655	631	629	629
query14	17869	13235	13587	13235
query15	365	230	242	230
query16	6449	262	259	259
query17	3201	863	878	863
query18	2260	402	404	402
query19	216	146	142	142
query20	76	76	73	73
query21	194	94	99	94
query22	5182	5219	5183	5183
query23	32568	31686	31717	31686
query24	6908	6521	6424	6424
query25	530	399	411	399
query26	528	158	155	155
query27	1856	290	300	290
query28	6136	2255	2217	2217
query29	2807	2662	2738	2662
query30	237	162	162	162
query31	897	731	725	725
query32	68	50	57	50
query33	393	256	250	250
query34	839	474	480	474
query35	1118	923	984	923
query36	1593	1636	1422	1422
query37	88	60	60	60
query38	3065	2932	2944	2932
query39	1373	1318	1319	1318
query40	207	96	91	91
query41	33	31	31	31
query42	81	77	89	77
query43	649	588	704	588
query44	1115	715	710	710
query45	242	227	232	227
query46	1211	961	970	961
query47	1823	1755	1961	1755
query48	993	677	676	676
query49	617	376	363	363
query50	867	618	616	616
query51	5638	5525	5545	5525
query52	98	83	93	83
query53	436	317	323	317
query54	2420	2235	2245	2235
query55	82	76	75	75
query56	211	204	210	204
query57	1197	1151	1037	1037
query58	218	176	201	176
query59	3369	3276	3398	3276
query60	204	187	209	187
query61	84	83	86	83
query62	799	467	494	467
query63	459	342	336	336
query64	2003	1294	1236	1236
query65	3678	3613	3641	3613
query66	816	359	358	358
query67	18434	16115	15355	15355
query68	8365	651	661	651
query69	544	353	347	347
query70	1958	1800	1852	1800
query71	372	316	295	295
query72	4358	3367	3344	3344
query73	719	332	315	315
query74	6280	5864	5813	5813
query75	4453	3536	3517	3517
query76	4973	1145	1172	1145
query77	784	242	242	242
query78	33855	50016	48626	48626
query79	15768	631	641	631
query80	4977	383	402	383
query81	578	228	234	228
query82	1141	104	92	92
query83	418	137	130	130
query84	254	69	65	65
query85	2279	284	275	275
query86	414	438	409	409
query87	3271	2962	3005	2962
query88	7107	2317	2308	2308
query89	456	270	296	270
query90	2423	210	202	202
query91	157	116	118	116
query92	70	51	51	51
query93	6421	587	553	553
query94	1534	197	202	197
query95	1110	1052	1073	1052
query96	649	331	327	327
query97	6634	6351	6387	6351
query98	186	173	163	163
query99	3808	870	897	870
Total cold run time: 350831 ms
Total hot run time: 238771 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d62e4942a80278e08f4ddd0a9f53f6b612d3044, data reload: false

query1	0.02	0.03	0.02
query2	0.06	0.03	0.02
query3	0.24	0.05	0.04
query4	1.83	0.08	0.06
query5	0.53	0.53	0.52
query6	1.34	0.62	0.62
query7	0.02	0.02	0.01
query8	0.04	0.03	0.02
query9	0.53	0.47	0.49
query10	0.53	0.54	0.54
query11	0.11	0.08	0.08
query12	0.11	0.09	0.08
query13	0.61	0.62	0.62
query14	0.77	0.79	0.78
query15	0.77	0.77	0.76
query16	0.36	0.35	0.37
query17	0.97	1.02	1.00
query18	0.21	0.26	0.26
query19	1.93	1.86	1.88
query20	0.02	0.00	0.01
query21	15.47	0.58	0.55
query22	2.14	2.45	1.33
query23	17.23	1.09	0.92
query24	6.22	3.98	1.06
query25	2.36	0.16	0.13
query26	0.15	0.14	0.14
query27	0.15	0.16	0.14
query28	4.79	0.74	0.72
query29	12.75	2.36	2.31
query30	0.56	0.52	0.53
query31	2.82	0.39	0.38
query32	3.40	0.49	0.50
query33	3.03	3.08	3.07
query34	15.28	4.83	4.78
query35	4.87	4.84	4.85
query36	1.07	1.01	1.00
query37	0.06	0.04	0.04
query38	0.04	0.02	0.03
query39	0.01	0.02	0.02
query40	0.16	0.13	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.67 s
Total hot run time: 30.73 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 9d62e4942a80278e08f4ddd0a9f53f6b612d3044 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@Hastyshell Hastyshell force-pushed the pick-audit-log-status-2.0 branch from 9d62e49 to 5939b49 Compare January 18, 2024 11:50
@Hastyshell
Copy link
Collaborator Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49811 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5939b49ea524538b7bf312c0d0568ebb1e09f71c, data reload: false

------ Round 1 ----------------------------------
q1	17675	4376	4383	4376
q2	2083	165	146	146
q3	10465	1883	1892	1883
q4	10303	1234	1306	1234
q5	8662	3938	3955	3938
q6	224	126	124	124
q7	2042	1582	1628	1582
q8	9326	2713	2712	2712
q9	10756	10558	10454	10454
q10	8637	3546	3520	3520
q11	426	228	244	228
q12	469	300	309	300
q13	18384	3930	3997	3930
q14	343	328	317	317
q15	501	463	443	443
q16	682	590	590	590
q17	1117	978	944	944
q18	7268	6890	6846	6846
q19	1696	1565	1505	1505
q20	506	303	297	297
q21	4465	4054	4117	4054
q22	490	400	388	388
Total cold run time: 116520 ms
Total hot run time: 49811 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4312	4255	4323	4255
q2	315	216	225	216
q3	4154	4154	4163	4154
q4	2745	2753	2733	2733
q5	7176	7170	7162	7162
q6	233	120	116	116
q7	3257	2853	2820	2820
q8	4361	4459	4470	4459
q9	17127	16935	16933	16933
q10	4228	4272	4288	4272
q11	746	678	686	678
q12	1027	857	863	857
q13	6897	3711	3739	3711
q14	441	426	429	426
q15	513	447	467	447
q16	749	714	709	709
q17	3887	4008	3767	3767
q18	8908	8924	8767	8767
q19	1698	1704	1648	1648
q20	2375	2156	2155	2155
q21	8567	8592	8475	8475
q22	1065	971	947	947
Total cold run time: 84781 ms
Total hot run time: 79707 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 240299 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5939b49ea524538b7bf312c0d0568ebb1e09f71c, data reload: false

query1	932	392	398	392
query2	6538	2237	1959	1959
query3	6914	201	195	195
query4	20115	17897	17996	17897
query5	19723	6265	6262	6262
query6	280	215	225	215
query7	4144	292	295	292
query8	269	268	222	222
query9	3089	2672	2604	2604
query10	413	302	287	287
query11	11323	10856	10728	10728
query12	120	76	67	67
query13	5650	644	623	623
query14	17284	13283	13321	13283
query15	369	234	228	228
query16	6449	263	268	263
query17	3187	876	863	863
query18	2271	408	408	408
query19	207	142	145	142
query20	71	73	74	73
query21	186	90	95	90
query22	5396	5141	5095	5095
query23	32455	32106	31795	31795
query24	7015	6481	6468	6468
query25	521	443	416	416
query26	531	162	158	158
query27	1896	298	299	298
query28	6002	2249	2212	2212
query29	2834	2684	2789	2684
query30	238	160	164	160
query31	887	718	738	718
query32	64	56	55	55
query33	382	240	252	240
query34	837	470	475	470
query35	1115	913	917	913
query36	1607	1458	1635	1458
query37	89	57	58	57
query38	3076	2911	2936	2911
query39	1358	1328	1314	1314
query40	203	95	96	95
query41	36	32	32	32
query42	87	89	91	89
query43	550	562	580	562
query44	1130	716	723	716
query45	239	219	231	219
query46	1233	965	975	965
query47	1978	1677	1648	1648
query48	992	691	652	652
query49	609	378	360	360
query50	858	598	618	598
query51	5717	5594	5496	5496
query52	84	86	91	86
query53	439	335	314	314
query54	2455	2243	2229	2229
query55	82	75	74	74
query56	227	200	194	194
query57	1094	1161	1048	1048
query58	210	183	199	183
query59	3485	3170	3163	3163
query60	201	202	189	189
query61	87	82	86	82
query62	855	481	457	457
query63	473	338	334	334
query64	1984	1300	1316	1300
query65	3688	3582	3586	3582
query66	822	372	361	361
query67	16066	16562	16027	16027
query68	6976	652	660	652
query69	546	351	350	350
query70	1855	1881	1662	1662
query71	364	294	303	294
query72	4473	3414	3339	3339
query73	728	323	322	322
query74	6329	5934	5915	5915
query75	4230	3488	3507	3488
query76	4465	1209	1230	1209
query77	546	241	251	241
query78	31719	54090	49602	49602
query79	15231	647	642	642
query80	4364	396	376	376
query81	577	229	230	229
query82	1511	100	93	93
query83	335	131	126	126
query84	259	69	69	69
query85	1698	276	284	276
query86	448	363	389	363
query87	3250	2969	3027	2969
query88	6929	2293	2289	2289
query89	426	316	292	292
query90	2526	208	200	200
query91	156	119	119	119
query92	68	50	47	47
query93	5520	549	573	549
query94	1666	201	205	201
query95	1123	1063	1050	1050
query96	660	330	335	330
query97	6408	6412	6429	6412
query98	187	172	174	172
query99	3665	860	851	851
Total cold run time: 340959 ms
Total hot run time: 240299 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5939b49ea524538b7bf312c0d0568ebb1e09f71c, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.04	0.04
query4	1.81	0.07	0.07
query5	0.53	0.53	0.52
query6	1.23	0.62	0.62
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.55	0.50	0.49
query10	0.54	0.52	0.52
query11	0.13	0.08	0.09
query12	0.12	0.08	0.09
query13	0.62	0.62	0.60
query14	0.77	0.78	0.79
query15	0.79	0.76	0.77
query16	0.35	0.36	0.37
query17	1.03	1.02	1.01
query18	0.22	0.26	0.24
query19	1.87	1.81	1.86
query20	0.01	0.01	0.01
query21	15.47	0.55	0.56
query22	2.01	2.24	1.72
query23	17.40	0.93	0.82
query24	7.70	0.70	0.81
query25	2.27	0.16	0.14
query26	0.15	0.13	0.15
query27	0.14	0.15	0.14
query28	5.54	0.80	0.74
query29	13.01	2.34	2.34
query30	0.56	0.54	0.51
query31	2.82	0.38	0.39
query32	3.37	0.50	0.49
query33	3.08	3.06	3.05
query34	15.24	4.80	4.80
query35	4.88	4.86	4.86
query36	1.06	1.02	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.03
query39	0.01	0.02	0.01
query40	0.16	0.14	0.15
query41	0.06	0.01	0.01
query42	0.03	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 106.04 s
Total hot run time: 30.69 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5939b49ea524538b7bf312c0d0568ebb1e09f71c with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 8ecca10 into apache:branch-2.0 Jan 31, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants