Skip to content

[Refactor](Variant) make many insterfaces exception safe #37640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eldenmoon merged 2 commits into from
Jul 12, 2024
Merged

[Refactor](Variant) make many insterfaces exception safe #37640

eldenmoon merged 2 commits into from
Jul 12, 2024

Conversation

@eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jul 11, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon
Copy link
Member Author

eldenmoon commented Jul 11, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-H: Total hot run time: 40438 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 160e343309e35fb2d12b0c1855c2e3d48c6cf4cd, data reload: false

------ Round 1 ----------------------------------
q1	17709	5003	4719	4719
q2	2020	192	182	182
q3	10521	1191	1064	1064
q4	10219	743	701	701
q5	7543	2701	2645	2645
q6	220	138	136	136
q7	964	598	599	598
q8	9212	2078	2102	2078
q9	8727	6601	6564	6564
q10	8859	3774	3756	3756
q11	455	233	231	231
q12	510	249	233	233
q13	17772	2977	2967	2967
q14	272	250	236	236
q15	537	489	485	485
q16	505	384	378	378
q17	972	605	629	605
q18	8241	7576	7400	7400
q19	5801	1517	1598	1517
q20	677	325	323	323
q21	5261	3285	3900	3285
q22	397	337	335	335
Total cold run time: 117394 ms
Total hot run time: 40438 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4442	4303	4273	4273
q2	384	277	256	256
q3	3059	2893	2913	2893
q4	2016	1742	1718	1718
q5	5624	5549	5455	5455
q6	224	134	132	132
q7	2219	1861	1889	1861
q8	3273	3439	3440	3439
q9	8770	8887	8813	8813
q10	4220	3762	3860	3762
q11	587	526	501	501
q12	835	642	661	642
q13	16127	3171	3176	3171
q14	326	290	283	283
q15	540	470	505	470
q16	489	451	434	434
q17	1827	1561	1496	1496
q18	8260	7954	7736	7736
q19	1792	1565	1612	1565
q20	2072	1864	1873	1864
q21	9509	4799	4890	4799
q22	587	543	574	543
Total cold run time: 77182 ms
Total hot run time: 56106 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-DS: Total hot run time: 174085 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 160e343309e35fb2d12b0c1855c2e3d48c6cf4cd, data reload: false

query1	927	385	378	378
query2	6453	2585	2414	2414
query3	6640	212	220	212
query4	28415	17577	17420	17420
query5	3672	479	474	474
query6	245	192	168	168
query7	4579	296	296	296
query8	326	321	299	299
query9	8444	2432	2400	2400
query10	443	280	289	280
query11	10816	10003	10178	10003
query12	121	88	90	88
query13	1641	378	394	378
query14	10375	7812	6875	6875
query15	278	184	188	184
query16	7912	315	317	315
query17	1805	535	548	535
query18	1977	275	271	271
query19	198	149	160	149
query20	92	84	82	82
query21	212	126	127	126
query22	4233	4049	3986	3986
query23	33935	33506	33503	33503
query24	10942	2936	2808	2808
query25	621	391	381	381
query26	1117	162	153	153
query27	2516	287	275	275
query28	7410	2170	2144	2144
query29	916	658	636	636
query30	253	150	153	150
query31	974	780	762	762
query32	99	55	55	55
query33	760	301	310	301
query34	1002	499	515	499
query35	694	601	583	583
query36	1147	1003	989	989
query37	144	92	92	92
query38	2957	2888	2820	2820
query39	920	820	814	814
query40	209	119	122	119
query41	51	52	50	50
query42	115	107	105	105
query43	590	588	578	578
query44	1225	733	733	733
query45	198	168	159	159
query46	1105	704	754	704
query47	1877	1756	1777	1756
query48	373	304	292	292
query49	847	407	413	407
query50	789	394	391	391
query51	6919	6828	6805	6805
query52	98	105	93	93
query53	363	300	294	294
query54	889	441	447	441
query55	78	75	77	75
query56	299	263	259	259
query57	1129	1030	1044	1030
query58	263	244	239	239
query59	3637	3158	3213	3158
query60	315	279	268	268
query61	95	96	93	93
query62	826	626	652	626
query63	330	300	297	297
query64	9509	2287	1661	1661
query65	3161	3110	3085	3085
query66	751	329	336	329
query67	15270	14950	14895	14895
query68	4598	557	562	557
query69	597	382	339	339
query70	1179	1120	1171	1120
query71	436	280	279	279
query72	7207	5589	5853	5589
query73	743	332	327	327
query74	5905	5540	5518	5518
query75	3754	2694	2727	2694
query76	2729	975	917	917
query77	595	316	337	316
query78	9960	9126	9025	9025
query79	2569	525	539	525
query80	1455	473	472	472
query81	585	223	224	223
query82	771	132	136	132
query83	315	171	174	171
query84	337	84	88	84
query85	1823	316	299	299
query86	479	332	304	304
query87	3296	3085	3143	3085
query88	4025	2387	2377	2377
query89	471	392	401	392
query90	1795	191	197	191
query91	133	102	102	102
query92	59	48	49	48
query93	2446	521	517	517
query94	1133	216	216	216
query95	403	309	328	309
query96	607	275	274	274
query97	3238	3023	3024	3023
query98	228	204	196	196
query99	1724	1224	1250	1224
Total cold run time: 283117 ms
Total hot run time: 174085 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

ClickBench: Total hot run time: 30.39 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 160e343309e35fb2d12b0c1855c2e3d48c6cf4cd, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.23	0.05	0.05
query4	1.66	0.09	0.09
query5	0.50	0.47	0.49
query6	1.15	0.73	0.74
query7	0.02	0.02	0.02
query8	0.06	0.04	0.04
query9	0.56	0.49	0.50
query10	0.55	0.55	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.13
query13	0.59	0.58	0.57
query14	0.75	0.79	0.77
query15	0.84	0.82	0.81
query16	0.38	0.36	0.36
query17	0.95	0.97	0.94
query18	0.24	0.21	0.22
query19	1.83	1.69	1.72
query20	0.01	0.01	0.01
query21	15.39	0.78	0.66
query22	4.26	8.11	1.66
query23	18.25	1.41	1.36
query24	2.11	0.23	0.21
query25	0.16	0.08	0.08
query26	0.31	0.22	0.20
query27	0.45	0.23	0.23
query28	13.30	1.01	1.01
query29	12.64	3.34	3.32
query30	0.26	0.06	0.05
query31	2.86	0.39	0.40
query32	3.25	0.48	0.47
query33	2.86	2.92	2.85
query34	17.24	4.34	4.37
query35	4.41	4.44	4.44
query36	0.64	0.47	0.49
query37	0.18	0.16	0.16
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.12	0.11
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.85 s
Total hot run time: 30.39 s

yiguolei
yiguolei reviewed Jul 11, 2024
View reviewed changes
be/src/vec/columns/column_object.cpp Outdated
ss << n << " greater than column size " << subcolumn.size()
<< " sub_column_info:" << subcolumn.dump_structure()
<< " total lines of this column " << num_rows;
throw doris::Exception(ErrorCode::INTERNAL_ERROR, ss.str());
Copy link
Contributor

@yiguolei yiguolei Jul 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exception 接口有格式化string的功能，比如 throw doris::Exception(ErrorCode::INTERNAL_ERROR, “afaf {}”, 123); 可以这么写

Copy link
Member Author

@eldenmoon eldenmoon Jul 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@eldenmoon
Copy link
Member Author

eldenmoon commented Jul 11, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-H: Total hot run time: 40288 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d30008b268ee5459a590add51edce922199343d5, data reload: false

------ Round 1 ----------------------------------
q1	17609	4485	4356	4356
q2	2025	195	189	189
q3	10509	1211	1102	1102
q4	10192	790	788	788
q5	7604	2717	2687	2687
q6	220	142	142	142
q7	959	612	604	604
q8	9213	2098	2070	2070
q9	8783	6577	6568	6568
q10	9051	3837	3791	3791
q11	464	248	244	244
q12	429	233	239	233
q13	17771	2973	3003	2973
q14	274	234	241	234
q15	528	491	506	491
q16	500	388	385	385
q17	984	695	717	695
q18	8193	7624	7431	7431
q19	5691	1525	1551	1525
q20	665	332	337	332
q21	4982	3116	3256	3116
q22	395	335	332	332
Total cold run time: 117041 ms
Total hot run time: 40288 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4450	4313	4323	4313
q2	377	258	256	256
q3	3027	2944	2945	2944
q4	1983	1678	1743	1678
q5	5624	5560	5506	5506
q6	236	138	139	138
q7	2264	1842	1868	1842
q8	3276	3463	3395	3395
q9	8868	8857	8890	8857
q10	4119	3851	3774	3774
q11	605	492	521	492
q12	815	616	659	616
q13	16912	3168	3209	3168
q14	315	298	291	291
q15	527	512	495	495
q16	493	444	422	422
q17	1838	1540	1508	1508
q18	8135	7921	7946	7921
q19	1811	1560	1759	1560
q20	2990	1865	1834	1834
q21	4904	4822	4989	4822
q22	652	558	551	551
Total cold run time: 74221 ms
Total hot run time: 56383 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-DS: Total hot run time: 175054 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d30008b268ee5459a590add51edce922199343d5, data reload: false

query1	918	379	366	366
query2	6365	2444	2420	2420
query3	6644	207	215	207
query4	28125	17362	17982	17362
query5	3673	493	481	481
query6	271	168	193	168
query7	4577	305	281	281
query8	317	305	304	304
query9	8581	2432	2422	2422
query10	446	305	270	270
query11	12081	10034	9864	9864
query12	116	87	83	83
query13	1660	384	373	373
query14	10162	7563	8055	7563
query15	244	184	191	184
query16	7772	307	307	307
query17	1819	544	529	529
query18	1961	285	276	276
query19	195	155	146	146
query20	89	79	78	78
query21	204	125	126	125
query22	4309	4176	4002	4002
query23	34630	33673	33461	33461
query24	10842	2897	2894	2894
query25	530	395	397	395
query26	701	148	146	146
query27	2244	270	287	270
query28	6078	2184	2161	2161
query29	869	644	635	635
query30	263	157	156	156
query31	975	773	787	773
query32	100	55	55	55
query33	646	341	302	302
query34	939	513	517	513
query35	730	575	583	575
query36	1153	982	946	946
query37	141	81	82	81
query38	2999	2776	2749	2749
query39	848	805	801	801
query40	208	115	117	115
query41	47	46	51	46
query42	116	101	99	99
query43	596	517	542	517
query44	1060	731	747	731
query45	198	176	160	160
query46	1075	731	725	725
query47	1840	1743	1739	1739
query48	373	300	294	294
query49	822	406	427	406
query50	766	423	400	400
query51	6898	6762	6817	6762
query52	103	98	89	89
query53	358	285	285	285
query54	871	455	442	442
query55	76	74	71	71
query56	290	264	266	264
query57	1093	1041	1038	1038
query58	247	235	243	235
query59	3606	3486	3310	3310
query60	292	294	269	269
query61	98	92	92	92
query62	785	626	659	626
query63	321	290	285	285
query64	9199	2306	1798	1798
query65	3157	3089	3086	3086
query66	693	325	326	325
query67	15461	15328	14831	14831
query68	4514	545	561	545
query69	604	429	343	343
query70	1120	1108	1175	1108
query71	365	277	271	271
query72	6945	6073	5843	5843
query73	760	326	328	326
query74	6026	5545	5507	5507
query75	3361	2677	2721	2677
query76	2161	953	959	953
query77	573	312	307	307
query78	11348	9286	9231	9231
query79	4500	527	523	523
query80	1449	472	468	468
query81	580	219	222	219
query82	1253	137	142	137
query83	200	169	175	169
query84	269	100	97	97
query85	805	373	307	307
query86	387	312	334	312
query87	3285	3141	3069	3069
query88	4670	2384	2376	2376
query89	498	396	384	384
query90	1634	186	187	186
query91	131	102	109	102
query92	59	47	47	47
query93	5052	525	513	513
query94	746	209	213	209
query95	396	315	328	315
query96	614	268	267	267
query97	3252	3028	3027	3027
query98	210	194	195	194
query99	1567	1258	1260	1258
Total cold run time: 285200 ms
Total hot run time: 175054 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

ClickBench: Total hot run time: 30.33 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d30008b268ee5459a590add51edce922199343d5, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.06
query4	1.67	0.07	0.09
query5	0.50	0.48	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.47	0.49
query10	0.53	0.54	0.54
query11	0.16	0.10	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.58
query14	0.76	0.79	0.77
query15	0.86	0.81	0.82
query16	0.36	0.34	0.35
query17	0.99	1.00	0.96
query18	0.23	0.22	0.22
query19	1.80	1.81	1.66
query20	0.01	0.01	0.01
query21	15.39	0.73	0.66
query22	4.00	7.21	1.87
query23	18.28	1.33	1.26
query24	2.14	0.21	0.24
query25	0.17	0.09	0.08
query26	0.28	0.21	0.21
query27	0.46	0.24	0.23
query28	13.20	1.01	1.00
query29	12.60	3.34	3.27
query30	0.25	0.06	0.05
query31	2.86	0.39	0.38
query32	3.27	0.49	0.48
query33	2.93	2.92	2.86
query34	17.06	4.33	4.34
query35	4.43	4.43	4.38
query36	0.64	0.45	0.49
query37	0.19	0.15	0.16
query38	0.15	0.14	0.15
query39	0.05	0.04	0.04
query40	0.15	0.12	0.11
query41	0.09	0.06	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.39 s
Total hot run time: 30.33 s

yiguolei
yiguolei approved these changes Jul 12, 2024
View reviewed changes
Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jul 12, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 12, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jul 12, 2024

PR approved by anyone and no changes requested.

csun5285
csun5285 approved these changes Jul 12, 2024
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit 94ffcb5 into apache:master Jul 12, 2024
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jul 12, 2024
@eldenmoon
[Refactor](Variant) make many insterfaces exception safe (apache#37640)
523912e
@eldenmoon eldenmoon mentioned this pull request Jul 12, 2024
Merged
airborne12

This comment was marked as outdated.

Sign in to view

@eldenmoon eldenmoon deleted the interface-opt branch July 12, 2024 10:07
xiaokang pushed a commit that referenced this pull request Jul 13, 2024
@eldenmoon
[Refactor](Variant) make many insterfaces exception safe (#37640) (#3…
5162789 
…7719)
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jul 16, 2024
@eldenmoon
[Refactor](Variant) make many insterfaces exception safe (apache#37640)
ff0af01
@eldenmoon eldenmoon mentioned this pull request Jul 16, 2024
Merged
xiaokang pushed a commit that referenced this pull request Jul 17, 2024
@eldenmoon
[Pick](Variant) pick some fix (#37926)
2994232 
#37674
#37839
#37883
#37857
#37794
#37640
#37557
#37217
#37121
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
@eldenmoon @seawinde
[Refactor](Variant) make many insterfaces exception safe (apache#37640)
3347f0d
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 left review comments

@yiguolei yiguolei yiguolei approved these changes

+1 more reviewer

@csun5285 csun5285 csun5285 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@eldenmoon @doris-robot @airborne12 @yiguolei @csun5285 @xiaokang @dataroaring