Skip to content

[branch-3.0: [Fix](Catalog)Ensure preExecutionAuthenticator is properly initialized #51280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 4 commits into from
May 29, 2025
Merged

[branch-3.0: [Fix](Catalog)Ensure preExecutionAuthenticator is properly initialized #51280

dataroaring merged 4 commits into from
May 29, 2025

Conversation

@CalvinKirs
Copy link
Member

@CalvinKirs CalvinKirs commented May 27, 2025

ghkang98 and others added 3 commits May 27, 2025 14:21
@ghkang98 @CalvinKirs
[fix](iceberg) fix iceberg hadoop type kerbros issue. (apache#50623)
0679977 
Support Pre-Execution Authentication for Hadoop Type Iceberg Catalog
Operations Summary
This PR fix the hadoop type iceberg catalog kerbros use the
PreExecutionAuthenticator class, This is especially useful in
environments where secure access is required, such as Kerberos-based
Hadoop ecosystems. By integrating PreExecutionAuthenticator, each
relevant operation will undergo an authentication step prior to
execution, maintaining security compliance.

- Behavior changed:
     No.

- Does this need documentation?
     No.

Co-authored-by: lik40 <lik40@chinatelecom.cn>
(cherry picked from commit 9256996)
@CalvinKirs
branch-3.0: [Fix](Catalog)Ensure preExecutionAuthenticator is properl…
a7bd291 
…y initialized (apache#50839)

cherrypick apache#50839
@CalvinKirs
fix checkstyle
7a2d053
@CalvinKirs CalvinKirs requested a review from dataroaring as a code owner May 27, 2025 06:28
@Thearas
Copy link
Contributor

Thearas commented May 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs
Copy link
Member Author

CalvinKirs commented May 27, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 27, 2025

TPC-H: Total hot run time: 39763 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7a2d05335c8427ba1b69876f8536d8600cd35b58, data reload: false

------ Round 1 ----------------------------------
q1	17583	6711	6586	6586
q2	2062	167	163	163
q3	10570	1054	1165	1054
q4	10532	785	714	714
q5	7819	2845	2811	2811
q6	209	134	128	128
q7	978	608	615	608
q8	9372	1962	2002	1962
q9	6728	6432	6439	6432
q10	7011	2296	2276	2276
q11	472	253	259	253
q12	397	211	206	206
q13	18070	2964	3003	2964
q14	253	208	199	199
q15	512	466	447	447
q16	488	379	384	379
q17	988	593	561	561
q18	7252	6724	6657	6657
q19	1437	1076	1098	1076
q20	466	201	201	201
q21	3978	3259	3104	3104
q22	1080	982	1007	982
Total cold run time: 108257 ms
Total hot run time: 39763 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6611	6663	6659	6659
q2	336	252	263	252
q3	3050	2937	3112	2937
q4	2076	1824	1876	1824
q5	5999	6144	5967	5967
q6	217	138	138	138
q7	2331	1859	1895	1859
q8	3456	3600	3608	3600
q9	9165	8808	8960	8808
q10	3578	3503	3486	3486
q11	591	495	491	491
q12	783	600	663	600
q13	14241	3157	3155	3155
q14	309	266	271	266
q15	518	472	476	472
q16	500	437	422	422
q17	1836	1617	1601	1601
q18	8272	7772	7676	7676
q19	1712	1583	1515	1515
q20	2091	1912	1804	1804
q21	5347	5176	5622	5176
q22	1125	1077	1049	1049
Total cold run time: 74144 ms
Total hot run time: 59757 ms

@doris-robot
Copy link

doris-robot commented May 27, 2025

TPC-DS: Total hot run time: 197068 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7a2d05335c8427ba1b69876f8536d8600cd35b58, data reload: false

query1	1281	909	897	897
query2	6375	1869	1842	1842
query3	10838	4265	4171	4171
query4	61595	28646	24039	24039
query5	5214	453	441	441
query6	405	172	176	172
query7	5447	314	314	314
query8	305	232	217	217
query9	8333	2583	2582	2582
query10	453	268	258	258
query11	17632	15136	15576	15136
query12	159	102	110	102
query13	1476	445	456	445
query14	11031	7206	7251	7206
query15	207	187	180	180
query16	7183	477	518	477
query17	1143	578	606	578
query18	1951	333	332	332
query19	236	164	162	162
query20	113	109	110	109
query21	205	106	110	106
query22	4547	4559	4437	4437
query23	34331	34067	34412	34067
query24	6116	2939	2825	2825
query25	514	387	387	387
query26	648	169	159	159
query27	1830	346	357	346
query28	4067	2167	2128	2128
query29	667	425	444	425
query30	234	161	153	153
query31	1034	843	845	843
query32	69	54	59	54
query33	418	295	289	289
query34	943	507	522	507
query35	849	736	728	728
query36	1088	986	946	946
query37	108	64	68	64
query38	4152	4024	4016	4016
query39	1527	1505	1430	1430
query40	210	99	97	97
query41	49	48	50	48
query42	107	101	105	101
query43	511	457	475	457
query44	1180	844	821	821
query45	185	169	171	169
query46	1135	728	714	714
query47	2047	1924	1919	1919
query48	485	415	393	393
query49	726	404	384	384
query50	849	435	427	427
query51	7435	7188	7250	7188
query52	105	88	97	88
query53	273	182	182	182
query54	581	461	465	461
query55	87	78	76	76
query56	269	254	252	252
query57	1227	1204	1177	1177
query58	229	205	212	205
query59	3123	3111	2980	2980
query60	274	257	251	251
query61	109	109	105	105
query62	757	688	679	679
query63	220	189	185	185
query64	1389	653	665	653
query65	3249	3159	3175	3159
query66	705	290	292	290
query67	15982	15533	15713	15533
query68	4248	597	581	581
query69	423	269	267	267
query70	1181	1101	1117	1101
query71	343	255	257	255
query72	6246	4075	4085	4075
query73	752	343	345	343
query74	10562	8951	9218	8951
query75	3355	2664	2654	2654
query76	2002	1058	979	979
query77	527	292	286	286
query78	10480	9595	9588	9588
query79	1374	596	599	596
query80	878	454	453	453
query81	489	223	222	222
query82	1267	94	92	92
query83	245	147	150	147
query84	284	86	85	85
query85	908	330	290	290
query86	322	304	287	287
query87	4456	4233	4315	4233
query88	3495	2392	2371	2371
query89	406	290	290	290
query90	1993	186	180	180
query91	180	150	146	146
query92	67	49	50	49
query93	1670	554	552	552
query94	734	288	292	288
query95	347	259	255	255
query96	603	272	287	272
query97	3275	3162	3144	3144
query98	223	203	200	200
query99	1521	1309	1272	1272
Total cold run time: 313078 ms
Total hot run time: 197068 ms

@doris-robot
Copy link

doris-robot commented May 27, 2025

ClickBench: Total hot run time: 30.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7a2d05335c8427ba1b69876f8536d8600cd35b58, data reload: false

query1	0.04	0.03	0.04
query2	0.06	0.03	0.04
query3	0.23	0.07	0.06
query4	1.63	0.10	0.10
query5	0.51	0.49	0.51
query6	1.14	0.72	0.72
query7	0.02	0.02	0.03
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.56	0.55	0.56
query11	0.14	0.10	0.10
query12	0.14	0.10	0.11
query13	0.60	0.60	0.60
query14	0.80	0.77	0.81
query15	0.84	0.82	0.81
query16	0.36	0.38	0.37
query17	1.03	1.03	1.00
query18	0.24	0.22	0.22
query19	1.97	1.81	1.86
query20	0.02	0.01	0.00
query21	15.40	0.60	0.59
query22	2.67	2.19	1.77
query23	17.10	0.89	0.88
query24	2.69	1.38	1.00
query25	0.19	0.23	0.12
query26	0.37	0.13	0.13
query27	0.05	0.04	0.04
query28	10.40	0.48	0.50
query29	12.59	3.24	3.24
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.27	0.46	0.46
query33	2.97	3.00	3.03
query34	17.16	4.46	4.53
query35	4.52	4.51	4.50
query36	0.66	0.47	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.03	0.02
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.55 s
Total hot run time: 30.17 s

@CalvinKirs
Copy link
Member Author

CalvinKirs commented May 27, 2025

run buildall

dataroaring
dataroaring approved these changes May 29, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 873d355 into apache:branch-3.0 May 29, 2025
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@CalvinKirs @Thearas @doris-robot @dataroaring @ghkang98