Skip to content

[fix](hudi catalog) Fix the Kerberos authentication error when querying hudi table #51713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Jun 18, 2025
Merged

[fix](hudi catalog) Fix the Kerberos authentication error when querying hudi table #51713

morningman merged 1 commit into from
Jun 18, 2025

Conversation

@heguanhui
Copy link
Contributor

@heguanhui heguanhui commented Jun 13, 2025 •
edited
Loading

What problem does this PR solve?

This PR has resolved the Kerberos authentication error that occurs when Doris queries Hudi tables with Kerberos authentication enabled for the Hudi catalog.

Problem Summary:

When Kerberos authentication is enabled for the Hudi catalog, an error occurs when Doris queries a Hudi table.

Check List (For Author)

- [ ] Manual test (add detailed scripts or steps below)

hudi外表查询测试.docx

  • Behavior changed:

    • No.

  • Does this need documentation?

    • No.

@Thearas
Copy link
Contributor

Thearas commented Jun 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@heguanhui heguanhui changed the title [fix](hudi catalog) Fix the Kerberos authentication error when queryi… [fix](hudi catalog) Fix the Kerberos authentication error when querying hudi table Jun 15, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 17, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jun 17, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 17, 2025

PR approved by anyone and no changes requested.

@heguanhui @morningman
[fix](hudi catalog) Fix the Kerberos authentication error when queryi…
25577fc 
…ng Hudi tables with Kerberos authentication enabled for the Hudi catalog.
@morningman morningman force-pushed the fixbug/kerberos-auth-error-with-query-hudi-table branch from c484994 to 25577fc Compare June 18, 2025 03:27
@morningman
Copy link
Contributor

morningman commented Jun 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jun 18, 2025

TPC-H: Total hot run time: 33843 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 25577fc21ae864214beb6d821225bc3331ec507c, data reload: false

------ Round 1 ----------------------------------
q1	17583	5226	5007	5007
q2	1925	279	195	195
q3	10382	1339	694	694
q4	10289	995	521	521
q5	8189	2436	2304	2304
q6	189	162	133	133
q7	897	747	624	624
q8	9330	1284	1011	1011
q9	7571	5153	5129	5129
q10	6890	2364	1986	1986
q11	502	285	273	273
q12	351	350	217	217
q13	17766	3733	3082	3082
q14	233	228	221	221
q15	543	483	485	483
q16	434	440	366	366
q17	597	864	370	370
q18	7527	7242	7087	7087
q19	1216	955	586	586
q20	335	343	220	220
q21	3715	2506	2348	2348
q22	1055	1039	986	986
Total cold run time: 107519 ms
Total hot run time: 33843 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5058	5049	5042	5042
q2	238	317	218	218
q3	2132	2697	2306	2306
q4	1362	1781	1353	1353
q5	4244	4090	4459	4090
q6	211	170	126	126
q7	2007	1954	1779	1779
q8	2645	2615	2552	2552
q9	7188	7117	7148	7117
q10	3060	3301	2801	2801
q11	617	518	513	513
q12	677	777	608	608
q13	3552	3874	3245	3245
q14	278	310	269	269
q15	522	486	465	465
q16	442	500	462	462
q17	1168	1523	1376	1376
q18	7726	7365	7297	7297
q19	812	845	1018	845
q20	2011	2028	1861	1861
q21	4897	4600	4280	4280
q22	1062	1000	984	984
Total cold run time: 51909 ms
Total hot run time: 49589 ms

@doris-robot
Copy link

doris-robot commented Jun 18, 2025

TPC-DS: Total hot run time: 185712 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 25577fc21ae864214beb6d821225bc3331ec507c, data reload: false

query1	1020	394	387	387
query2	6551	1866	1858	1858
query3	6742	221	234	221
query4	26098	24041	23357	23357
query5	4403	628	463	463
query6	311	208	198	198
query7	4629	483	295	295
query8	274	228	213	213
query9	8618	2695	2691	2691
query10	491	342	275	275
query11	15405	14996	14907	14907
query12	162	112	106	106
query13	1652	547	416	416
query14	8796	6123	6146	6123
query15	200	198	168	168
query16	7268	610	491	491
query17	1215	732	584	584
query18	1977	490	301	301
query19	196	185	156	156
query20	125	121	118	118
query21	214	139	108	108
query22	3996	4140	4018	4018
query23	34174	32975	33132	32975
query24	8469	2374	2401	2374
query25	544	460	419	419
query26	1223	271	153	153
query27	2747	510	349	349
query28	4299	2166	2133	2133
query29	794	553	423	423
query30	285	219	192	192
query31	925	859	731	731
query32	75	65	65	65
query33	555	378	328	328
query34	790	862	522	522
query35	784	807	723	723
query36	976	977	869	869
query37	113	98	81	81
query38	4088	4129	3986	3986
query39	1473	1446	1421	1421
query40	210	120	109	109
query41	61	57	56	56
query42	129	108	109	108
query43	495	495	488	488
query44	1345	820	828	820
query45	179	171	169	169
query46	849	1017	624	624
query47	1736	1750	1683	1683
query48	383	423	335	335
query49	747	500	398	398
query50	655	678	399	399
query51	4100	4130	4132	4130
query52	109	110	103	103
query53	221	255	189	189
query54	581	601	507	507
query55	90	81	81	81
query56	315	299	303	299
query57	1191	1216	1113	1113
query58	266	260	240	240
query59	2597	2707	2514	2514
query60	330	324	310	310
query61	129	128	123	123
query62	818	750	678	678
query63	230	189	188	188
query64	4396	1109	777	777
query65	4388	4147	4186	4147
query66	1160	408	307	307
query67	15867	15397	15398	15397
query68	8581	890	533	533
query69	472	317	266	266
query70	1152	1076	1140	1076
query71	450	310	315	310
query72	5676	4731	4868	4731
query73	743	671	356	356
query74	8816	8824	8689	8689
query75	3932	3201	2684	2684
query76	3676	1185	761	761
query77	779	378	312	312
query78	10014	10216	9462	9462
query79	2915	826	581	581
query80	661	578	439	439
query81	474	258	223	223
query82	492	132	96	96
query83	283	252	229	229
query84	287	110	94	94
query85	782	347	314	314
query86	337	361	280	280
query87	4465	4476	4273	4273
query88	2867	2275	2277	2275
query89	430	320	283	283
query90	1943	204	210	204
query91	154	143	118	118
query92	74	63	61	61
query93	2166	934	587	587
query94	684	403	274	274
query95	376	297	287	287
query96	495	584	276	276
query97	2705	2734	2667	2667
query98	236	206	206	206
query99	1441	1435	1275	1275
Total cold run time: 274792 ms
Total hot run time: 185712 ms

@doris-robot
Copy link

doris-robot commented Jun 18, 2025

ClickBench: Total hot run time: 29.43 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 25577fc21ae864214beb6d821225bc3331ec507c, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.24	0.06	0.08
query4	1.62	0.10	0.11
query5	0.43	0.42	0.43
query6	1.16	0.65	0.67
query7	0.02	0.02	0.01
query8	0.05	0.03	0.03
query9	0.58	0.52	0.50
query10	0.57	0.57	0.56
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.61
query14	0.80	0.85	0.80
query15	0.90	0.87	0.88
query16	0.40	0.39	0.38
query17	1.06	1.07	1.05
query18	0.23	0.21	0.22
query19	1.94	1.86	1.82
query20	0.01	0.01	0.02
query21	15.40	0.89	0.54
query22	0.76	1.21	0.78
query23	14.79	1.37	0.63
query24	7.11	0.96	0.56
query25	0.45	0.13	0.11
query26	0.68	0.16	0.14
query27	0.05	0.05	0.05
query28	8.80	0.91	0.47
query29	12.61	4.06	3.36
query30	0.25	0.10	0.07
query31	2.82	0.63	0.39
query32	3.24	0.56	0.47
query33	3.09	3.06	3.25
query34	16.10	5.40	4.79
query35	4.78	4.86	4.82
query36	0.69	0.50	0.49
query37	0.10	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.15	0.14
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 103.17 s
Total hot run time: 29.43 s

@morningman morningman merged commit 89af088 into apache:master Jun 18, 2025
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Jun 18, 2025
@heguanhui
[fix](hudi catalog) Fix the Kerberos authentication error when queryi…
85c718e 
…ng hudi table (#51713)

### What problem does this PR solve?

This PR has resolved the Kerberos authentication error that occurs when
Doris queries Hudi tables with Kerberos authentication enabled for the
Hudi catalog.

Problem Summary:

When Kerberos authentication is enabled for the Hudi catalog, an error
occurs when Doris queries a Hudi table.
@github-actions github-actions bot mentioned this pull request Jun 18, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Jun 18, 2025
@heguanhui
[fix](hudi catalog) Fix the Kerberos authentication error when queryi…
0b71274 
…ng hudi table (#51713)

### What problem does this PR solve?

This PR has resolved the Kerberos authentication error that occurs when
Doris queries Hudi tables with Kerberos authentication enabled for the
Hudi catalog.

Problem Summary:

When Kerberos authentication is enabled for the Hudi catalog, an error
occurs when Doris queries a Hudi table.
@github-actions github-actions bot mentioned this pull request Jun 18, 2025
Merged
@morningman morningman added the usercase Important user case type label label Jun 18, 2025
dataroaring pushed a commit that referenced this pull request Jun 20, 2025
@github-actions @heguanhui
branch-3.0: [fix](hudi catalog) Fix the Kerberos authentication error…
2251d1f 
… when querying hudi table #51713 (#51902)

Cherry-picked from #51713

Co-authored-by: heguanhui <hgh_wy163mail@163.com>
github-actions bot pushed a commit that referenced this pull request Jun 20, 2025
@heguanhui
[fix](hudi catalog) Fix the Kerberos authentication error when queryi…
c700bf4 
…ng hudi table (#51713)

### What problem does this PR solve?

This PR has resolved the Kerberos authentication error that occurs when
Doris queries Hudi tables with Kerberos authentication enabled for the
Hudi catalog.

Problem Summary:

When Kerberos authentication is enabled for the Hudi catalog, an error
occurs when Doris queries a Hudi table.
@github-actions github-actions bot mentioned this pull request Jun 20, 2025
Merged
morrySnow pushed a commit that referenced this pull request Jun 20, 2025
@github-actions @heguanhui
branch-3.1: [fix](hudi catalog) Fix the Kerberos authentication error…
1e430a4 
… when querying hudi table #51713 (#52027)

Cherry-picked from #51713

Co-authored-by: heguanhui <hgh_wy163mail@163.com>
yiguolei pushed a commit that referenced this pull request Jun 24, 2025
@heguanhui @yiguolei
[fix](hudi catalog) Fix the Kerberos authentication error when queryi…
ddf3845 
…ng hudi table (#51713)

### What problem does this PR solve?

This PR has resolved the Kerberos authentication error that occurs when
Doris queries Hudi tables with Kerberos authentication enabled for the
Hudi catalog.

Problem Summary:

When Kerberos authentication is enabled for the Hudi catalog, an error
occurs when Doris queries a Hudi table.
yiguolei pushed a commit that referenced this pull request Jun 24, 2025
@github-actions @heguanhui
branch-2.1: [fix](hudi catalog) Fix the Kerberos authentication error…
93d7eb5 
… when querying hudi table #51713 (#51903)

Cherry-picked from #51713

Co-authored-by: heguanhui <hgh_wy163mail@163.com>
@yiguolei yiguolei mentioned this pull request Aug 11, 2025
Open
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.7-merged dev/3.1.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@heguanhui @Thearas @morningman @doris-robot @CalvinKirs @yiguolei @dataroaring @morrySnow