Skip to content

[fix](hudi) Disable nested column pruning for HUDI tables to avoid potential issues #59860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Jan 14, 2026
Merged

[fix](hudi) Disable nested column pruning for HUDI tables to avoid potential issues #59860

morningman merged 1 commit into from
Jan 14, 2026
+6 −0

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Jan 14, 2026 •
edited
Loading

What problem does this PR solve?

Related PR: #57204

Problem Summary:

Disable nested column pruning for HUDI tables to avoid potential issues

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Jan 14, 2026

run buildall

@Thearas
Copy link
Contributor

Thearas commented Jan 14, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

doris-robot commented Jan 14, 2026

TPC-H: Total hot run time: 31894 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b903b944b9061e336a9686c8389d5ecb775a269c, data reload: false

------ Round 1 ----------------------------------
q1	17579	4228	4037	4037
q2	2012	362	252	252
q3	10161	1264	698	698
q4	10201	830	314	314
q5	7548	2079	1807	1807
q6	187	168	142	142
q7	912	803	648	648
q8	9267	1425	1143	1143
q9	4849	4624	4658	4624
q10	6800	1778	1412	1412
q11	525	292	276	276
q12	671	740	572	572
q13	17770	3842	3094	3094
q14	290	292	267	267
q15	572	511	501	501
q16	677	674	616	616
q17	647	798	488	488
q18	6392	6479	6758	6479
q19	1257	1071	644	644
q20	413	388	267	267
q21	3248	2590	2681	2590
q22	1181	1094	1023	1023
Total cold run time: 103159 ms
Total hot run time: 31894 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4330	4274	4270	4270
q2	330	421	326	326
q3	2390	2781	2373	2373
q4	1430	1881	1445	1445
q5	4468	4486	4234	4234
q6	207	165	126	126
q7	1967	1905	1821	1821
q8	2535	2411	2351	2351
q9	7340	6887	6990	6887
q10	2587	2746	2339	2339
q11	546	474	446	446
q12	721	749	677	677
q13	3674	3815	3098	3098
q14	269	291	259	259
q15	518	483	476	476
q16	620	662	595	595
q17	1108	1204	1304	1204
q18	7477	7313	7288	7288
q19	873	829	806	806
q20	1875	1963	1787	1787
q21	4594	4265	4160	4160
q22	1035	1019	958	958
Total cold run time: 50894 ms
Total hot run time: 47926 ms

@doris-robot
Copy link

doris-robot commented Jan 14, 2026

TPC-DS: Total hot run time: 173386 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b903b944b9061e336a9686c8389d5ecb775a269c, data reload: false

query5	4392	634	481	481
query6	325	226	212	212
query7	4224	451	258	258
query8	333	272	235	235
query9	8703	2892	2901	2892
query10	524	402	318	318
query11	15229	15089	14876	14876
query12	183	116	113	113
query13	1263	496	369	369
query14	6152	3136	2905	2905
query14_1	2764	2724	2773	2724
query15	208	190	173	173
query16	992	473	473	473
query17	1114	693	593	593
query18	2451	440	341	341
query19	235	233	207	207
query20	124	120	116	116
query21	221	138	118	118
query22	3798	3994	3786	3786
query23	15910	15573	15298	15298
query23_1	15382	15374	15466	15374
query24	7173	1595	1187	1187
query24_1	1203	1184	1215	1184
query25	568	485	424	424
query26	1241	265	163	163
query27	2752	446	264	264
query28	4594	2162	2170	2162
query29	782	544	456	456
query30	318	245	215	215
query31	818	639	559	559
query32	88	73	74	73
query33	546	388	298	298
query34	916	882	537	537
query35	710	757	669	669
query36	912	904	847	847
query37	135	98	79	79
query38	2699	2742	2701	2701
query39	763	773	749	749
query39_1	695	691	709	691
query40	218	132	118	118
query41	67	62	63	62
query42	108	107	108	107
query43	433	483	409	409
query44	1344	739	727	727
query45	188	183	179	179
query46	864	976	584	584
query47	1440	1420	1355	1355
query48	289	320	229	229
query49	607	426	339	339
query50	640	311	202	202
query51	3768	3721	3794	3721
query52	115	116	100	100
query53	296	328	275	275
query54	285	270	258	258
query55	82	81	76	76
query56	309	337	302	302
query57	1011	1007	940	940
query58	287	265	252	252
query59	2136	2234	2098	2098
query60	330	355	313	313
query61	158	154	161	154
query62	394	339	311	311
query63	313	266	277	266
query64	4867	1287	979	979
query65	3802	3712	3774	3712
query66	1463	428	322	322
query67	15083	14822	15597	14822
query68	5434	1021	701	701
query69	533	350	320	320
query70	1066	917	962	917
query71	363	319	301	301
query72	5681	3412	3436	3412
query73	775	776	304	304
query74	8742	8777	8576	8576
query75	2794	2834	2469	2469
query76	3460	1087	663	663
query77	595	402	307	307
query78	9660	9843	9252	9252
query79	999	896	569	569
query80	670	625	497	497
query81	488	266	235	235
query82	214	144	116	116
query83	258	257	246	246
query84	255	118	112	112
query85	869	518	451	451
query86	332	299	289	289
query87	2860	2886	2801	2801
query88	2971	2125	2101	2101
query89	392	348	342	342
query90	2043	171	165	165
query91	170	164	143	143
query92	76	79	66	66
query93	1033	946	532	532
query94	571	314	303	303
query95	578	339	368	339
query96	563	490	194	194
query97	2299	2379	2295	2295
query98	215	204	211	204
query99	587	571	524	524
Total cold run time: 248756 ms
Total hot run time: 173386 ms

@doris-robot
Copy link

doris-robot commented Jan 14, 2026

ClickBench: Total hot run time: 26.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b903b944b9061e336a9686c8389d5ecb775a269c, data reload: false

query1	0.05	0.04	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.61	0.12	0.11
query5	0.28	0.25	0.26
query6	1.14	0.67	0.66
query7	0.04	0.02	0.03
query8	0.05	0.05	0.04
query9	0.56	0.50	0.50
query10	0.54	0.55	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.59	0.58
query14	0.95	0.95	0.94
query15	0.79	0.78	0.78
query16	0.38	0.39	0.38
query17	1.04	1.05	1.07
query18	0.23	0.22	0.21
query19	1.94	1.89	1.85
query20	0.05	0.01	0.01
query21	15.41	0.24	0.14
query22	5.05	0.05	0.04
query23	15.94	0.29	0.11
query24	1.36	0.62	0.17
query25	0.08	0.05	0.06
query26	0.14	0.13	0.12
query27	0.08	0.05	0.04
query28	4.22	1.06	0.88
query29	12.58	3.90	3.13
query30	0.28	0.14	0.12
query31	2.83	0.64	0.39
query32	3.23	0.57	0.45
query33	2.98	3.00	3.10
query34	15.96	5.15	4.47
query35	4.45	4.45	4.54
query36	0.67	0.50	0.50
query37	0.10	0.06	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.16	0.14	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.7 s
Total hot run time: 26.71 s

@github-actions
Copy link
Contributor

github-actions bot commented Jan 14, 2026

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 14, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 14, 2026

PR approved by anyone and no changes requested.

@suxiaogang223 suxiaogang223 mentioned this pull request Jan 14, 2026
Merged
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 14, 2026

FE Regression Coverage Report

Increment line coverage 0.00% (0/3) 🎉
Increment coverage report
Complete coverage report

@morningman morningman merged commit f0ae066 into apache:master Jan 14, 2026
33 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 14, 2026
@suxiaogang223
[fix](hudi) Disable nested column pruning for HUDI tables to avoid po…
499c038 
…tential issues (#59860)

### What problem does this PR solve?

Related PR: #57204 

Problem Summary:

Disable nested column pruning for HUDI tables to avoid potential issues
@github-actions github-actions bot mentioned this pull request Jan 14, 2026
Closed
yiguolei pushed a commit that referenced this pull request Jan 15, 2026
@suxiaogang223
branch-4.0: [fix](hudi) Disable nested column pruning for HUDI tables…
555e561 
… to avoid potential issues (#59860) (#59869)

cherry pick from #59860
@suxiaogang223 suxiaogang223 deleted the disable_nested_prune_for_hudi branch January 17, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

+1 more reviewer

@hubgeter hubgeter hubgeter approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@suxiaogang223 @Thearas @doris-robot @hello-stephen @morningman @CalvinKirs @hubgeter @yiguolei