Skip to content

[opt](count) select smallest column for count star #56093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 4 commits into from
Sep 18, 2025
Merged

[opt](count) select smallest column for count star #56093

morningman merged 4 commits into from
Sep 18, 2025

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Sep 16, 2025 •
edited
Loading

What problem does this PR solve?

sql like select count(*) from tbl, we should choose column with smallest size to read, for performance.
This logic exists in old planner, but missing in Nereids

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 16, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

morrySnow
morrySnow reviewed Sep 16, 2025
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java Outdated
} else if (s.getType().isComplexType()) {
return 3;
} else {
return Integer.MAX_VALUE;
Copy link
Contributor

@morrySnow morrySnow Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so read date type column is slow than struct?

Copy link
Contributor Author

@morningman morningman Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

miss it. fixed

@morningman
Copy link
Contributor Author

morningman commented Sep 16, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 16, 2025

TPC-H: Total hot run time: 35023 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2b6d03589858cda9aca4feb10619b84430619850, data reload: false

------ Round 1 ----------------------------------
q1	17600	5264	5103	5103
q2	2031	320	217	217
q3	10264	1348	757	757
q4	10225	1033	543	543
q5	7570	2428	2403	2403
q6	186	170	139	139
q7	961	780	656	656
q8	9376	1355	1138	1138
q9	7159	5202	5227	5202
q10	6981	2404	1971	1971
q11	498	310	289	289
q12	382	370	245	245
q13	17781	3697	3015	3015
q14	245	250	225	225
q15	572	508	489	489
q16	1025	1008	944	944
q17	602	881	374	374
q18	7784	7235	7089	7089
q19	1118	973	575	575
q20	355	346	240	240
q21	3945	2617	2402	2402
q22	1080	1040	1007	1007
Total cold run time: 107740 ms
Total hot run time: 35023 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5138	5118	5103	5103
q2	257	333	231	231
q3	2229	2642	2292	2292
q4	1336	1804	1325	1325
q5	4215	4621	4542	4542
q6	215	192	133	133
q7	2239	1946	1930	1930
q8	2682	2654	2610	2610
q9	7447	7329	7529	7329
q10	3150	3353	2874	2874
q11	604	531	527	527
q12	722	799	600	600
q13	3568	3934	3290	3290
q14	285	315	269	269
q15	521	483	477	477
q16	1094	1118	1050	1050
q17	1189	1780	1399	1399
q18	8060	7774	7589	7589
q19	829	823	816	816
q20	1994	1966	1818	1818
q21	4820	4325	4295	4295
q22	1063	1044	1007	1007
Total cold run time: 53657 ms
Total hot run time: 51506 ms

@doris-robot
Copy link

doris-robot commented Sep 16, 2025

TPC-DS: Total hot run time: 188544 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2b6d03589858cda9aca4feb10619b84430619850, data reload: false

query1	1088	417	422	417
query2	6559	1702	1705	1702
query3	6754	222	226	222
query4	26094	23396	23002	23002
query5	4412	608	481	481
query6	327	225	225	225
query7	4645	507	306	306
query8	306	248	239	239
query9	8651	2650	2655	2650
query10	456	356	289	289
query11	15681	14972	14925	14925
query12	161	112	112	112
query13	1669	564	425	425
query14	10261	9074	9155	9074
query15	213	198	174	174
query16	7212	673	546	546
query17	1035	727	617	617
query18	2008	423	327	327
query19	205	202	171	171
query20	164	124	118	118
query21	220	143	115	115
query22	4022	4232	4013	4013
query23	33985	32931	32772	32772
query24	8512	2425	2442	2425
query25	611	507	434	434
query26	1236	277	167	167
query27	2740	511	371	371
query28	4384	2291	2237	2237
query29	828	592	526	526
query30	296	232	202	202
query31	913	816	740	740
query32	83	72	69	69
query33	582	383	348	348
query34	807	850	519	519
query35	813	853	750	750
query36	984	1049	939	939
query37	111	106	82	82
query38	3488	3564	3458	3458
query39	1500	1429	1419	1419
query40	216	132	122	122
query41	67	62	65	62
query42	136	122	114	114
query43	525	505	489	489
query44	1354	858	853	853
query45	191	180	170	170
query46	840	1033	655	655
query47	1759	1817	1750	1750
query48	392	426	316	316
query49	773	525	462	462
query50	650	692	412	412
query51	3962	4019	3889	3889
query52	121	110	111	110
query53	241	267	190	190
query54	596	597	528	528
query55	89	87	86	86
query56	331	331	305	305
query57	1165	1171	1145	1145
query58	272	267	264	264
query59	2556	2616	2510	2510
query60	344	328	357	328
query61	176	166	159	159
query62	806	765	661	661
query63	230	196	207	196
query64	4661	1255	916	916
query65	4079	3981	3964	3964
query66	1204	456	365	365
query67	15365	15263	15326	15263
query68	7906	939	595	595
query69	510	324	358	324
query70	1361	1301	1309	1301
query71	568	342	324	324
query72	5955	5069	5271	5069
query73	704	656	362	362
query74	9177	9220	8810	8810
query75	3966	3338	2792	2792
query76	3469	1159	726	726
query77	828	409	318	318
query78	9620	9765	8850	8850
query79	2287	837	591	591
query80	675	564	504	504
query81	495	254	228	228
query82	461	162	128	128
query83	301	260	252	252
query84	304	114	90	90
query85	873	469	412	412
query86	387	297	309	297
query87	3717	3771	3738	3738
query88	3635	2254	2235	2235
query89	391	329	305	305
query90	1902	213	217	213
query91	166	168	140	140
query92	90	68	62	62
query93	1919	1021	646	646
query94	692	427	322	322
query95	389	315	309	309
query96	478	567	279	279
query97	2971	2998	2909	2909
query98	254	220	210	210
query99	1432	1376	1311	1311
Total cold run time: 275000 ms
Total hot run time: 188544 ms

@doris-robot
Copy link

doris-robot commented Sep 16, 2025

ClickBench: Total hot run time: 30.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2b6d03589858cda9aca4feb10619b84430619850, data reload: false

query1	0.05	0.05	0.04
query2	0.10	0.06	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.11
query5	0.28	0.27	0.25
query6	1.19	0.65	0.64
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.63	0.53	0.52
query10	0.57	0.56	0.59
query11	0.16	0.11	0.12
query12	0.18	0.12	0.12
query13	0.64	0.63	0.62
query14	1.04	1.05	1.04
query15	0.86	0.86	0.87
query16	0.41	0.42	0.40
query17	1.06	1.05	1.05
query18	0.22	0.20	0.21
query19	1.92	1.91	1.85
query20	0.02	0.02	0.01
query21	15.42	0.96	0.60
query22	0.75	1.15	0.70
query23	14.96	1.41	0.58
query24	7.05	1.52	0.68
query25	0.49	0.24	0.16
query26	0.67	0.16	0.14
query27	0.07	0.06	0.05
query28	9.68	0.91	0.45
query29	12.53	3.92	3.26
query30	0.28	0.14	0.14
query31	2.82	0.61	0.40
query32	3.24	0.55	0.49
query33	3.05	3.16	3.11
query34	16.15	5.51	4.92
query35	4.92	4.94	5.00
query36	0.68	0.52	0.50
query37	0.11	0.08	0.07
query38	0.06	0.05	0.05
query39	0.04	0.03	0.03
query40	0.18	0.17	0.15
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 104.61 s
Total hot run time: 30.1 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 16, 2025

FE Regression Coverage Report

Increment line coverage 85.71% (18/21) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 18, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 18, 2025

PR approved by anyone and no changes requested.

@morningman morningman merged commit e0c60f3 into apache:master Sep 18, 2025
32 of 35 checks passed
morningman added a commit to morningman/doris that referenced this pull request Sep 25, 2025
@morningman
[opt](count) select smallest column for count star (apache#56093)
ef4ef16 
sql like `select count(*) from tbl`, we should choose column with
smallest size to read, for performance.
This logic exists in old planner, but missing in Nereids
@morningman morningman mentioned this pull request Sep 25, 2025
Merged
morningman added a commit that referenced this pull request Sep 28, 2025
@morningman
branch-3.1: [opt](count) select smallest column for count star (#56093)…
562f07a 
… (#56483)

bp #56093
yiguolei pushed a commit to yiguolei/incubator-doris that referenced this pull request Sep 28, 2025
@morningman
branch-3.0: [opt](count) select smallest column for count star apache…
c871919 
…#56093 (apache#4685)

bp apache#56093
related apache#4597
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CalvinKirs CalvinKirs CalvinKirs approved these changes

@morrySnow morrySnow morrySnow approved these changes

+2 more reviewers

@suxiaogang223 suxiaogang223 suxiaogang223 approved these changes

@hubgeter hubgeter hubgeter approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.x dev/3.0.x-conflict dev/3.1.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@morningman @hello-stephen @doris-robot @CalvinKirs @suxiaogang223 @hubgeter @morrySnow