Skip to content

branch-3.0: [Bug](distinct) fix distinct function with over return error result #51875 #52312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jun 28, 2025
Merged

branch-3.0: [Bug](distinct) fix distinct function with over return error result #51875 #52312

dataroaring merged 1 commit into from
Jun 28, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jun 25, 2025

Cherry-picked from #51875

@github-actions github-actions bot requested a review from dataroaring as a code owner June 25, 2025 11:39
@dataroaring dataroaring force-pushed the auto-pick-51875-branch-3.0 branch from de2354a to 51e9ee7 Compare June 26, 2025 02:30
@dataroaring
Copy link
Contributor

dataroaring commented Jun 26, 2025

run buildall

@zhangstar333 @dataroaring
[Bug](distinct) fix distinct function with over return error result (#…
3dc73b8 
…51875)

### What problem does this PR solve?
Problem Summary:
before the result is not correctly, as after insert the rows, should
reset all state of nested function.
So the result of the next line will not be affected by the previous one
```
mysql> select id, v1, multi_distinct_group_concat(v2) over() from multi;
+------+------+----------------------------------------+
| id   | v1   | multi_distinct_group_concat(v2) over() |
+------+------+----------------------------------------+
|    3 |    1 | a                                      |
|    1 |    1 | a,a                                    |
|    1 |    1 | a,a,a                                  |
|    2 |    1 | a,a,a,a                                |
+------+------+----------------------------------------+
4 rows in set (0.21 sec)

mysql> select id, v1, multi_distinct_sum(v1) over() from multi;
+------+------+-------------------------------+
| id   | v1   | multi_distinct_sum(v1) over() |
+------+------+-------------------------------+
|    1 |    1 |                             1 |
|    1 |    1 |                             2 |
|    2 |    1 |                             3 |
|    3 |    1 |                             4 |
+------+------+-------------------------------+
4 rows in set (0.06 sec)
```
@dataroaring dataroaring force-pushed the auto-pick-51875-branch-3.0 branch from 51e9ee7 to 3dc73b8 Compare June 26, 2025 11:18
@dataroaring
Copy link
Contributor

dataroaring commented Jun 26, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jun 26, 2025

TPC-H: Total hot run time: 39877 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3dc73b80876f93ab9a9f371cf790bc7f8b1b6f45, data reload: false

------ Round 1 ----------------------------------
q1	17728	6922	6610	6610
q2	2072	175	171	171
q3	10641	1077	1201	1077
q4	10524	725	681	681
q5	7737	2880	2815	2815
q6	212	136	133	133
q7	998	633	618	618
q8	9357	1956	2074	1956
q9	6567	6384	6429	6384
q10	6977	2271	2307	2271
q11	480	270	258	258
q12	401	215	212	212
q13	17806	3009	2995	2995
q14	245	208	208	208
q15	519	456	450	450
q16	487	370	379	370
q17	993	555	561	555
q18	7344	6826	6720	6720
q19	1398	1053	1135	1053
q20	481	200	200	200
q21	4111	3316	3128	3128
q22	1103	1012	1024	1012
Total cold run time: 108181 ms
Total hot run time: 39877 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6629	6547	6558	6547
q2	338	225	232	225
q3	2884	2737	2900	2737
q4	2025	1811	1741	1741
q5	5800	5742	5753	5742
q6	219	126	131	126
q7	2203	1812	1776	1776
q8	3404	3564	3509	3509
q9	8988	8699	8956	8699
q10	3547	3544	3519	3519
q11	597	513	497	497
q12	811	599	603	599
q13	8636	3181	3223	3181
q14	298	271	266	266
q15	518	454	458	454
q16	488	446	438	438
q17	1845	1632	1615	1615
q18	8195	7786	7845	7786
q19	1700	1662	1427	1427
q20	2157	1833	1829	1829
q21	5089	4965	5079	4965
q22	1166	1060	1024	1024
Total cold run time: 67537 ms
Total hot run time: 58702 ms

@doris-robot
Copy link

doris-robot commented Jun 26, 2025

TPC-DS: Total hot run time: 197761 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3dc73b80876f93ab9a9f371cf790bc7f8b1b6f45, data reload: false

query1	1294	921	899	899
query2	6355	1966	1902	1902
query3	10935	4380	4577	4380
query4	61525	29808	23584	23584
query5	5228	462	458	458
query6	391	190	193	190
query7	5436	317	314	314
query8	317	234	225	225
query9	8428	2572	2593	2572
query10	446	270	250	250
query11	17483	15427	15818	15427
query12	177	113	106	106
query13	1415	465	444	444
query14	10315	7619	7130	7130
query15	205	171	182	171
query16	7111	508	509	508
query17	1129	563	559	559
query18	1936	326	322	322
query19	212	162	166	162
query20	117	109	115	109
query21	203	112	105	105
query22	4616	4458	4709	4458
query23	34557	34157	34025	34025
query24	6262	3013	2882	2882
query25	539	414	408	408
query26	683	176	182	176
query27	1914	361	355	355
query28	4098	2213	2174	2174
query29	707	473	469	469
query30	243	159	157	157
query31	975	832	819	819
query32	71	56	61	56
query33	453	310	306	306
query34	939	525	528	525
query35	860	755	754	754
query36	1094	951	970	951
query37	119	65	72	65
query38	4094	4018	4005	4005
query39	1557	1498	1490	1490
query40	203	102	102	102
query41	50	50	48	48
query42	120	106	98	98
query43	571	486	489	486
query44	1225	849	819	819
query45	197	170	180	170
query46	1143	746	731	731
query47	2026	1888	1931	1888
query48	485	378	377	377
query49	723	397	394	394
query50	851	450	456	450
query51	7357	7244	7365	7244
query52	108	103	92	92
query53	290	188	185	185
query54	577	469	478	469
query55	79	76	80	76
query56	282	269	254	254
query57	1289	1209	1212	1209
query58	242	222	231	222
query59	3162	3043	3025	3025
query60	309	265	261	261
query61	113	106	109	106
query62	789	674	728	674
query63	226	198	197	197
query64	1497	688	650	650
query65	3260	3208	3204	3204
query66	720	286	290	286
query67	16030	15559	15502	15502
query68	3707	607	593	593
query69	429	268	268	268
query70	1198	1081	1093	1081
query71	354	269	257	257
query72	6230	4009	4032	4009
query73	764	351	354	351
query74	10136	8976	9061	8976
query75	3346	2652	2656	2652
query76	2109	1062	1056	1056
query77	476	270	281	270
query78	10685	9630	9597	9597
query79	1603	608	612	608
query80	894	430	427	427
query81	495	225	225	225
query82	1287	93	94	93
query83	245	141	144	141
query84	279	82	75	75
query85	931	303	288	288
query86	332	300	305	300
query87	4394	4321	4304	4304
query88	3977	2370	2377	2370
query89	422	302	290	290
query90	2017	186	185	185
query91	184	148	148	148
query92	61	51	53	51
query93	1890	570	558	558
query94	749	304	290	290
query95	353	259	255	255
query96	638	281	277	277
query97	3308	3158	3150	3150
query98	236	198	201	198
query99	1582	1319	1307	1307
Total cold run time: 313674 ms
Total hot run time: 197761 ms

@doris-robot
Copy link

doris-robot commented Jun 26, 2025

ClickBench: Total hot run time: 29.97 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3dc73b80876f93ab9a9f371cf790bc7f8b1b6f45, data reload: false

query1	0.04	0.03	0.04
query2	0.06	0.03	0.04
query3	0.24	0.07	0.06
query4	1.63	0.10	0.10
query5	0.51	0.51	0.52
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.03	0.03	0.03
query9	0.57	0.49	0.50
query10	0.56	0.55	0.57
query11	0.14	0.10	0.11
query12	0.14	0.10	0.11
query13	0.60	0.60	0.59
query14	0.77	0.79	0.82
query15	0.85	0.83	0.83
query16	0.39	0.38	0.38
query17	1.05	1.03	1.10
query18	0.22	0.22	0.21
query19	1.85	1.83	1.89
query20	0.01	0.02	0.01
query21	15.40	0.62	0.59
query22	2.29	2.15	1.53
query23	17.17	0.86	0.90
query24	2.97	0.96	1.73
query25	0.15	0.16	0.22
query26	0.42	0.13	0.14
query27	0.05	0.04	0.03
query28	9.87	0.50	0.50
query29	12.61	3.24	3.22
query30	0.24	0.06	0.06
query31	2.87	0.40	0.36
query32	3.24	0.46	0.46
query33	3.03	2.99	3.01
query34	17.18	4.57	4.50
query35	4.64	4.47	4.64
query36	0.68	0.46	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.12	0.11
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.12 s
Total hot run time: 29.97 s

dataroaring
dataroaring approved these changes Jun 28, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit c6a228e into branch-3.0 Jun 28, 2025
25 of 30 checks passed
@github-actions github-actions bot deleted the auto-pick-51875-branch-3.0 branch June 28, 2025 09:26
koarz pushed a commit to koarz/doris that referenced this pull request Jul 3, 2025
@github-actions @zhangstar333 @koarz
branch-3.0: [Bug](distinct) fix distinct function with over return er…
ccc0977 
…ror result apache#51875 (apache#52312)

Cherry-picked from apache#51875

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@dataroaring @doris-robot @zhangstar333