Skip to content

[fix](nereids) fix colocate agg + join compute wrong result #48934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
924060929 merged 1 commit into from
Mar 12, 2025
Merged

[fix](nereids) fix colocate agg + join compute wrong result #48934

924060929 merged 1 commit into from
Mar 12, 2025

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Mar 11, 2025

What problem does this PR solve?

fix colocate agg + join compute wrong result, introduced by #41730

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Mar 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

924060929 commented Mar 11, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Mar 11, 2025

TPC-H: Total hot run time: 32484 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 73880af83e42d30b52e6452f35251a88a0be6d5e, data reload: false

------ Round 1 ----------------------------------
q1	17639	5289	5106	5106
q2	2044	294	154	154
q3	10446	1231	752	752
q4	10292	1080	547	547
q5	8272	2375	2315	2315
q6	186	157	129	129
q7	906	737	616	616
q8	9301	1276	1099	1099
q9	5042	4851	4740	4740
q10	6805	2302	1892	1892
q11	472	281	264	264
q12	348	347	222	222
q13	17796	3693	3055	3055
q14	227	231	218	218
q15	527	483	477	477
q16	623	630	580	580
q17	567	854	337	337
q18	6718	6415	6299	6299
q19	1771	976	543	543
q20	325	324	186	186
q21	2704	2159	1950	1950
q22	1064	1040	1003	1003
Total cold run time: 104075 ms
Total hot run time: 32484 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5266	5169	5155	5155
q2	247	322	228	228
q3	2148	2692	2263	2263
q4	1471	1838	1392	1392
q5	4233	4109	4149	4109
q6	207	165	124	124
q7	1946	1951	1772	1772
q8	2640	2623	2583	2583
q9	7263	7268	7185	7185
q10	2998	3211	2781	2781
q11	592	517	484	484
q12	636	743	602	602
q13	3585	3948	3256	3256
q14	272	296	267	267
q15	536	465	466	465
q16	658	668	648	648
q17	1172	1579	1353	1353
q18	7850	7457	7581	7457
q19	820	808	855	808
q20	1976	2081	1949	1949
q21	5457	5025	4741	4741
q22	1137	1101	1062	1062
Total cold run time: 53110 ms
Total hot run time: 50684 ms

@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 11, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Mar 11, 2025

TPC-DS: Total hot run time: 192336 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 73880af83e42d30b52e6452f35251a88a0be6d5e, data reload: false

query1	1395	991	977	977
query2	6141	1910	1899	1899
query3	11020	4408	4394	4394
query4	54928	25848	22989	22989
query5	5115	561	486	486
query6	347	184	176	176
query7	4883	492	292	292
query8	312	249	235	235
query9	5704	2650	2655	2650
query10	441	328	249	249
query11	15236	15124	14890	14890
query12	154	107	102	102
query13	1043	529	381	381
query14	10619	6537	6486	6486
query15	208	193	177	177
query16	7093	658	481	481
query17	1099	742	588	588
query18	1508	426	347	347
query19	197	195	176	176
query20	131	129	124	124
query21	209	126	111	111
query22	4528	4515	4370	4370
query23	33896	33497	33225	33225
query24	5780	2479	2517	2479
query25	494	497	470	470
query26	727	269	152	152
query27	2056	482	336	336
query28	3001	2486	2471	2471
query29	567	571	438	438
query30	277	223	189	189
query31	865	848	800	800
query32	73	61	62	61
query33	479	366	316	316
query34	897	850	571	571
query35	791	836	781	781
query36	966	1003	903	903
query37	117	98	70	70
query38	4421	4159	4231	4159
query39	1478	1476	1473	1473
query40	216	118	104	104
query41	54	54	53	53
query42	120	107	102	102
query43	508	518	515	515
query44	1312	831	815	815
query45	182	184	167	167
query46	864	1070	634	634
query47	1828	1873	1802	1802
query48	382	432	318	318
query49	715	533	466	466
query50	712	762	431	431
query51	4269	4344	4235	4235
query52	114	111	103	103
query53	245	264	198	198
query54	514	500	424	424
query55	81	86	79	79
query56	280	270	278	270
query57	1153	1183	1098	1098
query58	244	241	255	241
query59	2832	2914	2590	2590
query60	288	300	304	300
query61	126	123	155	123
query62	755	725	675	675
query63	230	196	197	196
query64	1776	1041	702	702
query65	4562	4450	4381	4381
query66	786	399	299	299
query67	15879	15813	15534	15534
query68	7337	882	490	490
query69	539	297	258	258
query70	1223	1117	1060	1060
query71	482	286	263	263
query72	5571	3655	3773	3655
query73	1489	741	345	345
query74	9014	9152	9024	9024
query75	3878	3145	2755	2755
query76	4301	1199	771	771
query77	631	361	276	276
query78	10073	10106	9391	9391
query79	2545	813	581	581
query80	617	522	454	454
query81	493	270	220	220
query82	541	131	102	102
query83	274	176	148	148
query84	290	91	74	74
query85	765	350	392	350
query86	368	322	281	281
query87	4444	4580	4348	4348
query88	3666	2333	2267	2267
query89	403	306	288	288
query90	1913	210	213	210
query91	138	141	110	110
query92	78	57	60	57
query93	1927	1047	572	572
query94	663	399	303	303
query95	353	274	267	267
query96	480	572	278	278
query97	3359	3409	3303	3303
query98	241	211	204	204
query99	1474	1400	1302	1302
Total cold run time: 299793 ms
Total hot run time: 192336 ms

@doris-robot
Copy link

doris-robot commented Mar 11, 2025

ClickBench: Total hot run time: 30.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 73880af83e42d30b52e6452f35251a88a0be6d5e, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.11
query5	0.56	0.55	0.55
query6	1.21	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.52	0.52
query10	0.58	0.62	0.59
query11	0.16	0.10	0.10
query12	0.15	0.12	0.11
query13	0.62	0.59	0.60
query14	2.66	2.69	2.82
query15	0.92	0.85	0.86
query16	0.37	0.37	0.38
query17	1.02	1.03	1.02
query18	0.22	0.20	0.20
query19	1.93	1.74	1.93
query20	0.02	0.02	0.02
query21	15.35	0.89	0.55
query22	0.75	1.25	0.62
query23	14.95	1.36	0.62
query24	7.67	1.13	0.68
query25	0.52	0.25	0.07
query26	0.71	0.16	0.13
query27	0.05	0.04	0.04
query28	9.35	0.87	0.43
query29	12.62	4.01	3.35
query30	0.24	0.09	0.06
query31	2.82	0.59	0.39
query32	3.23	0.54	0.48
query33	2.99	3.05	3.04
query34	15.60	5.14	4.45
query35	4.52	4.54	4.58
query36	0.66	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.56 s
Total hot run time: 30.6 s

@924060929 924060929 merged commit 0e5abe8 into apache:master Mar 12, 2025
26 of 27 checks passed
@924060929 924060929 deleted the fix_colocate_agg_join branch March 12, 2025 08:21
@morrySnow morrySnow added usercase Important user case type label p0_w dev/2.1.x dev/3.0.x and removed usercase Important user case type label p0_w dev/2.1.x dev/3.0.x labels Mar 12, 2025
github-actions bot pushed a commit that referenced this pull request Mar 12, 2025
@924060929
[fix](nereids) fix colocate agg + join compute wrong result (#48934)
73d81dc 
fix colocate agg + join compute wrong result, introduced by #41730
@github-actions github-actions bot mentioned this pull request Mar 12, 2025
Closed
github-actions bot pushed a commit that referenced this pull request May 27, 2025
@924060929
[fix](nereids) fix colocate agg + join compute wrong result (#48934)
0b649ae 
fix colocate agg + join compute wrong result, introduced by #41730
@github-actions github-actions bot mentioned this pull request May 27, 2025
Merged
dataroaring pushed a commit that referenced this pull request May 28, 2025
@github-actions @924060929
branch-3.0: [fix](nereids) fix colocate agg + join compute wrong result
f134411 
#48934 (#51296)

Cherry-picked from #48934

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
924060929 added a commit to 924060929/incubator-doris that referenced this pull request May 28, 2025
@924060929
[fix](nereids) fix colocate agg + join compute wrong result (apache#4…
1d24f74 
…8934)

fix colocate agg + join compute wrong result, introduced by apache#41730

(cherry picked from commit 0e5abe8)
@924060929 924060929 mentioned this pull request May 28, 2025
Merged
yiguolei pushed a commit that referenced this pull request May 28, 2025
@924060929
[fix](nereids) fix colocate agg + join compute wrong result (#48934) (#…
25f5d5f 
…51313)

cherry pick from #48934
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@924060929 @koarz
[fix](nereids) fix colocate agg + join compute wrong result (apache#4…
26d29e5 
…8934)

fix colocate agg + join compute wrong result, introduced by apache#41730
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.6-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@924060929 @hello-stephen @doris-robot @starocean999 @morrySnow @yiguolei @dataroaring