Skip to content

Conversation

@yujun777
Copy link
Contributor

@yujun777 yujun777 commented Jul 3, 2025

What problem does this PR solve?

this PR optimize adjusting slot nullables, includes:

  1. Apply AdjustNullable rules both at the end of analyzed phase and rewrite phase. And at rewrite phase, if met a slot adjust from not-nullable to nullable, it will throw an exception.
  2. For LogicalApply, fix its computting output slots, and fix it in AdjustNullable rule;
  3. Opt NullableAggregateFunction's nullable, include:
    • for agg, its constructor will adjust its NullableAggregateFunction nullable;
    • for project and one row relation, will change NullableAggregateFunction to nullable;
    • for having and sort, use rule AdjustAggregateNullableForEmptySet to update their agg function nullable

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jul 3, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yujun777
Copy link
Contributor Author

yujun777 commented Jul 3, 2025

run buildall

@yujun777 yujun777 force-pushed the fix-agg-nullable branch from cf234b6 to d608b08 Compare July 3, 2025 23:59
@yujun777
Copy link
Contributor Author

yujun777 commented Jul 4, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33956 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d608b0870e829ad77e5e493073e03564fb1a10de, data reload: false

------ Round 1 ----------------------------------
q1	17583	5200	5031	5031
q2	1930	291	176	176
q3	10293	1304	739	739
q4	10214	983	511	511
q5	7510	2397	2315	2315
q6	175	156	124	124
q7	869	750	617	617
q8	9314	1294	1053	1053
q9	6912	5133	5138	5133
q10	6914	2371	1948	1948
q11	491	294	275	275
q12	343	350	218	218
q13	17769	3652	3107	3107
q14	237	217	205	205
q15	564	478	496	478
q16	424	418	381	381
q17	605	848	361	361
q18	7417	7190	7178	7178
q19	1496	941	537	537
q20	330	336	216	216
q21	3829	3158	2389	2389
q22	1019	1016	964	964
Total cold run time: 106238 ms
Total hot run time: 33956 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5170	5060	5083	5060
q2	239	329	224	224
q3	2127	2688	2283	2283
q4	1342	1804	1363	1363
q5	4169	4507	4548	4507
q6	220	176	129	129
q7	1998	1993	1804	1804
q8	2612	2665	2603	2603
q9	7354	7363	7357	7357
q10	3120	3288	2943	2943
q11	571	514	510	510
q12	701	980	651	651
q13	3478	3946	3554	3554
q14	285	304	296	296
q15	515	475	478	475
q16	461	504	425	425
q17	1153	1590	1427	1427
q18	7814	7737	7574	7574
q19	799	782	885	782
q20	2016	1957	1802	1802
q21	4673	4282	4323	4282
q22	1087	1044	1021	1021
Total cold run time: 51904 ms
Total hot run time: 51072 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185302 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d608b0870e829ad77e5e493073e03564fb1a10de, data reload: false

query1	1011	383	378	378
query2	6563	1668	1701	1668
query3	6745	216	217	216
query4	26554	23885	23671	23671
query5	4382	594	443	443
query6	312	225	202	202
query7	4623	488	292	292
query8	269	227	225	225
query9	8603	2631	2649	2631
query10	476	345	296	296
query11	15435	15577	15256	15256
query12	170	105	101	101
query13	1629	514	394	394
query14	9637	5719	5768	5719
query15	206	183	170	170
query16	7638	621	485	485
query17	1169	692	569	569
query18	2005	391	294	294
query19	217	192	162	162
query20	124	119	114	114
query21	214	123	105	105
query22	4047	4005	3956	3956
query23	34019	33014	32970	32970
query24	8425	2332	2388	2332
query25	534	464	400	400
query26	1221	268	150	150
query27	2759	510	348	348
query28	4293	2126	2109	2109
query29	711	569	433	433
query30	279	220	198	198
query31	939	844	759	759
query32	72	66	67	66
query33	557	387	322	322
query34	797	855	522	522
query35	788	801	752	752
query36	966	980	904	904
query37	112	103	84	84
query38	4112	4123	4092	4092
query39	1474	1569	1400	1400
query40	226	128	147	128
query41	55	53	54	53
query42	126	113	106	106
query43	502	502	488	488
query44	1356	831	817	817
query45	175	168	159	159
query46	840	997	622	622
query47	1725	1811	1703	1703
query48	378	417	320	320
query49	742	467	412	412
query50	686	710	415	415
query51	4193	4117	4064	4064
query52	118	114	101	101
query53	238	264	189	189
query54	578	568	499	499
query55	84	84	90	84
query56	336	310	305	305
query57	1154	1197	1120	1120
query58	264	261	267	261
query59	2597	2665	2526	2526
query60	322	306	310	306
query61	127	120	120	120
query62	839	724	686	686
query63	223	194	192	192
query64	4221	994	714	714
query65	4226	4165	4170	4165
query66	1053	406	337	337
query67	15638	15370	15163	15163
query68	7861	883	532	532
query69	471	307	280	280
query70	1165	1142	1073	1073
query71	440	318	299	299
query72	5377	4762	4743	4743
query73	651	601	352	352
query74	9227	8838	8639	8639
query75	3387	3177	2694	2694
query76	3202	1148	733	733
query77	667	389	284	284
query78	9916	10148	9335	9335
query79	1222	887	575	575
query80	620	519	459	459
query81	481	252	223	223
query82	212	131	95	95
query83	266	257	237	237
query84	241	105	79	79
query85	755	361	312	312
query86	333	378	270	270
query87	4386	4487	4305	4305
query88	2952	2294	2297	2294
query89	377	320	283	283
query90	1960	215	211	211
query91	138	143	108	108
query92	77	66	57	57
query93	1122	945	595	595
query94	668	409	296	296
query95	382	296	285	285
query96	501	565	280	280
query97	2676	2746	2633	2633
query98	235	205	203	203
query99	1315	1442	1295	1295
Total cold run time: 270392 ms
Total hot run time: 185302 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.33 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d608b0870e829ad77e5e493073e03564fb1a10de, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.08
query4	1.61	0.11	0.11
query5	0.44	0.42	0.41
query6	1.17	0.67	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.61	0.54	0.53
query10	0.57	0.58	0.56
query11	0.16	0.11	0.11
query12	0.16	0.11	0.11
query13	0.63	0.61	0.61
query14	0.80	0.82	0.82
query15	0.90	0.88	0.88
query16	0.39	0.38	0.40
query17	1.11	1.08	1.08
query18	0.23	0.22	0.22
query19	1.93	1.80	1.84
query20	0.01	0.02	0.01
query21	15.42	0.91	0.55
query22	0.75	1.24	0.91
query23	14.73	1.39	0.62
query24	7.55	1.17	0.29
query25	0.29	0.10	0.06
query26	0.53	0.17	0.15
query27	0.07	0.05	0.05
query28	8.81	0.94	0.45
query29	12.56	3.94	3.27
query30	0.25	0.09	0.07
query31	2.84	0.60	0.39
query32	3.23	0.57	0.47
query33	3.06	3.12	3.19
query34	16.06	5.50	4.78
query35	4.89	4.90	4.86
query36	0.71	0.50	0.49
query37	0.09	0.06	0.08
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.04
Total cold run time: 103.38 s
Total hot run time: 29.33 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (19/19) 🎉
Increment coverage report
Complete coverage report

@yujun777
Copy link
Contributor Author

yujun777 commented Jul 7, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33859 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a39b5162d0fb3b9a997572ce3d26e9348d635833, data reload: false

------ Round 1 ----------------------------------
q1	17684	5295	5087	5087
q2	1925	299	193	193
q3	10615	1297	706	706
q4	10290	1016	526	526
q5	8432	2406	2351	2351
q6	180	161	130	130
q7	875	749	594	594
q8	9300	1359	1001	1001
q9	6884	5122	5162	5122
q10	6876	2395	1954	1954
q11	477	289	281	281
q12	337	350	213	213
q13	17762	3692	3124	3124
q14	230	235	214	214
q15	532	484	471	471
q16	421	439	377	377
q17	580	870	365	365
q18	7730	7090	7215	7090
q19	1229	944	551	551
q20	317	353	221	221
q21	3750	2621	2321	2321
q22	1082	1006	967	967
Total cold run time: 107508 ms
Total hot run time: 33859 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5197	5122	5111	5111
q2	239	318	216	216
q3	2163	2648	2310	2310
q4	1389	1755	1343	1343
q5	4194	4596	4468	4468
q6	235	176	126	126
q7	2007	1943	1831	1831
q8	2708	2581	2509	2509
q9	7346	7311	7339	7311
q10	3116	3326	2933	2933
q11	571	509	508	508
q12	710	782	681	681
q13	3605	4006	3408	3408
q14	298	348	283	283
q15	521	480	475	475
q16	449	492	436	436
q17	1174	1583	1401	1401
q18	7910	7821	7018	7018
q19	759	760	859	760
q20	1903	1984	1845	1845
q21	4633	4427	4223	4223
q22	1095	1036	997	997
Total cold run time: 52222 ms
Total hot run time: 50193 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184853 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a39b5162d0fb3b9a997572ce3d26e9348d635833, data reload: false

query1	997	385	386	385
query2	6517	1756	1746	1746
query3	6757	225	212	212
query4	26328	23675	22986	22986
query5	4345	599	457	457
query6	308	212	206	206
query7	4628	497	287	287
query8	263	222	213	213
query9	8591	2624	2626	2624
query10	466	345	270	270
query11	15343	14966	14904	14904
query12	156	108	102	102
query13	1659	538	419	419
query14	8594	5797	5741	5741
query15	197	188	169	169
query16	7379	592	465	465
query17	1183	723	565	565
query18	1956	387	295	295
query19	189	184	163	163
query20	122	112	110	110
query21	205	123	117	117
query22	4143	4089	3993	3993
query23	33974	32847	33093	32847
query24	8467	2387	2371	2371
query25	545	472	400	400
query26	1220	268	149	149
query27	2761	519	344	344
query28	4382	2143	2121	2121
query29	754	567	428	428
query30	285	220	193	193
query31	903	843	753	753
query32	71	64	62	62
query33	568	367	306	306
query34	796	836	528	528
query35	781	826	771	771
query36	986	987	880	880
query37	117	99	78	78
query38	4138	4173	4093	4093
query39	1472	1429	1423	1423
query40	209	115	105	105
query41	54	54	51	51
query42	120	111	110	110
query43	494	530	466	466
query44	1338	820	811	811
query45	177	169	161	161
query46	843	1043	626	626
query47	1715	1760	1713	1713
query48	374	434	309	309
query49	735	494	386	386
query50	663	697	421	421
query51	4154	4226	4070	4070
query52	110	106	93	93
query53	224	252	184	184
query54	570	575	506	506
query55	81	84	85	84
query56	297	305	292	292
query57	1150	1184	1117	1117
query58	265	259	254	254
query59	2615	2724	2587	2587
query60	320	326	314	314
query61	128	120	117	117
query62	808	698	666	666
query63	212	187	189	187
query64	4325	1024	750	750
query65	4275	4182	4226	4182
query66	1161	430	336	336
query67	15676	15653	15419	15419
query68	7859	915	535	535
query69	475	315	285	285
query70	1193	1117	1168	1117
query71	468	334	301	301
query72	5839	4799	5009	4799
query73	706	707	360	360
query74	9060	9185	8638	8638
query75	3768	3199	2721	2721
query76	3506	1146	800	800
query77	788	402	279	279
query78	10093	10298	9401	9401
query79	2614	850	591	591
query80	629	505	457	457
query81	498	257	224	224
query82	452	124	99	99
query83	252	248	223	223
query84	249	112	87	87
query85	787	354	314	314
query86	394	298	299	298
query87	4344	4420	4346	4346
query88	3780	2305	2273	2273
query89	381	318	280	280
query90	1832	211	206	206
query91	139	187	115	115
query92	78	65	56	56
query93	2443	967	587	587
query94	657	408	278	278
query95	377	300	276	276
query96	490	580	280	280
query97	2719	2755	2663	2663
query98	229	204	226	204
query99	1378	1374	1267	1267
Total cold run time: 274116 ms
Total hot run time: 184853 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.77 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a39b5162d0fb3b9a997572ce3d26e9348d635833, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.08	0.08
query4	1.62	0.11	0.11
query5	0.42	0.42	0.42
query6	1.17	0.66	0.67
query7	0.02	0.01	0.03
query8	0.05	0.04	0.03
query9	0.60	0.53	0.53
query10	0.58	0.57	0.57
query11	0.16	0.11	0.12
query12	0.16	0.12	0.12
query13	0.63	0.62	0.61
query14	0.80	0.82	0.82
query15	0.91	0.89	0.87
query16	0.41	0.41	0.42
query17	1.06	1.07	1.08
query18	0.22	0.22	0.22
query19	1.99	1.93	1.93
query20	0.02	0.01	0.02
query21	15.38	0.93	0.54
query22	0.76	1.17	0.69
query23	14.89	1.39	0.69
query24	7.59	1.06	0.82
query25	0.49	0.23	0.08
query26	0.58	0.16	0.15
query27	0.08	0.06	0.06
query28	9.35	0.92	0.46
query29	12.58	4.00	3.28
query30	0.24	0.08	0.06
query31	2.85	0.61	0.38
query32	3.23	0.56	0.48
query33	3.09	3.11	3.08
query34	16.20	5.35	4.75
query35	4.85	4.85	4.77
query36	0.71	0.50	0.49
query37	0.09	0.07	0.07
query38	0.05	0.05	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.56 s
Total hot run time: 29.77 s

@yujun777 yujun777 force-pushed the fix-agg-nullable branch from a39b516 to d063dea Compare July 8, 2025 01:53
@yujun777
Copy link
Contributor Author

yujun777 commented Jul 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33529 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d063dead9958a08893563b689c33f7a341c5abb2, data reload: false

------ Round 1 ----------------------------------
q1	17669	5279	5105	5105
q2	1951	314	183	183
q3	10285	1287	729	729
q4	10223	1016	543	543
q5	7478	2418	2364	2364
q6	193	164	132	132
q7	900	750	608	608
q8	9321	1329	1135	1135
q9	6721	5139	5114	5114
q10	6881	2375	2009	2009
q11	490	292	265	265
q12	353	356	225	225
q13	17804	3712	3086	3086
q14	223	222	216	216
q15	540	479	481	479
q16	418	423	379	379
q17	583	894	355	355
q18	7550	7257	7138	7138
q19	1210	946	561	561
q20	329	367	218	218
q21	3743	3227	2393	2393
q22	358	322	292	292
Total cold run time: 105223 ms
Total hot run time: 33529 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5183	5151	5156	5151
q2	253	340	233	233
q3	2194	2717	2300	2300
q4	1424	1800	1364	1364
q5	4241	4326	4624	4326
q6	234	174	128	128
q7	2108	1995	1879	1879
q8	2713	2600	2555	2555
q9	7369	7296	7290	7290
q10	3175	3344	2861	2861
q11	630	525	502	502
q12	704	802	632	632
q13	3536	3960	3426	3426
q14	280	301	278	278
q15	534	480	468	468
q16	445	496	440	440
q17	1255	1656	1350	1350
q18	8124	7691	7622	7622
q19	830	839	1019	839
q20	2046	2098	1979	1979
q21	5040	4645	4743	4645
q22	622	586	570	570
Total cold run time: 52940 ms
Total hot run time: 50838 ms

@yujun777
Copy link
Contributor Author

yujun777 commented Jul 8, 2025

run buildall

@yujun777 yujun777 marked this pull request as ready for review July 8, 2025 02:14
@doris-robot
Copy link

TPC-H: Total hot run time: 33409 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f5d34cc0a423ba0006335e20951f539bc88054f9, data reload: false

------ Round 1 ----------------------------------
q1	17609	5172	5041	5041
q2	1921	281	181	181
q3	10412	1284	724	724
q4	10238	1007	572	572
q5	7874	2368	2317	2317
q6	176	155	129	129
q7	875	741	606	606
q8	9323	1238	1103	1103
q9	6821	5294	5189	5189
q10	6882	2356	1960	1960
q11	472	294	281	281
q12	343	339	215	215
q13	17763	3679	3120	3120
q14	231	237	211	211
q15	560	475	470	470
q16	428	436	371	371
q17	572	865	352	352
q18	7487	7213	7138	7138
q19	1264	919	548	548
q20	317	362	222	222
q21	3809	3146	2365	2365
q22	356	327	294	294
Total cold run time: 105733 ms
Total hot run time: 33409 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5094	5053	5062	5053
q2	234	320	215	215
q3	2164	2677	2338	2338
q4	1385	1791	1350	1350
q5	4223	4302	4607	4302
q6	218	172	128	128
q7	2129	1972	1836	1836
q8	2725	2554	2553	2553
q9	7314	7243	7107	7107
q10	3114	3386	2887	2887
q11	587	518	493	493
q12	739	791	680	680
q13	3692	4090	3536	3536
q14	273	312	276	276
q15	512	478	487	478
q16	462	473	434	434
q17	1136	1685	1416	1416
q18	8016	7832	7774	7774
q19	838	895	1145	895
q20	2118	2101	1970	1970
q21	4857	4355	4307	4307
q22	635	589	535	535
Total cold run time: 52465 ms
Total hot run time: 50563 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186207 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f5d34cc0a423ba0006335e20951f539bc88054f9, data reload: false

query1	1001	405	383	383
query2	6509	1743	1779	1743
query3	6739	215	216	215
query4	26292	23866	23587	23587
query5	4385	615	455	455
query6	310	219	202	202
query7	4629	489	294	294
query8	291	226	218	218
query9	8611	2621	2610	2610
query10	497	335	276	276
query11	15792	15195	14826	14826
query12	167	109	117	109
query13	1634	510	394	394
query14	9396	5964	5937	5937
query15	201	217	181	181
query16	7598	429	255	255
query17	1340	709	557	557
query18	2020	393	294	294
query19	194	185	174	174
query20	120	114	115	114
query21	208	122	105	105
query22	4122	4146	4036	4036
query23	33839	33032	33265	33032
query24	8465	2421	2427	2421
query25	512	469	399	399
query26	1220	261	153	153
query27	2760	517	364	364
query28	4276	2120	2090	2090
query29	712	550	436	436
query30	288	223	191	191
query31	961	854	767	767
query32	75	64	61	61
query33	558	343	297	297
query34	803	832	523	523
query35	613	647	598	598
query36	929	974	901	901
query37	106	101	74	74
query38	4123	4161	4183	4161
query39	1452	1418	1434	1418
query40	214	115	104	104
query41	64	65	63	63
query42	125	109	108	108
query43	511	510	494	494
query44	1345	821	814	814
query45	177	179	172	172
query46	862	1038	636	636
query47	1747	1819	1744	1744
query48	378	408	302	302
query49	727	481	391	391
query50	644	688	444	444
query51	4195	4265	4128	4128
query52	106	101	102	101
query53	226	261	185	185
query54	578	569	501	501
query55	81	81	78	78
query56	303	290	267	267
query57	1209	1185	1125	1125
query58	279	255	264	255
query59	2635	2730	2577	2577
query60	338	317	305	305
query61	123	116	119	116
query62	803	707	657	657
query63	224	188	192	188
query64	4331	1166	808	808
query65	4266	4153	4190	4153
query66	1058	423	326	326
query67	15778	15478	15568	15478
query68	6364	886	548	548
query69	503	312	267	267
query70	1173	1130	1100	1100
query71	392	308	295	295
query72	5107	4743	4659	4659
query73	645	586	351	351
query74	8987	9154	8739	8739
query75	3195	3165	2679	2679
query76	3180	1136	738	738
query77	501	387	298	298
query78	11159	11188	10132	10132
query79	999	815	584	584
query80	576	507	495	495
query81	466	254	224	224
query82	316	125	100	100
query83	253	244	226	226
query84	255	100	83	83
query85	738	353	324	324
query86	328	295	294	294
query87	4411	4418	4329	4329
query88	2863	2236	2257	2236
query89	377	325	277	277
query90	1631	209	203	203
query91	132	137	111	111
query92	61	61	56	56
query93	1060	946	585	585
query94	613	309	193	193
query95	363	298	311	298
query96	483	555	282	282
query97	2706	2801	2681	2681
query98	228	216	205	205
query99	1309	1371	1244	1244
Total cold run time: 268449 ms
Total hot run time: 186207 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.94 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f5d34cc0a423ba0006335e20951f539bc88054f9, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.24	0.08	0.07
query4	1.63	0.11	0.12
query5	0.44	0.44	0.43
query6	1.17	0.67	0.66
query7	0.03	0.01	0.02
query8	0.05	0.03	0.04
query9	0.60	0.54	0.51
query10	0.57	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	0.78	0.82	0.82
query15	0.88	0.88	0.84
query16	0.39	0.38	0.39
query17	1.05	1.10	1.05
query18	0.22	0.22	0.21
query19	1.94	1.84	1.81
query20	0.02	0.01	0.01
query21	15.40	0.87	0.54
query22	0.75	1.16	0.66
query23	14.98	1.39	0.63
query24	6.72	2.09	1.21
query25	0.50	0.18	0.08
query26	0.58	0.16	0.15
query27	0.08	0.05	0.05
query28	10.05	0.89	0.42
query29	12.66	3.96	3.30
query30	0.26	0.09	0.07
query31	2.84	0.58	0.38
query32	3.26	0.55	0.50
query33	3.05	3.11	3.09
query34	16.09	5.42	4.76
query35	4.86	4.91	4.88
query36	0.68	0.52	0.48
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.15	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.31 s
Total hot run time: 29.94 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 92.59% (50/54) 🎉
Increment coverage report
Complete coverage report

@yujun777
Copy link
Contributor Author

yujun777 commented Jul 8, 2025

run buildall

1 similar comment
@yujun777
Copy link
Contributor Author

yujun777 commented Jul 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33276 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fae8b53ff2ad801267b5a4c11881e07715efd8b7, data reload: false

------ Round 1 ----------------------------------
q1	17627	5183	5072	5072
q2	1939	274	185	185
q3	10320	1292	711	711
q4	10239	1026	525	525
q5	7540	2466	2347	2347
q6	174	160	129	129
q7	889	749	598	598
q8	9307	1284	1077	1077
q9	6966	5181	5108	5108
q10	6867	2398	1962	1962
q11	468	291	266	266
q12	345	341	219	219
q13	17769	3642	3155	3155
q14	219	230	230	230
q15	552	479	489	479
q16	425	428	382	382
q17	575	863	350	350
q18	7620	7194	7086	7086
q19	1219	944	532	532
q20	331	336	216	216
q21	3693	3187	2350	2350
q22	359	308	297	297
Total cold run time: 105443 ms
Total hot run time: 33276 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5102	5093	5066	5066
q2	241	328	224	224
q3	2208	2677	2304	2304
q4	1369	1784	1322	1322
q5	4229	4293	4671	4293
q6	212	168	131	131
q7	2103	2033	1858	1858
q8	2603	2672	2588	2588
q9	7391	7271	7296	7271
q10	3197	3305	2995	2995
q11	643	512	500	500
q12	691	751	609	609
q13	3437	4027	3498	3498
q14	267	277	295	277
q15	515	492	480	480
q16	474	498	451	451
q17	1187	1565	1386	1386
q18	7999	7840	7523	7523
q19	797	801	955	801
q20	2215	2128	1879	1879
q21	4884	4361	4281	4281
q22	621	594	523	523
Total cold run time: 52385 ms
Total hot run time: 50260 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185912 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fae8b53ff2ad801267b5a4c11881e07715efd8b7, data reload: false

query1	997	403	381	381
query2	6528	1716	1749	1716
query3	6736	208	210	208
query4	26548	23504	23223	23223
query5	4382	580	446	446
query6	310	209	207	207
query7	4622	494	287	287
query8	291	217	216	216
query9	8597	2590	2614	2590
query10	484	331	268	268
query11	15455	15412	14772	14772
query12	167	109	102	102
query13	1664	518	423	423
query14	8619	5889	5889	5889
query15	201	193	180	180
query16	7457	440	268	268
query17	1326	716	584	584
query18	2003	410	305	305
query19	198	199	166	166
query20	128	117	116	116
query21	212	128	107	107
query22	4091	4084	3941	3941
query23	34111	33377	33191	33191
query24	8476	2378	2388	2378
query25	522	463	397	397
query26	1228	264	145	145
query27	2751	508	350	350
query28	4294	2129	2087	2087
query29	748	556	428	428
query30	287	220	192	192
query31	919	832	767	767
query32	68	67	58	58
query33	561	346	283	283
query34	792	849	519	519
query35	598	636	562	562
query36	930	966	868	868
query37	108	97	74	74
query38	4115	4176	4060	4060
query39	1493	1412	1424	1412
query40	214	121	105	105
query41	58	54	54	54
query42	123	106	110	106
query43	485	549	483	483
query44	1296	797	804	797
query45	180	172	171	171
query46	826	1033	634	634
query47	1748	1808	1681	1681
query48	384	408	316	316
query49	714	457	394	394
query50	635	672	421	421
query51	4083	4292	4205	4205
query52	118	105	107	105
query53	222	263	187	187
query54	570	565	514	514
query55	85	80	80	80
query56	310	295	272	272
query57	1174	1173	1105	1105
query58	261	256	256	256
query59	2752	2762	2613	2613
query60	335	321	304	304
query61	141	125	118	118
query62	826	731	657	657
query63	225	181	184	181
query64	4395	1222	806	806
query65	4245	4192	4174	4174
query66	1150	409	309	309
query67	15879	15589	15406	15406
query68	8960	883	535	535
query69	515	340	280	280
query70	1200	1104	1146	1104
query71	490	325	301	301
query72	5426	4720	4844	4720
query73	706	561	344	344
query74	8936	9237	8972	8972
query75	4090	3151	2688	2688
query76	3699	1141	723	723
query77	793	388	347	347
query78	11001	11180	10345	10345
query79	1842	827	589	589
query80	641	507	449	449
query81	482	256	219	219
query82	328	126	100	100
query83	290	254	232	232
query84	287	103	86	86
query85	753	348	311	311
query86	385	293	282	282
query87	4453	4476	4394	4394
query88	2810	2272	2228	2228
query89	379	316	281	281
query90	2068	207	200	200
query91	136	137	111	111
query92	75	56	53	53
query93	1180	943	592	592
query94	678	318	195	195
query95	369	287	285	285
query96	486	558	273	273
query97	2720	2787	2576	2576
query98	230	213	206	206
query99	1458	1387	1271	1271
Total cold run time: 274317 ms
Total hot run time: 185912 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fae8b53ff2ad801267b5a4c11881e07715efd8b7, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.24	0.09	0.08
query4	1.61	0.11	0.12
query5	0.46	0.43	0.40
query6	1.14	0.65	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.53	0.52
query10	0.58	0.57	0.56
query11	0.15	0.10	0.11
query12	0.15	0.12	0.12
query13	0.62	0.61	0.62
query14	0.80	0.81	0.81
query15	0.88	0.87	0.86
query16	0.37	0.40	0.39
query17	1.04	1.06	1.05
query18	0.22	0.21	0.21
query19	1.99	1.80	1.86
query20	0.02	0.01	0.02
query21	15.39	0.88	0.57
query22	0.75	1.17	0.62
query23	15.02	1.38	0.60
query24	6.73	0.98	0.62
query25	0.56	0.19	0.06
query26	0.60	0.17	0.13
query27	0.06	0.06	0.05
query28	9.47	0.92	0.45
query29	12.53	3.96	3.26
query30	0.25	0.10	0.07
query31	2.83	0.61	0.38
query32	3.24	0.57	0.47
query33	3.16	3.08	3.20
query34	16.10	5.37	4.77
query35	4.80	4.81	4.81
query36	0.71	0.50	0.48
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.14	0.14
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.04	0.04
Total cold run time: 103.75 s
Total hot run time: 29.18 s

@yujun777
Copy link
Contributor Author

yujun777 commented Jul 8, 2025

run buildall

1 similar comment
@yujun777
Copy link
Contributor Author

yujun777 commented Jul 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33008 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6d5a84bfc0342a385d2cdf8510fb37e8353b5f4, data reload: false

------ Round 1 ----------------------------------
q1	17623	5180	5091	5091
q2	1942	282	184	184
q3	10373	1253	703	703
q4	10247	994	521	521
q5	8056	2402	2347	2347
q6	181	165	131	131
q7	883	747	601	601
q8	9316	1293	1027	1027
q9	6685	5174	5018	5018
q10	6879	2362	1978	1978
q11	495	302	261	261
q12	347	357	222	222
q13	17765	3679	3072	3072
q14	223	235	219	219
q15	557	482	474	474
q16	417	424	379	379
q17	588	868	365	365
q18	7412	7245	6974	6974
q19	2053	929	535	535
q20	327	346	215	215
q21	3965	3237	2400	2400
q22	366	317	291	291
Total cold run time: 106700 ms
Total hot run time: 33008 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5243	5157	5104	5104
q2	250	326	218	218
q3	2213	2708	2310	2310
q4	1416	1795	1338	1338
q5	4187	4395	4562	4395
q6	208	165	130	130
q7	2069	1970	1792	1792
q8	2658	2546	2503	2503
q9	7218	7146	7137	7137
q10	3179	3319	2916	2916
q11	568	508	481	481
q12	688	786	611	611
q13	3582	3959	3458	3458
q14	303	302	290	290
q15	508	485	495	485
q16	457	492	438	438
q17	1224	1533	1393	1393
q18	7835	7762	7441	7441
q19	839	774	912	774
q20	1906	1945	1875	1875
q21	4841	4320	4292	4292
q22	597	609	517	517
Total cold run time: 51989 ms
Total hot run time: 49898 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184780 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6d5a84bfc0342a385d2cdf8510fb37e8353b5f4, data reload: false

query1	1001	372	377	372
query2	6534	1748	1773	1748
query3	6751	219	216	216
query4	25949	23588	23210	23210
query5	4355	570	442	442
query6	308	224	202	202
query7	4625	484	283	283
query8	266	223	218	218
query9	8649	2583	2591	2583
query10	495	323	257	257
query11	15682	14996	14783	14783
query12	167	105	107	105
query13	1652	540	417	417
query14	9514	5798	5693	5693
query15	202	186	176	176
query16	7662	632	470	470
query17	1199	732	599	599
query18	2037	418	297	297
query19	197	205	159	159
query20	120	112	111	111
query21	206	123	107	107
query22	4186	4147	4108	4108
query23	33930	32881	32933	32881
query24	8461	2374	2357	2357
query25	518	457	386	386
query26	1215	261	144	144
query27	2733	497	351	351
query28	4226	2108	2077	2077
query29	691	560	422	422
query30	282	215	193	193
query31	934	865	751	751
query32	72	62	63	62
query33	546	368	323	323
query34	792	846	532	532
query35	770	795	732	732
query36	913	971	856	856
query37	105	99	74	74
query38	4115	4097	4078	4078
query39	1461	1422	1495	1422
query40	203	117	102	102
query41	55	53	49	49
query42	122	114	103	103
query43	518	526	464	464
query44	1343	834	823	823
query45	171	167	162	162
query46	838	997	623	623
query47	1732	1814	1678	1678
query48	390	406	310	310
query49	763	497	396	396
query50	625	686	398	398
query51	4072	4135	4160	4135
query52	106	103	92	92
query53	215	250	188	188
query54	554	560	495	495
query55	85	77	81	77
query56	308	288	292	288
query57	1177	1176	1123	1123
query58	258	257	247	247
query59	2670	2765	2682	2682
query60	337	330	293	293
query61	122	124	124	124
query62	820	732	655	655
query63	221	187	185	185
query64	4228	1000	653	653
query65	4259	4232	4173	4173
query66	1057	424	314	314
query67	15651	15562	15417	15417
query68	8537	890	521	521
query69	455	303	255	255
query70	1223	1150	1135	1135
query71	444	325	297	297
query72	5692	4650	4690	4650
query73	688	587	345	345
query74	9165	9047	9046	9046
query75	4009	3183	2692	2692
query76	3699	1134	711	711
query77	778	375	281	281
query78	9990	10156	9395	9395
query79	2107	820	581	581
query80	658	502	461	461
query81	463	268	221	221
query82	442	123	93	93
query83	285	248	299	248
query84	298	97	87	87
query85	775	351	304	304
query86	325	286	284	284
query87	4526	4447	4308	4308
query88	2879	2261	2264	2261
query89	477	311	287	287
query90	1966	204	209	204
query91	153	141	105	105
query92	74	56	54	54
query93	1286	942	596	596
query94	667	403	269	269
query95	358	279	280	279
query96	488	565	279	279
query97	2679	2785	2631	2631
query98	225	202	207	202
query99	1437	1402	1252	1252
Total cold run time: 273669 ms
Total hot run time: 184780 ms

@yujun777
Copy link
Contributor Author

run buildall

@yujun777
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 86.43% (172/199) 🎉
Increment coverage report
Complete coverage report

@yujun777
Copy link
Contributor Author

run performance

@doris-robot
Copy link

TPC-H: Total hot run time: 33938 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0eed73a2c6e6a808578a59b8ce113ae204fe3ae9, data reload: false

------ Round 1 ----------------------------------
q1	17595	5261	5072	5072
q2	1931	283	181	181
q3	10311	1263	706	706
q4	10221	1006	515	515
q5	7519	2350	2346	2346
q6	172	161	129	129
q7	881	737	591	591
q8	9313	1315	1082	1082
q9	6833	5153	5113	5113
q10	6897	2386	1963	1963
q11	474	291	282	282
q12	337	343	217	217
q13	17764	3663	3017	3017
q14	246	224	221	221
q15	530	484	472	472
q16	422	426	379	379
q17	606	859	387	387
q18	7589	7213	7154	7154
q19	1217	939	563	563
q20	344	355	237	237
q21	3609	3152	2318	2318
q22	1056	1036	993	993
Total cold run time: 105867 ms
Total hot run time: 33938 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5561	5118	5103	5103
q2	240	328	219	219
q3	2207	2631	2325	2325
q4	1360	1785	1342	1342
q5	4147	4568	4574	4568
q6	215	173	129	129
q7	2025	1978	1846	1846
q8	2660	2659	2472	2472
q9	7359	7211	7374	7211
q10	3129	3311	2892	2892
q11	573	519	509	509
q12	700	772	647	647
q13	3623	3957	3317	3317
q14	306	307	286	286
q15	538	491	512	491
q16	457	492	437	437
q17	1176	1492	1363	1363
q18	8081	7644	7667	7644
q19	825	856	1078	856
q20	2030	2120	1837	1837
q21	4716	4306	4192	4192
q22	1090	1064	1012	1012
Total cold run time: 53018 ms
Total hot run time: 50698 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186286 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0eed73a2c6e6a808578a59b8ce113ae204fe3ae9, data reload: false

query1	972	384	401	384
query2	6523	1691	1680	1680
query3	6736	219	213	213
query4	26577	23704	23288	23288
query5	4387	569	435	435
query6	301	216	199	199
query7	4641	499	293	293
query8	266	222	210	210
query9	8598	2611	2634	2611
query10	450	325	283	283
query11	15196	14981	14799	14799
query12	152	107	114	107
query13	1679	543	399	399
query14	8494	5721	5644	5644
query15	195	194	173	173
query16	7133	647	426	426
query17	915	694	556	556
query18	1976	394	298	298
query19	203	178	161	161
query20	116	116	109	109
query21	213	126	107	107
query22	4330	4356	4113	4113
query23	34173	32965	33036	32965
query24	8447	2397	2351	2351
query25	547	460	398	398
query26	1229	268	145	145
query27	2780	501	344	344
query28	4316	2160	2139	2139
query29	749	547	431	431
query30	282	221	192	192
query31	915	827	737	737
query32	71	63	65	63
query33	563	350	308	308
query34	789	837	505	505
query35	796	830	724	724
query36	1001	1001	921	921
query37	102	94	77	77
query38	4055	4164	4033	4033
query39	1506	1431	1421	1421
query40	216	114	102	102
query41	60	56	54	54
query42	121	105	112	105
query43	522	503	468	468
query44	1306	824	808	808
query45	178	165	164	164
query46	835	989	615	615
query47	1810	1822	1736	1736
query48	403	418	305	305
query49	747	466	401	401
query50	625	684	408	408
query51	5455	5526	5416	5416
query52	109	101	93	93
query53	220	247	185	185
query54	562	573	499	499
query55	85	78	81	78
query56	335	306	278	278
query57	1210	1218	1137	1137
query58	285	277	256	256
query59	2596	2662	2596	2596
query60	312	327	299	299
query61	131	128	128	128
query62	789	734	680	680
query63	233	196	191	191
query64	4414	1087	756	756
query65	4247	4201	4166	4166
query66	1155	433	346	346
query67	16088	15796	15357	15357
query68	7798	892	536	536
query69	481	311	280	280
query70	1159	1116	1128	1116
query71	531	328	293	293
query72	5578	4720	4696	4696
query73	648	580	349	349
query74	9081	9100	8735	8735
query75	3858	3178	2762	2762
query76	3553	1165	728	728
query77	794	382	280	280
query78	10097	10156	9433	9433
query79	3719	793	561	561
query80	659	524	516	516
query81	479	257	215	215
query82	504	137	96	96
query83	283	260	244	244
query84	293	111	79	79
query85	779	354	318	318
query86	328	294	282	282
query87	4502	4471	4396	4396
query88	3529	2272	2289	2272
query89	440	316	285	285
query90	1932	209	218	209
query91	146	143	111	111
query92	74	58	56	56
query93	2882	946	584	584
query94	672	416	313	313
query95	379	287	295	287
query96	492	574	286	286
query97	2717	2752	2631	2631
query98	217	210	206	206
query99	1435	1389	1262	1262
Total cold run time: 277462 ms
Total hot run time: 186286 ms

@yujun777
Copy link
Contributor Author

run external

@doris-robot
Copy link

ClickBench: Total hot run time: 32.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0eed73a2c6e6a808578a59b8ce113ae204fe3ae9, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.05
query3	0.24	0.07	0.07
query4	1.62	0.12	0.11
query5	0.43	0.45	0.42
query6	1.21	0.67	0.68
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.58	0.53	0.51
query10	0.57	0.57	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.62	0.61
query14	0.79	0.82	0.82
query15	0.89	0.87	0.88
query16	0.39	0.40	0.41
query17	1.05	1.05	1.06
query18	0.22	0.21	0.21
query19	1.91	1.86	1.82
query20	0.01	0.01	0.01
query21	15.40	0.87	0.54
query22	0.77	1.13	0.64
query23	15.01	1.34	0.62
query24	6.38	2.18	0.34
query25	0.29	0.19	0.24
query26	0.53	0.16	0.15
query27	0.07	0.06	0.06
query28	9.79	0.91	0.43
query29	12.56	4.00	3.33
query30	3.04	2.97	2.95
query31	2.83	0.59	0.39
query32	3.23	0.55	0.48
query33	3.09	3.07	3.09
query34	16.03	5.37	4.80
query35	4.88	4.88	4.87
query36	0.69	0.51	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.18	0.15	0.13
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.1 s
Total hot run time: 32.13 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@yujun777 yujun777 changed the title [fix](nereids) fix agg nullable [opt](nereids) adjust slot nullable Jul 18, 2025
@yujun777 yujun777 changed the title [opt](nereids) adjust slot nullable [opt](nereids) opt for adjustting slot nullable and add exception for changing slot nullable Jul 18, 2025
@morrySnow morrySnow merged commit fa36652 into apache:master Jul 18, 2025
28 of 30 checks passed
morrySnow pushed a commit that referenced this pull request Jul 18, 2025
…xception for changing slot nullable #52748 (#53533)

cherry pick from #52748
morrySnow pushed a commit that referenced this pull request Jul 24, 2025
### What problem does this PR solve?

for sql 
select avg(distinct a), sum(distinct b) from t group by c
if "a" is not nullable, then "avg" is not nullable,

AvgDistinctToSumDivCount will rewrite "avg" to
"sum(distinct a) / count(distinct a)", but the divide "/" is nullable,

then AdjustNullable will throw exception for this changed nullable
(introduced by #52748):
"AdjustNullable convert slot avg(..)#10 from not-nullable to nullable. You can disable check by set fe_debug = false."

need add a nonNullable function to make the rewritten expression not-nullable.
github-actions bot pushed a commit that referenced this pull request Jul 24, 2025
### What problem does this PR solve?

for sql 
select avg(distinct a), sum(distinct b) from t group by c
if "a" is not nullable, then "avg" is not nullable,

AvgDistinctToSumDivCount will rewrite "avg" to
"sum(distinct a) / count(distinct a)", but the divide "/" is nullable,

then AdjustNullable will throw exception for this changed nullable
(introduced by #52748):
"AdjustNullable convert slot avg(..)#10 from not-nullable to nullable. You can disable check by set fe_debug = false."

need add a nonNullable function to make the rewritten expression not-nullable.
github-actions bot pushed a commit that referenced this pull request Jul 24, 2025
### What problem does this PR solve?

for sql 
select avg(distinct a), sum(distinct b) from t group by c
if "a" is not nullable, then "avg" is not nullable,

AvgDistinctToSumDivCount will rewrite "avg" to
"sum(distinct a) / count(distinct a)", but the divide "/" is nullable,

then AdjustNullable will throw exception for this changed nullable
(introduced by #52748):
"AdjustNullable convert slot avg(..)#10 from not-nullable to nullable. You can disable check by set fe_debug = false."

need add a nonNullable function to make the rewritten expression not-nullable.
yujun777 added a commit to yujun777/doris that referenced this pull request Jul 24, 2025
…#53787)

### What problem does this PR solve?

for sql 
select avg(distinct a), sum(distinct b) from t group by c
if "a" is not nullable, then "avg" is not nullable,

AvgDistinctToSumDivCount will rewrite "avg" to
"sum(distinct a) / count(distinct a)", but the divide "/" is nullable,

then AdjustNullable will throw exception for this changed nullable
(introduced by apache#52748):
"AdjustNullable convert slot avg(..)apache#10 from not-nullable to nullable. You can disable check by set fe_debug = false."

need add a nonNullable function to make the rewritten expression not-nullable.
w41ter pushed a commit to w41ter/incubator-doris that referenced this pull request Jul 30, 2025
…#53787)

### What problem does this PR solve?

for sql 
select avg(distinct a), sum(distinct b) from t group by c
if "a" is not nullable, then "avg" is not nullable,

AvgDistinctToSumDivCount will rewrite "avg" to
"sum(distinct a) / count(distinct a)", but the divide "/" is nullable,

then AdjustNullable will throw exception for this changed nullable
(introduced by apache#52748):
"AdjustNullable convert slot avg(..)apache#10 from not-nullable to nullable. You can disable check by set fe_debug = false."

need add a nonNullable function to make the rewritten expression not-nullable.
zhiqiang-hhhh pushed a commit to zhiqiang-hhhh/doris that referenced this pull request Aug 4, 2025
…#53787)

### What problem does this PR solve?

for sql 
select avg(distinct a), sum(distinct b) from t group by c
if "a" is not nullable, then "avg" is not nullable,

AvgDistinctToSumDivCount will rewrite "avg" to
"sum(distinct a) / count(distinct a)", but the divide "/" is nullable,

then AdjustNullable will throw exception for this changed nullable
(introduced by apache#52748):
"AdjustNullable convert slot avg(..)apache#10 from not-nullable to nullable. You can disable check by set fe_debug = false."

need add a nonNullable function to make the rewritten expression not-nullable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants