Skip to content

branch-3.0: [fix](inverted index) fix query error #50860 #50868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
May 14, 2025
Merged

branch-3.0: [fix](inverted index) fix query error #50860 #50868

dataroaring merged 1 commit into from
May 14, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented May 14, 2025

Cherry-picked from #50860

@csun5285
[fix](inverted index) fix query error (#50860)
7c58ba3 
‌When the IN predicate in an expression contains parameters generated
through CAST operations, utilizing inverted indexes may produce
erroneous query results.
@github-actions github-actions bot requested a review from dataroaring as a code owner May 14, 2025 01:54
@hello-stephen
Copy link
Contributor

hello-stephen commented May 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this May 14, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented May 14, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 14, 2025

TPC-H: Total hot run time: 40311 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c58ba380c539c4c0c07f1a851e410daff24afe1, data reload: false

------ Round 1 ----------------------------------
q1	17569	7422	6592	6592
q2	2078	179	161	161
q3	10581	1116	1241	1116
q4	10567	735	758	735
q5	7759	2845	2789	2789
q6	230	134	131	131
q7	982	615	619	615
q8	9355	1981	1992	1981
q9	6593	6472	6485	6472
q10	7007	2267	2301	2267
q11	462	261	262	261
q12	401	219	213	213
q13	17785	2962	3011	2962
q14	235	206	213	206
q15	514	471	473	471
q16	665	588	579	579
q17	980	634	584	584
q18	7370	6632	6645	6632
q19	1392	1068	1060	1060
q20	469	204	207	204
q21	4032	3325	3272	3272
q22	1127	1022	1008	1008
Total cold run time: 108153 ms
Total hot run time: 40311 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6581	6563	6995	6563
q2	328	236	229	229
q3	2960	2844	2936	2844
q4	2066	1786	1816	1786
q5	5729	5758	5756	5756
q6	218	131	133	131
q7	2255	1813	1756	1756
q8	3402	3534	3519	3519
q9	8914	8898	8866	8866
q10	3576	3554	3541	3541
q11	605	488	506	488
q12	838	643	599	599
q13	8951	3147	3205	3147
q14	311	266	280	266
q15	515	473	479	473
q16	696	653	649	649
q17	1841	1623	1608	1608
q18	8269	7836	7862	7836
q19	1725	1600	1508	1508
q20	2090	1852	1861	1852
q21	5630	5366	5322	5322
q22	1165	1067	1047	1047
Total cold run time: 68665 ms
Total hot run time: 59786 ms

@doris-robot
Copy link

doris-robot commented May 14, 2025

TPC-DS: Total hot run time: 199040 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c58ba380c539c4c0c07f1a851e410daff24afe1, data reload: false

query1	1303	907	919	907
query2	6237	2063	2059	2059
query3	10958	4352	4521	4352
query4	61300	29406	23617	23617
query5	5225	474	458	458
query6	399	173	195	173
query7	5487	310	305	305
query8	331	225	224	224
query9	8614	2629	2606	2606
query10	454	282	267	267
query11	17531	15470	15726	15470
query12	162	108	106	106
query13	1480	451	445	445
query14	9939	7612	7315	7315
query15	205	179	183	179
query16	7164	479	483	479
query17	1161	581	591	581
query18	1862	330	328	328
query19	215	167	161	161
query20	125	113	111	111
query21	205	104	103	103
query22	4951	4483	4786	4483
query23	34801	34064	34411	34064
query24	6210	2959	2979	2959
query25	537	421	437	421
query26	653	178	168	168
query27	1912	374	363	363
query28	4495	2481	2438	2438
query29	752	495	440	440
query30	248	165	167	165
query31	989	837	870	837
query32	70	60	62	60
query33	408	286	298	286
query34	932	514	532	514
query35	844	768	748	748
query36	1092	952	957	952
query37	122	73	72	72
query38	4123	4017	3972	3972
query39	1534	1448	1517	1448
query40	201	101	104	101
query41	52	50	48	48
query42	121	98	106	98
query43	541	488	494	488
query44	1185	832	824	824
query45	190	169	174	169
query46	1135	735	742	735
query47	1999	1926	1897	1897
query48	476	377	400	377
query49	739	395	400	395
query50	897	436	436	436
query51	7336	7154	7239	7154
query52	101	93	93	93
query53	276	198	194	194
query54	591	480	483	480
query55	79	77	78	77
query56	269	255	236	236
query57	1266	1179	1171	1171
query58	226	205	213	205
query59	3469	3124	3095	3095
query60	293	255	257	255
query61	111	148	109	109
query62	779	715	686	686
query63	213	186	188	186
query64	1371	673	685	673
query65	3316	3198	3247	3198
query66	704	297	292	292
query67	16136	15852	15789	15789
query68	4222	578	583	578
query69	439	273	275	273
query70	1152	1105	1123	1105
query71	356	265	266	265
query72	6388	4089	3957	3957
query73	749	352	349	349
query74	10286	9306	9319	9306
query75	3407	2711	2661	2661
query76	1985	1204	1012	1012
query77	479	276	269	269
query78	10750	9622	9655	9622
query79	1811	605	585	585
query80	1358	456	432	432
query81	532	243	240	240
query82	1286	94	89	89
query83	273	148	143	143
query84	285	83	84	83
query85	1022	305	292	292
query86	387	296	268	268
query87	4429	4318	4286	4286
query88	4033	2403	2383	2383
query89	423	294	299	294
query90	2001	189	189	189
query91	176	147	151	147
query92	59	49	52	49
query93	2398	552	552	552
query94	794	306	305	305
query95	370	266	267	266
query96	623	283	285	283
query97	3353	3176	3170	3170
query98	222	205	194	194
query99	2096	1299	1296	1296
Total cold run time: 317246 ms
Total hot run time: 199040 ms

@doris-robot
Copy link

doris-robot commented May 14, 2025

ClickBench: Total hot run time: 31.88 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7c58ba380c539c4c0c07f1a851e410daff24afe1, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.02
query3	0.24	0.07	0.06
query4	1.61	0.10	0.11
query5	0.53	0.53	0.52
query6	1.13	0.73	0.72
query7	0.03	0.02	0.01
query8	0.06	0.03	0.02
query9	0.56	0.52	0.52
query10	0.55	0.58	0.57
query11	0.15	0.10	0.11
query12	0.14	0.11	0.10
query13	0.61	0.59	0.60
query14	2.74	2.72	2.84
query15	0.89	0.81	0.83
query16	0.40	0.39	0.40
query17	1.07	1.05	1.02
query18	0.24	0.21	0.22
query19	1.98	1.89	2.05
query20	0.01	0.02	0.01
query21	15.36	0.59	0.59
query22	2.57	2.04	1.58
query23	16.93	0.91	0.81
query24	3.68	1.12	0.70
query25	0.21	0.20	0.11
query26	0.52	0.14	0.13
query27	0.04	0.05	0.04
query28	10.19	0.54	0.46
query29	12.59	3.28	3.26
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.22	0.47	0.46
query33	2.98	3.00	3.05
query34	17.11	4.53	4.54
query35	4.63	4.59	4.58
query36	0.66	0.47	0.47
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.14	0.13
query41	0.09	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.33 s
Total hot run time: 31.88 s

@doris-robot
Copy link

doris-robot commented May 14, 2025

BE UT Coverage Report

Increment line coverage 50.00% (2/4) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.06% (10831/26381)
Line Coverage 31.87% (92443/290093)
Region Coverage 30.95% (47728/154197)
Branch Coverage 27.44% (24446/89100)

dataroaring
dataroaring approved these changes May 14, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit a9abe89 into branch-3.0 May 14, 2025
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-50860-branch-3.0 branch May 14, 2025 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @dataroaring @csun5285