Skip to content

[fix](search) fix search result for non-tokenizer match #56914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
airborne12 merged 1 commit into from
Oct 14, 2025
Merged

[fix](search) fix search result for non-tokenizer match #56914

airborne12 merged 1 commit into from
Oct 14, 2025

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Oct 13, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Allow search() on MOW unique tables by accepting single-chain filter pipelines

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@airborne12
Copy link
Member Author

airborne12 commented Oct 13, 2025

run buildall

@Thearas
Copy link
Contributor

Thearas commented Oct 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12 airborne12 requested a review from Copilot October 13, 2025 10:43
Copilot AI reviewed Oct 13, 2025
View reviewed changes
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@doris-robot
Copy link

doris-robot commented Oct 13, 2025

TPC-DS: Total hot run time: 189248 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 242eeeff94555917c0bb010533e1522a78ab03e8, data reload: false

query1	1087	433	402	402
query2	6562	1705	1704	1704
query3	6749	217	225	217
query4	26753	23872	22984	22984
query5	5227	641	492	492
query6	355	234	213	213
query7	4651	485	295	295
query8	311	256	240	240
query9	8732	2562	2542	2542
query10	548	340	322	322
query11	15800	15036	14762	14762
query12	183	114	119	114
query13	1698	548	435	435
query14	11515	9262	9264	9262
query15	205	189	181	181
query16	7678	700	519	519
query17	1255	781	640	640
query18	2096	509	388	388
query19	244	230	204	204
query20	135	133	160	133
query21	279	148	127	127
query22	4880	4860	4672	4672
query23	34939	34125	33801	33801
query24	8413	2488	2503	2488
query25	616	541	473	473
query26	1489	277	176	176
query27	2806	517	376	376
query28	4827	2219	2173	2173
query29	848	627	491	491
query30	310	241	203	203
query31	953	834	766	766
query32	88	74	101	74
query33	1120	384	325	325
query34	908	892	531	531
query35	832	875	749	749
query36	1008	1109	900	900
query37	132	111	84	84
query38	3573	3688	3512	3512
query39	1526	1421	1451	1421
query40	220	135	117	117
query41	61	58	59	58
query42	118	110	116	110
query43	482	505	468	468
query44	1314	813	818	813
query45	183	180	171	171
query46	837	999	632	632
query47	1795	1825	1737	1737
query48	383	419	306	306
query49	769	474	403	403
query50	651	690	414	414
query51	3916	3901	3968	3901
query52	108	112	98	98
query53	233	265	200	200
query54	604	585	521	521
query55	86	83	82	82
query56	320	312	315	312
query57	1198	1220	1131	1131
query58	290	275	275	275
query59	2622	2655	2537	2537
query60	370	350	329	329
query61	190	188	181	181
query62	786	737	680	680
query63	235	194	191	191
query64	4578	1191	820	820
query65	4045	3982	3961	3961
query66	1124	438	332	332
query67	15455	15272	15064	15064
query68	9474	932	587	587
query69	497	329	296	296
query70	1440	1249	1303	1249
query71	495	339	333	333
query72	5361	4798	4798	4798
query73	681	573	364	364
query74	8918	8815	8647	8647
query75	4565	3281	2813	2813
query76	5288	1159	719	719
query77	1033	385	314	314
query78	9693	9803	8910	8910
query79	1389	843	584	584
query80	710	564	497	497
query81	497	261	231	231
query82	224	172	129	129
query83	294	277	250	250
query84	303	120	97	97
query85	874	469	421	421
query86	351	306	315	306
query87	3780	3668	3654	3654
query88	2824	2242	2204	2204
query89	381	325	307	307
query90	2095	213	218	213
query91	216	159	134	134
query92	84	66	66	66
query93	1108	1003	645	645
query94	692	430	336	336
query95	397	317	316	316
query96	484	569	286	286
query97	2957	2986	2830	2830
query98	228	208	216	208
query99	1437	1410	1269	1269
Total cold run time: 283074 ms
Total hot run time: 189248 ms

@doris-robot
Copy link

doris-robot commented Oct 13, 2025

ClickBench: Total hot run time: 30.62 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 242eeeff94555917c0bb010533e1522a78ab03e8, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.61	0.12	0.11
query5	0.27	0.27	0.25
query6	1.14	0.67	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.61	0.53	0.52
query10	0.59	0.58	0.58
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.63	0.62	0.62
query14	1.05	1.03	1.01
query15	0.85	0.85	0.87
query16	0.41	0.38	0.40
query17	1.02	1.06	1.09
query18	0.21	0.19	0.20
query19	1.96	1.80	1.87
query20	0.02	0.01	0.02
query21	15.45	0.89	0.61
query22	0.77	1.16	0.73
query23	14.85	1.37	0.66
query24	7.20	1.16	0.92
query25	0.46	0.13	0.10
query26	0.65	0.16	0.14
query27	0.07	0.06	0.05
query28	9.74	1.39	0.93
query29	12.62	4.04	3.31
query30	0.28	0.14	0.11
query31	2.84	0.59	0.39
query32	3.24	0.55	0.48
query33	3.03	3.08	3.08
query34	16.23	5.44	4.84
query35	4.95	4.90	4.91
query36	0.67	0.52	0.50
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.15	0.16
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.76 s
Total hot run time: 30.62 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 13, 2025

FE UT Coverage Report

Increment line coverage 76.47% (13/17) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 13, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.53% (17754/33796)
Line Coverage 37.72% (161279/427539)
Region Coverage 32.20% (123166/382504)
Branch Coverage 33.59% (54008/160797)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 13, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.13% (23554/33114)
Line Coverage 57.58% (245934/427129)
Region Coverage 52.73% (204253/387336)
Branch Coverage 54.54% (88137/161612)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 13, 2025

FE UT Coverage Report

Increment line coverage 76.47% (13/17) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 13, 2025

FE Regression Coverage Report

Increment line coverage 70.59% (12/17) 🎉
Increment coverage report
Complete coverage report

@airborne12
Copy link
Member Author

airborne12 commented Oct 14, 2025

run buildall

@airborne12 airborne12 requested a review from Copilot October 14, 2025 03:12
Copilot AI reviewed Oct 14, 2025
View reviewed changes
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@doris-robot
Copy link

doris-robot commented Oct 14, 2025

TPC-DS: Total hot run time: 190740 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b0874cc20f6bf1a88f5d63b2e40bbd6ba0626e94, data reload: false

query1	1042	459	419	419
query2	6558	1667	1698	1667
query3	6755	226	222	222
query4	26337	23391	23105	23105
query5	5053	643	505	505
query6	328	235	224	224
query7	4681	500	303	303
query8	312	259	256	256
query9	8747	2587	2567	2567
query10	547	368	294	294
query11	15911	15152	14964	14964
query12	190	120	117	117
query13	1696	541	425	425
query14	12664	9327	9416	9327
query15	226	198	180	180
query16	7763	671	510	510
query17	1636	775	598	598
query18	2058	495	357	357
query19	265	222	201	201
query20	157	141	134	134
query21	224	140	147	140
query22	4872	4666	4774	4666
query23	34852	33890	33718	33718
query24	9991	2491	2496	2491
query25	570	541	539	539
query26	1457	293	164	164
query27	3450	525	368	368
query28	4306	2227	2201	2201
query29	829	649	524	524
query30	307	241	210	210
query31	964	845	811	811
query32	88	76	70	70
query33	607	389	397	389
query34	1117	898	555	555
query35	833	866	810	810
query36	1002	1037	933	933
query37	128	122	98	98
query38	3622	3799	3647	3647
query39	1612	1487	1474	1474
query40	224	132	117	117
query41	63	58	60	58
query42	121	115	117	115
query43	485	494	452	452
query44	1320	836	821	821
query45	187	184	177	177
query46	838	1001	646	646
query47	1782	1828	1762	1762
query48	386	424	317	317
query49	766	477	402	402
query50	656	683	401	401
query51	3935	3906	3873	3873
query52	105	104	140	104
query53	233	264	195	195
query54	604	584	525	525
query55	91	83	85	83
query56	320	337	308	308
query57	1184	1192	1129	1129
query58	294	281	275	275
query59	2553	2629	2660	2629
query60	352	343	328	328
query61	169	155	159	155
query62	798	711	691	691
query63	235	200	191	191
query64	4420	1164	840	840
query65	4059	3959	3961	3959
query66	1114	469	356	356
query67	15471	15100	15186	15100
query68	7999	922	593	593
query69	497	325	292	292
query70	1374	1290	1340	1290
query71	459	352	318	318
query72	5712	4986	4791	4791
query73	588	584	356	356
query74	8941	9174	8661	8661
query75	3520	3336	2992	2992
query76	3387	1158	741	741
query77	729	411	311	311
query78	9705	9626	8922	8922
query79	2668	836	603	603
query80	702	556	511	511
query81	514	268	237	237
query82	459	167	139	139
query83	268	284	247	247
query84	322	115	99	99
query85	893	490	423	423
query86	390	324	303	303
query87	3835	3799	3704	3704
query88	3844	2253	2237	2237
query89	395	392	306	306
query90	1988	219	217	217
query91	180	163	141	141
query92	82	68	70	68
query93	2346	993	649	649
query94	747	460	336	336
query95	407	322	315	315
query96	489	559	286	286
query97	2925	2997	2926	2926
query98	251	216	213	213
query99	1391	1415	1321	1321
Total cold run time: 284529 ms
Total hot run time: 190740 ms

@doris-robot
Copy link

doris-robot commented Oct 14, 2025

ClickBench: Total hot run time: 30.82 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b0874cc20f6bf1a88f5d63b2e40bbd6ba0626e94, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.06	0.06
query3	0.25	0.09	0.08
query4	1.61	0.11	0.12
query5	0.27	0.27	0.25
query6	1.17	0.65	0.65
query7	0.03	0.03	0.03
query8	0.05	0.05	0.04
query9	0.67	0.53	0.54
query10	0.60	0.57	0.59
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.63	0.63	0.62
query14	1.04	1.04	1.05
query15	0.88	0.88	0.87
query16	0.40	0.39	0.40
query17	1.11	1.07	1.02
query18	0.22	0.20	0.20
query19	1.97	1.89	1.82
query20	0.02	0.02	0.01
query21	15.42	0.88	0.58
query22	0.77	1.19	0.77
query23	14.77	1.38	0.63
query24	6.76	2.18	0.96
query25	0.45	0.29	0.17
query26	0.66	0.16	0.15
query27	0.08	0.05	0.06
query28	10.18	1.36	0.93
query29	12.60	3.98	3.31
query30	0.28	0.13	0.12
query31	2.83	0.62	0.39
query32	3.26	0.56	0.47
query33	3.09	3.14	3.10
query34	16.32	5.42	4.82
query35	4.96	4.96	4.92
query36	0.68	0.52	0.50
query37	0.10	0.08	0.07
query38	0.07	0.05	0.05
query39	0.04	0.03	0.03
query40	0.18	0.14	0.14
query41	0.10	0.04	0.04
query42	0.05	0.04	0.03
query43	0.05	0.04	0.04
Total cold run time: 105.08 s
Total hot run time: 30.82 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.53% (17753/33796)
Line Coverage 37.72% (161270/427539)
Region Coverage 32.19% (123140/382504)
Branch Coverage 33.59% (54018/160797)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

FE UT Coverage Report

Increment line coverage 76.47% (13/17) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

FE Regression Coverage Report

Increment line coverage 70.59% (12/17) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.12% (23552/33114)
Line Coverage 57.56% (245845/427129)
Region Coverage 52.63% (203863/387336)
Branch Coverage 54.45% (87996/161612)

eldenmoon
eldenmoon approved these changes Oct 14, 2025
View reviewed changes
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 14, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 14, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 14, 2025

PR approved by anyone and no changes requested.

zzzxl1993
zzzxl1993 approved these changes Oct 14, 2025
View reviewed changes
Copy link
Contributor

@zzzxl1993 zzzxl1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit 807a87f into apache:master Oct 14, 2025
28 of 30 checks passed
@airborne12 airborne12 deleted the fix-search branch October 14, 2025 06:19
github-actions bot pushed a commit that referenced this pull request Oct 14, 2025
@airborne12
[fix](search) fix search result for non-tokenizer match (#56914)
cce885f 
Problem Summary:

Allow search() on MOW unique tables by accepting single-chain filter
pipelines
@github-actions github-actions bot mentioned this pull request Oct 14, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 16, 2025
@github-actions @airborne12
branch-4.0: [fix](search) fix search result for non-tokenizer match #…
1eccb47 
…56914 (#56938)

Cherry-picked from #56914

Co-authored-by: Jack <jiangkai@selectdb.com>
@yiguolei yiguolei mentioned this pull request Nov 5, 2025
Closed
airborne12 added a commit to airborne12/apache-doris that referenced this pull request Jan 7, 2026
@airborne12
[fix](search) fix search result for non-tokenizer match (apache#56914)
69dc5a3 
Problem Summary:

Allow search() on MOW unique tables by accepting single-chain filter
pipelines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@zzzxl1993 zzzxl1993 zzzxl1993 approved these changes

@eldenmoon eldenmoon eldenmoon approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@airborne12 @Thearas @doris-robot @hello-stephen @zzzxl1993 @eldenmoon @yiguolei