Skip to content

[fix](search) add restriction for search function #56706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
airborne12 merged 3 commits into from
Oct 5, 2025
Merged

[fix](search) add restriction for search function #56706

airborne12 merged 3 commits into from
Oct 5, 2025

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Oct 5, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #56139

Problem Summary:
This PR adds restrictions for the search() function to ensure it can only be used in WHERE clauses on single-table OLAP scans. The implementation includes validation rules that reject search() usage in other contexts like SELECT projections, GROUP BY clauses, HAVING clauses, and multi-table scenarios.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Oct 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12 airborne12 requested a review from Copilot October 5, 2025 04:30
@airborne12
Copy link
Member Author

airborne12 commented Oct 5, 2025

run buildall

Copilot AI reviewed Oct 5, 2025
View reviewed changes
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds restrictions for the search() function to ensure it can only be used in WHERE clauses on single-table OLAP scans. The implementation includes validation rules that reject search() usage in other contexts like SELECT projections, GROUP BY clauses, HAVING clauses, and multi-table scenarios.

  • Added a new analysis rule CheckSearchUsage that validates search() function usage patterns
  • Created comprehensive unit tests covering valid and invalid search() usage scenarios
  • Integrated the validation rule into the analyzer pipeline before other optimizations

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
CheckSearchUsage.java Main validation rule that checks search() usage restrictions recursively through the plan tree
CheckSearchUsageTest.java Comprehensive unit tests covering valid WHERE clause usage and invalid usage in projections, GROUP BY, joins, etc.
RuleType.java Added CHECK_SEARCH_USAGE rule type enum
Analyzer.java Integrated CheckSearchUsage rule into analyzer pipeline
test_search_usage_restrictions.out Test output file for regression tests

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@airborne12
Copy link
Member Author

airborne12 commented Oct 5, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 5, 2025

ClickBench: Total hot run time: 30.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 77ed78c0fe0a23f34c50214b77034d9b42d66b3c, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.25	0.08	0.09
query4	1.60	0.12	0.12
query5	0.29	0.27	0.25
query6	1.18	0.64	0.64
query7	0.03	0.02	0.02
query8	0.06	0.05	0.04
query9	0.63	0.53	0.52
query10	0.57	0.58	0.57
query11	0.17	0.12	0.12
query12	0.16	0.12	0.12
query13	0.64	0.62	0.62
query14	1.05	1.02	1.03
query15	0.88	0.84	0.85
query16	0.40	0.41	0.39
query17	1.05	1.05	1.05
query18	0.22	0.20	0.19
query19	1.92	1.88	1.85
query20	0.02	0.02	0.01
query21	15.51	0.95	0.57
query22	0.75	1.18	0.66
query23	14.97	1.40	0.62
query24	6.47	1.93	0.54
query25	0.52	0.19	0.19
query26	0.58	0.16	0.13
query27	0.06	0.06	0.05
query28	9.54	1.38	0.92
query29	12.55	3.94	3.26
query30	0.28	0.15	0.12
query31	2.82	0.59	0.40
query32	3.24	0.57	0.49
query33	3.20	3.13	3.13
query34	16.15	5.48	4.85
query35	4.90	4.90	4.92
query36	0.71	0.51	0.51
query37	0.10	0.07	0.08
query38	0.06	0.06	0.05
query39	0.03	0.04	0.03
query40	0.19	0.16	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.05	0.03	0.03
Total cold run time: 104.08 s
Total hot run time: 30.25 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 5, 2025

FE UT Coverage Report

Increment line coverage 90.00% (45/50) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 5, 2025

FE Regression Coverage Report

Increment line coverage 90.00% (45/50) 🎉
Increment coverage report
Complete coverage report

1 similar comment
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 5, 2025

FE Regression Coverage Report

Increment line coverage 90.00% (45/50) 🎉
Increment coverage report
Complete coverage report

HappenLee
HappenLee approved these changes Oct 5, 2025
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2025

PR approved by at least one committer and no changes requested.

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 5, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2025

PR approved by anyone and no changes requested.

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2025

PR approved by anyone and no changes requested.

@airborne12 airborne12 merged commit 155cd67 into apache:master Oct 5, 2025
32 checks passed
@airborne12 airborne12 deleted the fix-search1 branch October 5, 2025 13:09
github-actions bot pushed a commit that referenced this pull request Oct 5, 2025
@airborne12
[fix](search) add restriction for search function (#56706)
47e5574 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #56139 

Problem Summary:
This PR adds restrictions for the search() function to ensure it can
only be used in WHERE clauses on single-table OLAP scans. The
implementation includes validation rules that reject search() usage in
other contexts like SELECT projections, GROUP BY clauses, HAVING
clauses, and multi-table scenarios.
@github-actions github-actions bot mentioned this pull request Oct 5, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 6, 2025
@github-actions @airborne12
branch-4.0: [fix](search) add restriction for search function #56706 (#…
6e075ee 
…56707)

Cherry-picked from #56706

Co-authored-by: Jack <jiangkai@selectdb.com>
airborne12 added a commit to airborne12/apache-doris that referenced this pull request Jan 7, 2026
@airborne12
[fix](search) add restriction for search function (apache#56706)
a033a24 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: apache#56139 

Problem Summary:
This PR adds restrictions for the search() function to ensure it can
only be used in WHERE clauses on single-table OLAP scans. The
implementation includes validation rules that reject search() usage in
other contexts like SELECT projections, GROUP BY clauses, HAVING
clauses, and multi-table scenarios.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@HappenLee HappenLee HappenLee approved these changes

@zclllyybb zclllyybb zclllyybb approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@airborne12 @Thearas @doris-robot @hello-stephen @HappenLee @zclllyybb @yiguolei