Skip to content

[fix](nereids)relase physical plan in Profile.releaseMemory() #57257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Oct 24, 2025
Merged

[fix](nereids)relase physical plan in Profile.releaseMemory() #57257

yiguolei merged 1 commit into from
Oct 24, 2025

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Oct 23, 2025

What problem does this PR solve?

The profile holds a physical plan object. In profile.releaseMemory(), the physicalPlan needs to be set to null to release it, preventing the groupExpression in Memo from failing to be released in a timely manner.
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@englefly
Copy link
Contributor Author

englefly commented Oct 23, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 23, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 23, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 23, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Oct 23, 2025

ClickBench: Total hot run time: 27.74 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 923c17d174c940bd49e05656bf66af245db5aa35, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.06	0.06
query3	0.26	0.09	0.09
query4	1.60	0.12	0.12
query5	0.28	0.26	0.24
query6	1.17	0.67	0.65
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.63	0.54	0.53
query10	0.59	0.60	0.58
query11	0.17	0.12	0.12
query12	0.16	0.13	0.13
query13	0.64	0.61	0.60
query14	1.02	1.05	1.03
query15	0.87	0.85	0.88
query16	0.39	0.41	0.39
query17	1.07	1.04	1.03
query18	0.22	0.21	0.21
query19	1.99	1.80	1.84
query20	0.02	0.02	0.01
query21	15.48	0.20	0.13
query22	5.10	0.08	0.05
query23	15.65	0.29	0.11
query24	3.04	1.30	0.31
query25	0.08	0.06	0.05
query26	0.16	0.14	0.15
query27	0.06	0.06	0.05
query28	4.15	1.15	0.95
query29	12.63	4.03	3.38
query30	0.30	0.17	0.12
query31	2.83	0.62	0.38
query32	3.23	0.56	0.46
query33	3.09	3.04	3.15
query34	15.93	5.19	4.58
query35	4.56	4.58	4.63
query36	0.67	0.52	0.49
query37	0.11	0.07	0.06
query38	0.07	0.05	0.04
query39	0.04	0.03	0.04
query40	0.19	0.16	0.14
query41	0.08	0.04	0.04
query42	0.04	0.03	0.04
query43	0.04	0.04	0.03
Total cold run time: 98.85 s
Total hot run time: 27.74 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 23, 2025

FE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

@englefly
Copy link
Contributor Author

englefly commented Oct 24, 2025

run p0

zhiqiang-hhhh
zhiqiang-hhhh reviewed Oct 24, 2025
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java
this.changedSessionVarCache = "";
this.physicalPlan = null;
this.rowsProducedMap = null;
this.physicalRelations = null;
Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does planNodeMap need to be removed?

@yiguolei yiguolei merged commit ca8c12d into apache:master Oct 24, 2025
36 of 39 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 24, 2025
@englefly
[fix](nereids)relase physical plan in Profile.releaseMemory() (#57257)
dfd2251 
### What problem does this PR solve?
The profile holds a physical plan object. In profile.releaseMemory(),
the physicalPlan needs to be set to null to release it, preventing the
groupExpression in Memo from failing to be released in a timely manner.
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Oct 24, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Oct 24, 2025
@englefly
[fix](nereids)relase physical plan in Profile.releaseMemory() (#57257)
cd981c6 
### What problem does this PR solve?
The profile holds a physical plan object. In profile.releaseMemory(),
the physicalPlan needs to be set to null to release it, preventing the
groupExpression in Memo from failing to be released in a timely manner.
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Oct 24, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 25, 2025
@github-actions @englefly
branch-4.0: [fix](nereids)relase physical plan in Profile.releaseMemo…
ba3c0f3 
…ry() #57257 (#57317)

Cherry-picked from #57257

Co-authored-by: minghong <zhouminghong@selectdb.com>
morrySnow pushed a commit that referenced this pull request Oct 28, 2025
@github-actions @englefly
branch-3.1: [fix](nereids)relase physical plan in Profile.releaseMemo…
311b78e 
…ry() #57257 (#57316)

Cherry-picked from #57257

Co-authored-by: minghong <zhouminghong@selectdb.com>
dwdwqfwe pushed a commit to dwdwqfwe/doris that referenced this pull request Oct 31, 2025
@englefly @dwdwqfwe
[fix](nereids)relase physical plan in Profile.releaseMemory() (apache…
1eeca6c 
…#57257)

### What problem does this PR solve?
The profile holds a physical plan object. In profile.releaseMemory(),
the physicalPlan needs to be set to null to release it, preventing the
groupExpression in Memo from failing to be released in a timely manner.
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@yiguolei yiguolei mentioned this pull request Nov 5, 2025
Closed
Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Nov 12, 2025
@selectdb-robot @englefly
branch-selectdb-doris-3.0: [fix](nereids)relase physical plan in Prof…
d55cd3a 
…ile.releaseMemory() apache#57257 (apache#5499)

cherry-picks from apache#57257
Related to apache#5307

Co-authored-by: minghong <zhouminghong@selectdb.com>
@wm1581066 wm1581066 added the p0_c label Jan 12, 2026
@englefly englefly mentioned this pull request Jan 12, 2026
Open
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

@starocean999 starocean999 starocean999 approved these changes

+1 more reviewer

@zhiqiang-hhhh zhiqiang-hhhh zhiqiang-hhhh approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.3-merged dev/4.0.1-merged p0_c reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@englefly @doris-robot @hello-stephen @yiguolei @starocean999 @zhiqiang-hhhh @wm1581066 @morrySnow