Skip to content

executor: reuse chunk for GetMatchedRowsAndPtrs calls (#48090)#48106

Merged
ti-chi-bot[bot] merged 1 commit intopingcap:release-7.5from
ti-chi-bot:cherry-pick-48090-to-release-7.5
Nov 7, 2023
Merged

executor: reuse chunk for GetMatchedRowsAndPtrs calls (#48090)#48106
ti-chi-bot[bot] merged 1 commit intopingcap:release-7.5from
ti-chi-bot:cherry-pick-48090-to-release-7.5

Conversation

@ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #48090

What problem does this PR solve?

Issue Number: close #48082
Problem Summary:

What is changed and how it works?

Reuse chunk for GetMatchedRowsAndPtrs.
Add a new benchmark case.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    BenchmarkHashJoinExec
Old:
 go test -benchmem -run=^$ -bench ^BenchmarkHashJoinExec$ github.com/pingcap/tidb/pkg/executor
goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/pkg/executor
cpu: 13th Gen Intel(R) Core(TM) i9-13900KF
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                   2         625585277 ns/op        2428636920 B/op   512892 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                    19          64306968 ns/op        278193261 B/op    112877 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                      2         510499472 ns/op        3491117756 B/op  2213412 allocs/op
BenchmarkHashJoinExec/(rows:5,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                             26020             48058 ns/op           87853 B/op        934 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                         79          16758415 ns/op        15902526 B/op     512736 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                           92          15112195 ns/op        12524965 B/op     112666 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                                    9         123780841 ns/op        894195201 B/op   1714292 allocs/op


New:
go test -benchmem -run=^$ -bench ^BenchmarkHashJoinExec$ github.com/pingcap/tidb/pkg/executor                      
goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/pkg/executor
cpu: 13th Gen Intel(R) Core(TM) i9-13900KF
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                   2         627044000 ns/op        2428678160 B/op   512971 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                    18          63483060 ns/op        278192079 B/op    112888 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_var_string(5)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                      2         502814122 ns/op        3639970448 B/op  1216829 allocs/op
BenchmarkHashJoinExec/(rows:5,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                             28047             48197 ns/op           87847 B/op        934 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0_1],_disk:false)-32                         70          16853512 ns/op        15902144 B/op     512739 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)_double],_concurency:4,_joinKeyIdx:_[0],_disk:false)-32                           82          14872368 ns/op        12524746 B/op     112660 allocs/op
BenchmarkHashJoinExec/(rows:100000,_cols:[bigint(20)],_concurency:4,_joinKeyIdx:_[0],_disk:true)-32                                   22          53258487 ns/op        40282174 B/op    1113179 allocs/op

Profile for the new case: left is the master, right is this pr.
image

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-7.5 This PR is cherry-picked to release-7.5 from a source PR. labels Oct 30, 2023
Copy link
Member

@YangKeao YangKeao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 31, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Nov 2, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lcwangchao, YangKeao

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [YangKeao,lcwangchao]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot added the cherry-pick-approved Cherry pick PR approved by release team. label Nov 2, 2023
@ti-chi-bot ti-chi-bot bot added lgtm and removed do-not-merge/cherry-pick-not-approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 2, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Nov 2, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-31 03:53:10.88318905 +0000 UTC m=+2925188.470299196: ☑️ agreed by YangKeao.
  • 2023-11-02 09:12:54.568026364 +0000 UTC m=+3117172.155136510: ☑️ agreed by lcwangchao.

@lcwangchao
Copy link
Collaborator

/retest

@wshwsh12
Copy link
Contributor

wshwsh12 commented Nov 7, 2023

/run-unit-test

@wshwsh12
Copy link
Contributor

wshwsh12 commented Nov 7, 2023

/retest

@ti-chi-bot ti-chi-bot bot merged commit 817b4fc into pingcap:release-7.5 Nov 7, 2023
@ti-chi-bot ti-chi-bot deleted the cherry-pick-48090-to-release-7.5 branch April 15, 2025 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved cherry-pick-approved Cherry pick PR approved by release team. lgtm release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-7.5 This PR is cherry-picked to release-7.5 from a source PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants