add GeneratedColumnPlaceholderInputStream by guo-shaoge · Pull Request #6796 · pingcap/tiflash

guo-shaoge · 2023-02-10T03:29:52Z

Signed-off-by: guo-shaoge shaoge1994@163.com

What problem does this PR solve?

Issue Number: close #6801 , ref pingcap/tidb#40663

Problem Summary: Check description in pingcap/tidb#41261

What is changed and how it works?

Add GeneratedColumnPlaceholderBlockInputStream.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

Signed-off-by: guo-shaoge <shaoge1994@163.com>

ti-chi-bot · 2023-02-10T03:29:54Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

SeaRise
windtalker

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: guo-shaoge <shaoge1994@163.com>

close pingcap#6660

close pingcap#6734

ref pingcap#6528

ref pingcap#6728

close pingcap#6767

close pingcap#6678

ref pingcap#6233

…p#6770) close pingcap#6765

ref pingcap#6678

ref pingcap#6233

close pingcap#4488, close pingcap#6393, close pingcap#6462

…age calculation for aggregation (pingcap#6708) ref pingcap#6528

Signed-off-by: guo-shaoge <shaoge1994@163.com>

dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.cpp

dbms/src/DataStreams/GeneratedColumnPlaceholderInputStream.h

AilinKid · 2023-02-13T04:34:43Z

dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.cpp

        analyzer = std::make_unique<DAGExpressionAnalyzer>(std::move(names_and_types), context);
        pipeline.streams.insert(pipeline.streams.end(), mock_table_scan_streams.begin(), mock_table_scan_streams.end());
    }
+    std::vector<std::pair<UInt64, DataTypePtr>> generated_column_infos;


should we do it also in handleTableScan?

You mean DAGQueryBlockInterpreter::handleTableScan?
DAGQueryBlockInterpreter::handleTableScan will call DAGStorageInterpter::execute(). so no need to handle.

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge · 2023-02-13T06:14:13Z

/merge

ti-chi-bot · 2023-02-13T06:14:14Z

@guo-shaoge: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-02-13T06:14:16Z

This pull request has been accepted and is ready to merge.

Details

Commit hash: ec1914d

ti-chi-bot · 2023-02-13T06:45:28Z

@AilinKid: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

guo-shaoge · 2023-02-13T07:20:07Z

/test integration-test

guo-shaoge · 2023-02-13T07:20:35Z

/run-integration-test

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2023-02-13T11:30:40Z

In response to a cherrypick label: new pull request created to branch release-6.1: #6812.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2023-02-13T11:31:21Z

In response to a cherrypick label: new pull request created to branch release-6.5: #6813.

close #6801, ref pingcap/tidb#40663

add GeneratedColumnPlaceholderInputStream

ade7a44

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge added 2 commits February 10, 2023 11:55

clear gtest change

b58875e

Signed-off-by: guo-shaoge <shaoge1994@163.com>

fix log

fddbabe

Signed-off-by: guo-shaoge <shaoge1994@163.com>

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 10, 2023

guo-shaoge mentioned this pull request Feb 10, 2023

planner: fix tiflash cannot find generated column pingcap/tidb#41261

Merged

12 tasks

add case

93a4c11

Signed-off-by: guo-shaoge <shaoge1994@163.com>

ti-chi-bot added do-not-merge/needs-triage-completed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 10, 2023

ywqzzy and others added 14 commits February 10, 2023 13:53

least/greatest function for string (pingcap#6662)

a6daf46

close pingcap#6660

fix tls ipv6 metric (pingcap#6736)

5b033a1

close pingcap#6734

Refine spiller (pingcap#6780)

6a1bee4

ref pingcap#6528

remove legacy code in RegionPersister (pingcap#6781)

9f8724e

ref pingcap#6728

fix RegionKVStoreTest (pingcap#6768)

3e8ce03

close pingcap#6767

Collect request unit for tiflash compute (pingcap#6649)

8981dfa

close pingcap#6678

Remove useless IDAsPathUpgrader (pingcap#6782)

c6493d4

ref pingcap#6233

fix tsan fail in unit test JoinExecutorTestRunner.MultiJoin (pingca…

76e725e

…p#6770) close pingcap#6765

Refine the hostname backlist in MetricsPrometheus (pingcap#6784)

c5ea678

ref pingcap#6678

Add more information to the error message in MPPTunnel (pingcap#6787)

0d25d4d

ref pingcap#6233

fix the issue that decimal divide not round. (pingcap#6471)

206b807

close pingcap#4488, close pingcap#6393, close pingcap#6462

Refine the spill threshold for aggregation/sort, refine the memory us…

ff2949f

…age calculation for aggregation (pingcap#6708) ref pingcap#6528

fix format

8de0167

Signed-off-by: guo-shaoge <shaoge1994@163.com>

Merge branch 'master' into gen_col_placeholder

8eb8ab7

SeaRise self-requested a review February 13, 2023 04:00

SeaRise reviewed Feb 13, 2023

View reviewed changes

AilinKid reviewed Feb 13, 2023

View reviewed changes

guo-shaoge requested review from AilinKid and SeaRise February 13, 2023 05:45

guo-shaoge added 2 commits February 13, 2023 13:47

add getGeneratedColumnInfos

2c24e58

Signed-off-by: guo-shaoge <shaoge1994@163.com>

del getGeneratedColumnInfos

ec1914d

Signed-off-by: guo-shaoge <shaoge1994@163.com>

SeaRise approved these changes Feb 13, 2023

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Feb 13, 2023

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 13, 2023

AilinKid approved these changes Feb 13, 2023

View reviewed changes

Merge branch 'master' into gen_col_placeholder

53dbdef

This was referenced Feb 13, 2023

planner: fix tiflash cannot find generated column (#41261) pingcap/tidb#41344

Merged

planner: fix tiflash cannot find generated column (#41261) pingcap/tidb#41345

Merged

ti-chi-bot merged commit e84ed48 into pingcap:master Feb 13, 2023

ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Feb 13, 2023

This is an automated cherry-pick of pingcap#6796

3834650

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Feb 13, 2023

add GeneratedColumnPlaceholderInputStream (#6796) #6812

Merged

12 tasks

ti-chi-bot mentioned this pull request Feb 13, 2023

add GeneratedColumnPlaceholderInputStream (#6796) #6813

Merged

12 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Feb 13, 2023

This is an automated cherry-pick of pingcap#6796

631499f

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot added a commit that referenced this pull request Mar 29, 2023

add GeneratedColumnPlaceholderInputStream (#6796) (#6812)

9b3e36d

close #6801, ref pingcap/tidb#40663

ti-chi-bot added a commit that referenced this pull request Apr 4, 2023

add GeneratedColumnPlaceholderInputStream (#6796) (#6813)

16223d8

close #6801, ref pingcap/tidb#40663

Conversation

guo-shaoge commented Feb 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

Uh oh!

ti-chi-bot commented Feb 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AilinKid Feb 13, 2023

Choose a reason for hiding this comment

Uh oh!

guo-shaoge Feb 13, 2023

Choose a reason for hiding this comment

Uh oh!

guo-shaoge commented Feb 13, 2023

Uh oh!

ti-chi-bot commented Feb 13, 2023

Uh oh!

ti-chi-bot commented Feb 13, 2023

Uh oh!

ti-chi-bot commented Feb 13, 2023

Uh oh!

guo-shaoge commented Feb 13, 2023

Uh oh!

guo-shaoge commented Feb 13, 2023

Uh oh!

ti-chi-bot commented Feb 13, 2023

Uh oh!

ti-chi-bot commented Feb 13, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

guo-shaoge commented Feb 10, 2023 •

edited

Loading

ti-chi-bot commented Feb 10, 2023 •

edited

Loading