Skip to content

Interpreter: Print BlockInputStream#4911

Merged
ti-chi-bot merged 55 commits intopingcap:masterfrom
ywqdev:print_inputstream
May 25, 2022
Merged

Interpreter: Print BlockInputStream#4911
ti-chi-bot merged 55 commits intopingcap:masterfrom
ywqdev:print_inputstream

Conversation

@ywqzzy
Copy link
Contributor

@ywqzzy ywqzzy commented May 17, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #4609

Problem Summary:

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

ywqzzy added 30 commits April 27, 2022 15:40
@ywqzzy
init.
8e17622
@ywqzzy
fix.
c8fd369
@ywqzzy
Merge branch 'master' of https://github.com/pingcap/tiflash into run_…
4a29a27 
…interpreter_test
@ywqzzy
fix restoreConcurrency.
491dafd
@ywqzzy
some tests
fbd66bb
@ywqzzy
format.
b5e5614
@ywqzzy
fix build fail.
99830e3
@ywqzzy
format, more tests.
347bf8a
@ywqzzy
clean
a7d4602
@ywqzzy
refactor
9a7b988
@ywqzzy
getColumnsFromTableScan utils and clean code.
781489e
@ywqzzy
refactor gencolumn related functions.
283639e
@ywqzzy
remove useless cout.
7dbd5c5
@ywqzzy
all user choose stream size.
120a983
@ywqzzy
address comments.
0ce3668
@ywqzzy
rename.
02f3509
@ywqzzy
rename.
3dad372
@ywqzzy
address comments.
4910d32
@ywqzzy
clean
cc81990
@ywqzzy
address comments.
fc46cd5
@ywqzzy
address comments
b23f514
@ywqzzy
Merge branch 'master' of https://github.com/pingcap/tiflash into run_…
0216daa 
…interpreter_test
@ywqzzy
address comments.
927234c
@ywqzzy
format.
2639b7f
@ywqzzy
address comment.
eca8746
@ywqzzy
Merge branch 'master' of https://github.com/pingcap/tiflash into run_…
eb50f1e 
…interpreter_test
@ywqzzy
fix.
589263d
@ywqzzy
init.
caf6b73
@ywqzzy
update.
0874ad5
@ywqzzy
update.
dc82bcb
SeaRise
SeaRise reviewed May 25, 2022
View reviewed changes
Copy link
Contributor

@SeaRise SeaRise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

other LGTM

SeaRise
SeaRise approved these changes May 25, 2022
View reviewed changes
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 25, 2022
@ywqzzy ywqzzy requested review from fuzhe1989 and yibin87 May 25, 2022 03:49
yibin87
yibin87 approved these changes May 25, 2022
View reviewed changes
Copy link
Contributor

@yibin87 yibin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 25, 2022
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-all-tests

1 similar comment
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented May 25, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/FilterBlockInputStream.cpp                       75                67    10.67%           4                 2    50.00%         122               105    13.93%          56                53     5.36%
DataStreams/HashJoinBuildBlockInputStream.cpp                 8                 5    37.50%           2                 1    50.00%          24                 8    66.67%           4                 3    25.00%
DataStreams/HashJoinBuildBlockInputStream.h                   3                 0   100.00%           3                 0   100.00%           6                 0   100.00%           0                 0         -
DataStreams/IBlockInputStream.cpp                            27                11    59.26%           5                 2    60.00%          64                17    73.44%          22                 8    63.64%
DataStreams/IBlockInputStream.h                              30                20    33.33%          17                 9    47.06%          50                38    24.00%          16                13    18.75%
DataStreams/LimitBlockInputStream.cpp                        25                23     8.00%           3                 1    66.67%          42                36    14.29%          16                16     0.00%
DataStreams/LimitBlockInputStream.h                           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
DataStreams/MergeSortingBlockInputStream.cpp                114                45    60.53%          10                 0   100.00%         171                57    66.67%          76                36    52.63%
DataStreams/MergeSortingBlockInputStream.h                   11                 7    36.36%          11                 7    36.36%          11                 7    36.36%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp         113               101    10.62%          13                 9    30.77%         164               147    10.37%          72                67     6.94%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 4    33.33%           6                 4    33.33%          14                12    14.29%           0                 0         -
DataStreams/PartialSortingBlockInputStream.cpp                2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
DataStreams/PartialSortingBlockInputStream.h                  6                 3    50.00%           6                 3    50.00%           8                 3    62.50%           0                 0         -
DataStreams/TiRemoteBlockInputStream.h                      117               117     0.00%          17                17     0.00%         164               164     0.00%          60                60     0.00%
Flash/Coprocessor/DAGContext.h                               37                12    67.57%          28                10    64.29%          83                30    63.86%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              229                80    65.07%          38                 4    89.47%         574               127    77.87%         138                55    60.14%
Flash/Coprocessor/DAGQueryBlockInterpreter.h                  2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGStorageInterpreter.cpp                 424               424     0.00%          32                32     0.00%         870               870     0.00%         282               282     0.00%
Flash/Coprocessor/InterpreterDAG.cpp                         23                 8    65.22%           5                 0   100.00%          60                10    83.33%          16                 8    50.00%
Flash/Coprocessor/InterpreterUtils.cpp                       32                 8    75.00%           4                 0   100.00%          55                17    69.09%          24                10    58.33%
Flash/tests/gtest_interpreter.cpp                            82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
Interpreters/ExpressionActions.cpp                          439               260    40.77%          34                12    64.71%         787               441    43.96%         392               253    35.46%
Interpreters/ExpressionActions.h                             19                 2    89.47%          10                 0   100.00%          34                 2    94.12%          10                 2    80.00%
Interpreters/executeQuery.cpp                               199               133    33.17%          15                 8    46.67%         409               213    47.92%         136               112    17.65%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      2025              1368    32.44%         272               121    55.51%        4063              2304    43.29%        1340               992    25.97%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18262      9775             46.47%    204342  97942        52.07%

full coverage report (for internal network access only)

@ywqzzy ywqzzy force-pushed the print_inputstream branch from 1e80335 to e8fda6f Compare May 25, 2022 08:34
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-all-tests

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/run-unit-test

@sre-bot
Copy link
Collaborator

sre-bot commented May 25, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/FilterBlockInputStream.cpp                       75                67    10.67%           4                 2    50.00%         122               105    13.93%          56                53     5.36%
DataStreams/HashJoinBuildBlockInputStream.cpp                 8                 5    37.50%           2                 1    50.00%          28                 8    71.43%           4                 3    25.00%
DataStreams/HashJoinBuildBlockInputStream.h                   3                 0   100.00%           3                 0   100.00%           6                 0   100.00%           0                 0         -
DataStreams/IBlockInputStream.cpp                            27                11    59.26%           5                 2    60.00%          64                17    73.44%          22                 8    63.64%
DataStreams/IBlockInputStream.h                              30                20    33.33%          17                 9    47.06%          50                38    24.00%          16                13    18.75%
DataStreams/LimitBlockInputStream.cpp                        25                23     8.00%           3                 1    66.67%          42                36    14.29%          16                16     0.00%
DataStreams/LimitBlockInputStream.h                           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
DataStreams/MergeSortingBlockInputStream.cpp                114                45    60.53%          10                 0   100.00%         171                57    66.67%          76                36    52.63%
DataStreams/MergeSortingBlockInputStream.h                   11                 7    36.36%          11                 7    36.36%          11                 7    36.36%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp         113               101    10.62%          13                 9    30.77%         164               147    10.37%          72                67     6.94%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 4    33.33%           6                 4    33.33%          14                12    14.29%           0                 0         -
DataStreams/PartialSortingBlockInputStream.cpp                2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
DataStreams/PartialSortingBlockInputStream.h                  6                 3    50.00%           6                 3    50.00%           8                 3    62.50%           0                 0         -
DataStreams/TiRemoteBlockInputStream.h                      117               117     0.00%          17                17     0.00%         164               164     0.00%          60                60     0.00%
Flash/Coprocessor/DAGContext.h                               37                11    70.27%          28                 9    67.86%          83                29    65.06%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              229                80    65.07%          38                 4    89.47%         574               127    77.87%         138                55    60.14%
Flash/Coprocessor/DAGQueryBlockInterpreter.h                  2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGStorageInterpreter.cpp                 424               424     0.00%          32                32     0.00%         870               870     0.00%         282               282     0.00%
Flash/Coprocessor/InterpreterDAG.cpp                         25                 4    84.00%           5                 0   100.00%          60                 9    85.00%          18                 7    61.11%
Flash/Coprocessor/InterpreterUtils.cpp                       32                 8    75.00%           4                 0   100.00%          55                17    69.09%          24                10    58.33%
Flash/tests/gtest_interpreter.cpp                            82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
Interpreters/ExpressionActions.cpp                          439               260    40.77%          34                12    64.71%         787               441    43.96%         392               253    35.46%
Interpreters/ExpressionActions.h                             19                 2    89.47%          10                 0   100.00%          34                 2    94.12%          10                 2    80.00%
Interpreters/executeQuery.cpp                               199               133    33.17%          15                 8    46.67%         409               213    47.92%         136               112    17.65%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      2027              1363    32.76%         272               120    55.88%        4067              2302    43.40%        1342               991    26.15%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18265      9778             46.47%    204517  97932        52.12%

full coverage report (for internal network access only)

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/merge

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 25, 2022

@ywqzzy: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 25, 2022

This pull request has been accepted and is ready to merge.

DetailsCommit hash: e8fda6f

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 25, 2022
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/hold

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 25, 2022
@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022 •
edited
Loading

/run-unit-test

@sre-bot
Copy link
Collaborator

sre-bot commented May 25, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DataStreams/FilterBlockInputStream.cpp                       75                67    10.67%           4                 2    50.00%         122               105    13.93%          56                53     5.36%
DataStreams/HashJoinBuildBlockInputStream.cpp                 8                 5    37.50%           2                 1    50.00%          28                 8    71.43%           4                 3    25.00%
DataStreams/HashJoinBuildBlockInputStream.h                   3                 0   100.00%           3                 0   100.00%           6                 0   100.00%           0                 0         -
DataStreams/IBlockInputStream.cpp                            27                11    59.26%           5                 2    60.00%          64                17    73.44%          22                 8    63.64%
DataStreams/IBlockInputStream.h                              30                20    33.33%          17                 9    47.06%          50                38    24.00%          16                13    18.75%
DataStreams/LimitBlockInputStream.cpp                        25                23     8.00%           3                 1    66.67%          42                36    14.29%          16                16     0.00%
DataStreams/LimitBlockInputStream.h                           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
DataStreams/MergeSortingBlockInputStream.cpp                114                45    60.53%          10                 0   100.00%         171                57    66.67%          76                36    52.63%
DataStreams/MergeSortingBlockInputStream.h                   11                 7    36.36%          11                 7    36.36%          11                 7    36.36%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp         113               101    10.62%          13                 9    30.77%         164               147    10.37%          72                67     6.94%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 4    33.33%           6                 4    33.33%          14                12    14.29%           0                 0         -
DataStreams/PartialSortingBlockInputStream.cpp                2                 0   100.00%           2                 0   100.00%           8                 0   100.00%           0                 0         -
DataStreams/PartialSortingBlockInputStream.h                  6                 3    50.00%           6                 3    50.00%           8                 3    62.50%           0                 0         -
DataStreams/TiRemoteBlockInputStream.h                      117               117     0.00%          17                17     0.00%         164               164     0.00%          60                60     0.00%
Flash/Coprocessor/DAGContext.h                               37                11    70.27%          28                 9    67.86%          83                29    65.06%          12                 6    50.00%
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              229                80    65.07%          38                 4    89.47%         574               127    77.87%         138                55    60.14%
Flash/Coprocessor/DAGQueryBlockInterpreter.h                  2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
Flash/Coprocessor/DAGStorageInterpreter.cpp                 424               424     0.00%          32                32     0.00%         870               870     0.00%         282               282     0.00%
Flash/Coprocessor/InterpreterDAG.cpp                         25                 4    84.00%           5                 0   100.00%          60                 9    85.00%          18                 7    61.11%
Flash/Coprocessor/InterpreterUtils.cpp                       32                 8    75.00%           4                 0   100.00%          55                17    69.09%          24                10    58.33%
Flash/tests/gtest_interpreter.cpp                            82                38    53.66%           3                 0   100.00%         339                 0   100.00%           8                 8     0.00%
Interpreters/ExpressionActions.cpp                          439               260    40.77%          34                12    64.71%         787               441    43.96%         392               253    35.46%
Interpreters/ExpressionActions.h                             19                 2    89.47%          10                 0   100.00%          34                 2    94.12%          10                 2    80.00%
Interpreters/executeQuery.cpp                               199               133    33.17%          15                 8    46.67%         409               213    47.92%         136               112    17.65%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      2027              1363    32.76%         272               120    55.88%        4067              2302    43.40%        1342               991    26.15%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18265      9778             46.47%    204517  97943        52.11%

full coverage report (for internal network access only)

@ywqzzy
Copy link
Contributor Author

ywqzzy commented May 25, 2022

/unhold

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 25, 2022
@ti-chi-bot ti-chi-bot merged commit 35c8ea1 into pingcap:master May 25, 2022
@ywqzzy ywqzzy mentioned this pull request May 27, 2022
Closed
54 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yibin87 yibin87 yibin87 approved these changes

@fuzhe1989 fuzhe1989 Awaiting requested review from fuzhe1989

+1 more reviewer

@SeaRise SeaRise SeaRise approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@ywqzzy @ti-chi-bot @sre-bot @fuzhe1989 @yibin87 @SeaRise