Skip to content

async server#4120

Merged
ti-chi-bot merged 166 commits intopingcap:masterfrom
bestwoody:async_server_base
Mar 13, 2022
Merged

async server#4120
ti-chi-bot merged 166 commits intopingcap:masterfrom
bestwoody:async_server_base

Conversation

@bestwoody
Copy link
Contributor

@bestwoody bestwoody commented Feb 23, 2022

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

https://pingcap.feishu.cn/docs/doccnjURwS2PfTpz0tDolmUtsAe

Concept:
CallData: grpc session container with context data for that session. A state machine to react for the gRPC event(such as: write, writeDone, return Status ...)
Tag: a unique instance of CallData
CQ is the queue for polling the gRPC events.

How to develop a async server:
Use WithAsyncMethod_AAA template class to implement a async service.
For the api AAA you want it to be async, implement CallData for it.

  1. support both sync and async server.
  2. growing threads contributed by MppTunnel::Sendloop & FlashService::establishMpp is solved.
  3. identical exception handle with the sync version.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@bestwoody
Copy link
Contributor Author

/rebuild

1 similar comment
@bestwoody
Copy link
Contributor Author

/rebuild

Signed-off-by: bestwoody <bestwoody@163.com>
@bestwoody
Copy link
Contributor Author

/rebuild

@bestwoody bestwoody removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/needs-linked-issue labels Mar 13, 2022
@bestwoody
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 13, 2022

Coverage for changed files

Filename                              Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                    18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
Flash/EstablishCall.h                       2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/FlashService.h                        4                 4     0.00%           4                 4     0.00%          13                13     0.00%           0                 0         -
Flash/Mpp/GRPCReceiverContext.cpp          41                41     0.00%          15                15     0.00%         104               104     0.00%          22                22     0.00%
Flash/Mpp/MPPTask.cpp                     349               349     0.00%          17                17     0.00%         313               313     0.00%         160               160     0.00%
Flash/Mpp/MPPTunnel.cpp                   172               172     0.00%          16                16     0.00%         231               231     0.00%          88                88     0.00%
Flash/Mpp/MPPTunnel.h                       8                 8     0.00%           8                 8     0.00%          14                14     0.00%           0                 0         -
Flash/Mpp/PacketWriter.h                    2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Interpreters/Settings.h                     1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                     597               578     3.18%          76                64    15.79%         727               680     6.46%         278               270     2.88%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16853      9456             43.89%    189597  95830        49.46%

full coverage report (for internal network access only)

@bestwoody
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 13, 2022

Coverage for changed files

Filename                              Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                    18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
Flash/EstablishCall.h                       2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/FlashService.h                        4                 4     0.00%           4                 4     0.00%          13                13     0.00%           0                 0         -
Flash/Mpp/GRPCReceiverContext.cpp          41                41     0.00%          15                15     0.00%         104               104     0.00%          22                22     0.00%
Flash/Mpp/MPPTask.cpp                     349               349     0.00%          17                17     0.00%         313               313     0.00%         160               160     0.00%
Flash/Mpp/MPPTunnel.cpp                   172               172     0.00%          16                16     0.00%         231               231     0.00%          88                88     0.00%
Flash/Mpp/MPPTunnel.h                       8                 8     0.00%           8                 8     0.00%          14                14     0.00%           0                 0         -
Flash/Mpp/PacketWriter.h                    2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Interpreters/Settings.h                     1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                     597               578     3.18%          76                64    15.79%         727               680     6.46%         278               270     2.88%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16853      9456             43.89%    189597  95856        49.44%

full coverage report (for internal network access only)

Signed-off-by: bestwoody <bestwoody@163.com>
@ti-chi-bot
Copy link
Member

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

📖 For more info, you can check the "Contribute Code" section in the development guide.

@bestwoody
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 13, 2022

Coverage for changed files

Filename                              Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                    18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
Flash/EstablishCall.h                       2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/FlashService.h                        4                 4     0.00%           4                 4     0.00%          13                13     0.00%           0                 0         -
Flash/Mpp/GRPCReceiverContext.cpp          41                41     0.00%          15                15     0.00%         104               104     0.00%          22                22     0.00%
Flash/Mpp/MPPTask.cpp                     351               351     0.00%          17                17     0.00%         313               313     0.00%         164               164     0.00%
Flash/Mpp/MPPTunnel.cpp                   172               172     0.00%          16                16     0.00%         231               231     0.00%          88                88     0.00%
Flash/Mpp/MPPTunnel.h                       8                 8     0.00%           8                 8     0.00%          14                14     0.00%           0                 0         -
Flash/Mpp/PacketWriter.h                    2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Interpreters/Settings.h                     1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                     599               580     3.17%          76                64    15.79%         727               680     6.46%         282               274     2.84%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16853      9456             43.89%    189597  95838        49.45%

full coverage report (for internal network access only)

@bestwoody
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@bestwoody: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

DetailsCommit hash: 57580bf

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 13, 2022
@sre-bot
Copy link
Collaborator

sre-bot commented Mar 13, 2022

Coverage for changed files

Filename                              Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                    18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
Flash/EstablishCall.h                       2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/FlashService.h                        4                 4     0.00%           4                 4     0.00%          13                13     0.00%           0                 0         -
Flash/Mpp/GRPCReceiverContext.cpp          41                41     0.00%          15                15     0.00%         104               104     0.00%          22                22     0.00%
Flash/Mpp/MPPTask.cpp                     351               351     0.00%          17                17     0.00%         313               313     0.00%         164               164     0.00%
Flash/Mpp/MPPTunnel.cpp                   172               172     0.00%          16                16     0.00%         231               231     0.00%          88                88     0.00%
Flash/Mpp/MPPTunnel.h                       8                 8     0.00%           8                 8     0.00%          14                14     0.00%           0                 0         -
Flash/Mpp/PacketWriter.h                    2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Interpreters/Settings.h                     1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                     599               580     3.17%          76                64    15.79%         727               680     6.46%         282               274     2.84%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16853      9456             43.89%    189597  95849        49.45%

full coverage report (for internal network access only)

@bestwoody
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@bestwoody: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

flash_service = std::make_unique<AsyncFlashService>(server);
else
flash_service = std::make_unique<FlashService>(server);
diagnostics_service = std::make_unique<DiagnosticsService>(server);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these args reasonalbe for async server?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants