[Dubbo-1876] Enhancements for the new async way of Dubbo by chickenlj · Pull Request #1957 · apache/dubbo

chickenlj · 2018-06-18T15:24:37Z

What is the purpose of the change

Enhancements for the new async way of Dubbo introduced in #1876. Below is the main issues this PR trys to solve:

Support service signature with original CompletableFuture returntype.

      public interface DemoService {
          CompletableFuture<String> sayHello(String name);
      }

Provide context restore API when switching threads.

      AsyncContext.signalContextSwitch();

Postprocess of async result in filter chain.

Brief changelog

Add AsyncRpcResult to represent result value of async invoke, equivalent to RpcResult for sync invoke.
Add PostProcessFilter to support postprocess of result in filter chain
Change AbstractProxyInvoker and DubboInvoker to support services with original Future signature, supplementary for generated Future service.

Verifying this change

Run samples for how to use async in a new way.
Still lack of UT, expect to add soon.

Follow this checklist to help us incorporate your contribution quickly and easily:

Make sure there is a GITHUB_issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
Run mvn clean install -DskipTests & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
If this contribution is large, please follow the Software Donation Guide.

# Conflicts: # dubbo-rpc/dubbo-rpc-api/pom.xml

…develop-async-response

# Conflicts: # dubbo-cluster/src/main/java/com/alibaba/dubbo/rpc/cluster/support/ClusterUtils.java # dubbo-common/src/main/java/com/alibaba/dubbo/common/Constants.java # dubbo-config/dubbo-config-api/src/main/java/com/alibaba/dubbo/config/ReferenceConfig.java # dubbo-demo/dubbo-demo-api/src/main/java/com/alibaba/dubbo/demo/DemoService.java # dubbo-demo/dubbo-demo-consumer/src/main/java/com/alibaba/dubbo/demo/consumer/Consumer.java # dubbo-demo/dubbo-demo-provider/src/main/java/com/alibaba/dubbo/demo/provider/DemoServiceImpl.java # dubbo-demo/dubbo-demo-provider/src/main/resources/META-INF/spring/dubbo-demo-provider.xml # dubbo-registry/dubbo-registry-api/src/main/java/com/alibaba/dubbo/registry/integration/RegistryProtocol.java # dubbo-remoting/dubbo-remoting-api/src/main/java/com/alibaba/dubbo/remoting/exchange/ExchangeHandler.java # dubbo-remoting/dubbo-remoting-api/src/main/java/com/alibaba/dubbo/remoting/exchange/support/ExchangeHandlerAdapter.java # dubbo-remoting/dubbo-remoting-api/src/main/java/com/alibaba/dubbo/remoting/exchange/support/ExchangeHandlerDispatcher.java # dubbo-remoting/dubbo-remoting-api/src/main/java/com/alibaba/dubbo/remoting/exchange/support/header/HeaderExchangeChannel.java # dubbo-remoting/dubbo-remoting-api/src/main/java/com/alibaba/dubbo/remoting/exchange/support/header/HeaderExchangeHandler.java # dubbo-remoting/dubbo-remoting-api/src/test/java/com/alibaba/dubbo/remoting/PerformanceServerTest.java # dubbo-remoting/dubbo-remoting-api/src/test/java/com/alibaba/dubbo/remoting/handler/HeaderExchangeHandlerTest.java # dubbo-remoting/dubbo-remoting-netty/src/test/java/com/alibaba/dubbo/remoting/exchange/support/header/HeartbeatHandlerTest.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/Result.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/RpcContext.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/RpcResult.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/filter/ConsumerContextFilter.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/filter/ContextFilter.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/filter/ExceptionFilter.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/proxy/AbstractProxyFactory.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/proxy/AbstractProxyInvoker.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/proxy/InvokerInvocationHandler.java # dubbo-rpc/dubbo-rpc-api/src/main/java/com/alibaba/dubbo/rpc/support/RpcUtils.java # dubbo-rpc/dubbo-rpc-dubbo/src/main/java/com/alibaba/dubbo/rpc/protocol/dubbo/DubboCodec.java # dubbo-rpc/dubbo-rpc-dubbo/src/main/java/com/alibaba/dubbo/rpc/protocol/dubbo/DubboInvoker.java # dubbo-rpc/dubbo-rpc-dubbo/src/main/java/com/alibaba/dubbo/rpc/protocol/dubbo/DubboProtocol.java # dubbo-rpc/dubbo-rpc-dubbo/src/main/java/com/alibaba/dubbo/rpc/protocol/dubbo/FutureAdapter.java # dubbo-rpc/dubbo-rpc-thrift/src/main/java/com/alibaba/dubbo/rpc/protocol/thrift/ThriftProtocol.java # dubbo-test/dubbo-test-benchmark/pom.xml # dubbo-test/dubbo-test-benchmark/src/main/java/com/alibaba/dubbo/rpc/benchmark/AbstractBenchmarkServer.java

# Conflicts: # dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/AsyncContext.java # dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/AsyncContextImpl.java

codecov-io · 2018-06-19T12:18:08Z

Codecov Report

Merging #1957 into master will increase coverage by <.01%.
The diff coverage is 38.03%.

@@             Coverage Diff              @@
##             master    #1957      +/-   ##
============================================
+ Coverage     53.07%   53.07%   +<.01%     
- Complexity     4930     4956      +26     
============================================
  Files           557      559       +2     
  Lines         24802    24892      +90     
  Branches       4427     4431       +4     
============================================
+ Hits          13164    13212      +48     
- Misses         9618     9660      +42     
  Partials       2020     2020

Impacted Files	Coverage Δ	Complexity Δ
...oting/exchange/support/ExchangeHandlerAdapter.java	`0% <ø> (ø)`	`0 <0> (ø)`	⬇️
.../src/main/java/org/apache/dubbo/rpc/RpcResult.java	`72.72% <ø> (+21.44%)`	`10 <0> (ø)`	⬇️
...c/main/java/org/apache/dubbo/common/Constants.java	`88.88% <ø> (ø)`	`1 <0> (ø)`	⬇️
...ache/dubbo/rpc/proxy/InvokerInvocationHandler.java	`30.76% <0%> (-4.02%)`	`3 <0> (ø)`
...exchange/support/header/HeaderExchangeHandler.java	`30.15% <0%> (ø)`	`10 <0> (ø)`	⬇️
...ava/org/apache/dubbo/rpc/SimpleAsyncRpcResult.java	`0% <0%> (ø)`	`0 <0> (?)`
...le/src/main/java/com/alibaba/dubbo/rpc/Result.java	`22.22% <0%> (-11.12%)`	`0 <0> (ø)`
...main/java/org/apache/dubbo/rpc/AsyncRpcResult.java	`0% <0%> (ø)`	`0 <0> (ø)`	⬇️
...apache/dubbo/rpc/filter/ConsumerContextFilter.java	`85.71% <100%> (+1.09%)`	`3 <2> (+1)`	⬆️
.../dubbo/rpc/protocol/dubbo/filter/FutureFilter.java	`59.09% <100%> (+2.52%)`	`16 <6> (+3)`	⬆️
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c36be6...0ff5134. Read the comment docs.

# Conflicts: # dubbo-rpc/dubbo-rpc-api/src/test/java/org/apache/dubbo/rpc/support/DemoService.java # dubbo-rpc/dubbo-rpc-api/src/test/java/org/apache/dubbo/rpc/support/DemoServiceImpl.java

nobodyiam · 2018-07-05T09:07:08Z

Looks fantastic! This new async way of Dubbo is quite amazing!

(It may be better to squash the commits into one commit)

chickenlj · 2018-07-06T03:17:43Z

https://github.com/chickenlj/incubator-dubbo/wiki/%E5%A6%82%E4%BD%95%E5%9F%BA%E4%BA%8EDubbo%E5%AE%9E%E7%8E%B0%E5%85%A8%E5%BC%82%E6%AD%A5%E8%B0%83%E7%94%A8%E9%93%BE

An article introducing the new async way was recorded here, written in Chinese.

chickenlj · 2018-07-06T03:23:10Z

Thank @hengyunabc for help reviewing, he totally agrees with the new async way showed in the demo.
And he'll dive into the code soon.

# Conflicts: # dubbo-rpc/dubbo-rpc-thrift/src/main/java/org/apache/dubbo/rpc/protocol/thrift/ThriftProtocol.java

* commit 'b055991b317f4e58d256721875a00c52fe415510': (271 commits) Merge pull request apache#1957, enhancements for the new async way of Dubbo. rename log file from alibaba to custom-access (apache#2057) Merge pull request apache#2049, upgrade netty4 to the latest release and make it the default option for transporter. Format style. Restore the badges in README. Polish README. Refactor README. Merge pull request apache#2047, deprecate dubbo-rpc-thrift. Merge pull request apache#2005, change maven parent from sonatype to apache. rename access log in unit test from 'alibaba' to 'alibaba.log' so that it cannot be committed by accident. (apache#2051) add test cases for injvm rpc protocol (apache#2041) add test cases for rpc thrift protocol (apache#2042) Merge pull request apache#1966, introduces dubbo metrics API module. [Dubbo- unit test class not found] fix class not found "hi" (apache#2034) add README for compatible module (apache#2019) Polish README. update README (apache#2025) [Dubbo-1695] Enhance the test coverage part-16 : dubbo-rpc/dubbo-rpc-api module (apache#2004) Merge pull request apache#1997, clienthandler in netty4 should trigger heartbeat handler. Update issue template description. ...

chickenlj added 30 commits May 25, 2018 17:10

Async support start

55d77b7

A basically worked version for provider async

6aec4d2

asynchronously send response

7b8e8e9

should send Response for sync invoke

94c21b5

demo

7d46be7

Support method with Future return type.

6ee2b60

Merge branch 'develop-async-response' into develop-demo-test

0c35d8f

Add async interface annotation and processor

a1305f6

demo

c06ff03

polish async interface processor

4a891d8

fix concurrent write back problem when use provider async.

fcd30ec

revert demo changes

fe0fe6b

config async processor as spi.

77c52c2

add missed pom files

f2d0476

make DubboAsync work at source compile phase.

f1f0cf0

change annotation value to array

3e9ffb4

transfer async-processor module to eco-system

ff22444

Fix NPE in UT

0380d1b

add async module to pom for package and bom dependency.

b08454c

Delete dubbo-async module, it's not necessary.

a82db94

handle exception throw by provider

a79d7f4

remove jdk7 support

39d9a9c

support method with original CompletableFuture return type

452638c

Merge branch 'master' into local/chickenlj/develop-async-response

f7c19fc

# Conflicts: # dubbo-rpc/dubbo-rpc-api/pom.xml

Merge branch 'master' into local/chickenlj/develop-async-response

274d594

support method with original future return type.

d65835f

Merge remote-tracking branch 'chickenlj/develop-async-response' into …

9a96df2

…develop-async-response

demo test

0a81d5c

optimization

a9ca101

test biz exception

2b3b2cf

chickenlj added 13 commits June 15, 2018 14:28

Manually resolve conflicts after merge master(repackage)

bae63ea

support context switch & optimization

321bf46

fix several bugs: context restore, consumer callback

2a6a0b4

consumerFilter: clear attachments instead of remove context

c0d79d6

change async API in RpcContext to CompletableFuture.

14ed6a3

Delete demo codes

063a80e

fix compilation problem

63db2d8

Fix RpcUtils, get the right method return type.

71a2206

unified to AsyncRpcResult on consumer side.

84b8b48

fix UT error

f21fc21

Merge branch 'master' into develop-async-response

4c5d60f

# Conflicts: # dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/AsyncContext.java # dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/AsyncContextImpl.java

resolve compile problem after merge master

2671a04

chickenlj added 3 commits June 30, 2018 17:47

Fix bug when use startAsync().

4977d6c

Merge branch 'master' into develop-async-response

16ea830

# Conflicts: # dubbo-rpc/dubbo-rpc-api/src/test/java/org/apache/dubbo/rpc/support/DemoService.java # dubbo-rpc/dubbo-rpc-api/src/test/java/org/apache/dubbo/rpc/support/DemoServiceImpl.java

Fix async UT

75b6f55

chickenlj added this to the 2.7.0 milestone Jul 9, 2018

Merge branch 'master' into develop-async-response

0ff5134

# Conflicts: # dubbo-rpc/dubbo-rpc-thrift/src/main/java/org/apache/dubbo/rpc/protocol/thrift/ThriftProtocol.java

chickenlj merged commit b055991 into apache:master Jul 11, 2018

This was referenced Jul 31, 2018

server async without callback #85

Closed

Dubbo处理异步请求的方式 #122

Closed

provider如何异步处理请求 #177

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Dubbo-1876] Enhancements for the new async way of Dubbo#1957

[Dubbo-1876] Enhancements for the new async way of Dubbo#1957
chickenlj merged 47 commits intoapache:masterfrom
chickenlj:develop-async-response

chickenlj commented Jun 18, 2018 •

edited

Loading

Uh oh!

codecov-io commented Jun 19, 2018 •

edited

Loading

Uh oh!

nobodyiam commented Jul 5, 2018

Uh oh!

chickenlj commented Jul 6, 2018

Uh oh!

chickenlj commented Jul 6, 2018 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

chickenlj commented Jun 18, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief changelog

Verifying this change

Uh oh!

codecov-io commented Jun 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nobodyiam commented Jul 5, 2018

Uh oh!

chickenlj commented Jul 6, 2018

Uh oh!

chickenlj commented Jul 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chickenlj commented Jun 18, 2018 •

edited

Loading

codecov-io commented Jun 19, 2018 •

edited

Loading

chickenlj commented Jul 6, 2018 •

edited

Loading