Skip to content

add Retryer to retry Task Execute ack gracefully#2679

Closed
gabry-lab wants to merge 3 commits intoapache:devfrom
gabry-lab:optimizeTaskAck
Closed

add Retryer to retry Task Execute ack gracefully#2679
gabry-lab wants to merge 3 commits intoapache:devfrom
gabry-lab:optimizeTaskAck

Conversation

@gabry-lab
Copy link
Copy Markdown
Member

Tips

What is the purpose of the pull request

Now, dolphin already refactor worker ,and netty is being used.
It can be predicted that async method will be more and more used ,and function retryer is necessary .
If we simply call function twice or more , it's ugly and beyonds understanding

So I suggest we introduce a retryer class ,for example https://github.com/rholder/guava-retrying

Brief change log

  • Add guava-retrying to root pom.xml and dolphinscheduler-server pom.xml
  • Add retry logic to TaskExecuteProcessor

@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #2679 into dev will decrease coverage by 0.11%.
The diff coverage is 46.66%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev    #2679      +/-   ##
============================================
- Coverage     36.32%   36.20%   -0.12%     
+ Complexity     2481     2475       -6     
============================================
  Files           431      431              
  Lines         19893    19900       +7     
  Branches       2421     2421              
============================================
- Hits           7226     7205      -21     
- Misses        12025    12052      +27     
- Partials        642      643       +1     
Impacted Files Coverage Δ Complexity Δ
.../server/worker/processor/TaskExecuteProcessor.java 19.40% <46.66%> (-17.27%) 2.00 <1.00> (-1.00)
...he/dolphinscheduler/api/service/LoggerService.java 82.60% <0.00%> (-8.70%) 9.00% <0.00%> (-1.00%)
...nscheduler/server/entity/TaskExecutionContext.java 73.26% <0.00%> (-5.95%) 53.00% <0.00%> (-3.00%)
...e/dolphinscheduler/remote/NettyRemotingClient.java 51.07% <0.00%> (-2.88%) 9.00% <0.00%> (-2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68e3487...709ccdc. Read the comment docs.

@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

46.7% 46.7% Coverage
0.0% 0.0% Duplication

Copy link
Copy Markdown
Contributor

@qiaozhanwei qiaozhanwei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is best to encapsulate a tool class, because it needs to be called in many places, not only in ack and response

for example master send task to worker,and worker fetch ip from zookeeper etc,all need this tool

@gabry-lab
Copy link
Copy Markdown
Member Author

I think it is best to encapsulate a tool class, because it needs to be called in many places, not only in ack and response

for example master send task to worker,and worker fetch ip from zookeeper etc,all need this tool

great, I will close this PR, and create another which encapsulate a tool class to make it reused easier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants