Skip to content

[Feature#3234][cluster]enhanced load balancing#3235

Merged
qiaozhanwei merged 12 commits intoapache:devfrom
CalvinKirs:loadbalance
Aug 10, 2020
Merged

[Feature#3234][cluster]enhanced load balancing#3235
qiaozhanwei merged 12 commits intoapache:devfrom
CalvinKirs:loadbalance

Conversation

@CalvinKirs
Copy link
Copy Markdown
Member

@CalvinKirs CalvinKirs commented Jul 18, 2020

weight-based load balancing algorithm
this closes #3234

What is the purpose of the pull request

weight-based load balancing algorithm

Implementation of load balancing based on weight.
Implementation of load balancing based on workgroups.

Related mail:https://lists.apache.org/thread.html/r06af73b0897d8a0c97a16ca1b9316d5d0bc29de833f60c10ca46415a%40%3Cdev.dolphinscheduler.apache.org%3E
https://lists.apache.org/thread.html/r5a70a5f43aa49973923ef23d5b079b883cda746a14e91f9c083a985a%40%3Cdev.dolphinscheduler.apache.org%3E
this closes #3234

Verify this pull request

This pull request is already covered by existing tests, such as (please describe tests).

This change added tests and can be verified as follows:

weight-based load balancing algorithm
this close # 3234
@CalvinKirs CalvinKirs changed the title [Future#3234][cluster]enhanced load balancing [WIP][cluster]enhanced load balancing Jul 18, 2020
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jul 18, 2020

Codecov Report

Merging #3235 into dev will increase coverage by 1.53%.
The diff coverage is 57.89%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev    #3235      +/-   ##
============================================
+ Coverage     34.15%   35.69%   +1.53%     
- Complexity     2422     2576     +154     
============================================
  Files           443      448       +5     
  Lines         20637    21017     +380     
  Branches       2531     2572      +41     
============================================
+ Hits           7049     7501     +452     
+ Misses        12926    12821     -105     
- Partials        662      695      +33     
Impacted Files Coverage Δ Complexity Δ
...org/apache/dolphinscheduler/remote/utils/Host.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...server/master/dispatch/host/CommonHostManager.java 9.09% <0.00%> (-2.03%) 1.00 <0.00> (ø)
...er/master/dispatch/host/RoundRobinHostManager.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...inscheduler/server/worker/config/WorkerConfig.java 14.81% <0.00%> (+14.81%) 4.00 <0.00> (+4.00)
...er/master/dispatch/host/assign/RandomSelector.java 77.77% <77.77%> (-22.23%) 3.00 <3.00> (+1.00) ⬇️
...aster/dispatch/host/assign/RoundRobinSelector.java 85.71% <85.71%> (-14.29%) 8.00 <8.00> (+6.00) ⬇️
...server/master/dispatch/host/RandomHostManager.java 75.00% <100.00%> (ø) 1.00 <0.00> (ø)
...heduler/server/worker/registry/WorkerRegistry.java 84.90% <100.00%> (+0.90%) 11.00 <1.00> (+1.00)
...g/apache/dolphinscheduler/api/dto/ProcessMeta.java 0.00% <0.00%> (-91.84%) 0.00% <0.00%> (-31.00%)
.../master/dispatch/host/assign/AbstractSelector.java 66.66% <0.00%> (-33.34%) 3.00% <0.00%> (-1.00%)
... and 84 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b54c76...1c70447. Read the comment docs.

@CalvinKirs
Copy link
Copy Markdown
Member Author

todo:RoundRobinSelector

@CalvinKirs CalvinKirs changed the title [WIP][cluster]enhanced load balancing [Future#3234][cluster]enhanced load balancing Jul 19, 2020
@CalvinKirs
Copy link
Copy Markdown
Member Author

done

@CalvinKirs CalvinKirs changed the title [Future#3234][cluster]enhanced load balancing [Feature#3234][cluster]enhanced load balancing Jul 21, 2020
@CalvinKirs
Copy link
Copy Markdown
Member Author

TBR.

@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 2 Security Hotspots to review)
Code Smell A 0 Code Smells

62.1% 62.1% Coverage
0.0% 0.0% Duplication

Copy link
Copy Markdown
Contributor

@qiaozhanwei qiaozhanwei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1


# default worker group
#worker.group=${WORKER_GROUP} No newline at end of file
#worker.group=${WORKER_GROUP}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worker.properties worker.group change to worker.groups

@qiaozhanwei qiaozhanwei merged commit d6a32ac into apache:dev Aug 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][cluster] enhanced load balancing

4 participants