Skip to content

[DS-6694][Master] Reduce the refresh resource interval of LowerWeight#6695

Merged
CalvinKirs merged 2 commits intoapache:devfrom
caishunfeng:reduce_refresh_resource_interval
Nov 5, 2021
Merged

[DS-6694][Master] Reduce the refresh resource interval of LowerWeight#6695
CalvinKirs merged 2 commits intoapache:devfrom
caishunfeng:reduce_refresh_resource_interval

Conversation

@caishunfeng
Copy link
Copy Markdown
Contributor

this pr close #6694

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

Copy link
Copy Markdown
Member

@CalvinKirs CalvinKirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is best that the user can configure this parameter.

@caishunfeng
Copy link
Copy Markdown
Contributor Author

It is best that the user can configure this parameter.

I thought about this problem, but it just a data transform in memory, it's pointless to configure this parameter.
And user can config worker heartbeat interval to control sync node info rate.

this.lock = new ReentrantLock();
this.executorService = Executors.newSingleThreadScheduledExecutor(new NamedThreadFactory("LowerWeightHostManagerExecutor"));
this.executorService.scheduleWithFixedDelay(new RefreshResourceTask(), 0, 5, TimeUnit.SECONDS);
this.executorService.scheduleWithFixedDelay(new RefreshResourceTask(), 0, 1, TimeUnit.SECONDS);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add listener to ServerNodeManager and register RefreshResourceTask as a listener so that we can immediately refresh the resources when ServerNodeManager updates the nodes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. I found that WorkerDataListener just handle add and remove event, but not update event.
I will try to add update event handle for it.

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Nov 5, 2021

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

Copy link
Copy Markdown
Member

@CalvinKirs CalvinKirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job

@CalvinKirs CalvinKirs merged commit 72daba5 into apache:dev Nov 5, 2021
@CalvinKirs CalvinKirs added this to the 2.0.0-release milestone Nov 5, 2021
lenboo pushed a commit to lenboo/dolphinscheduler that referenced this pull request Nov 8, 2021
…apache#6695)

* [DS-6694][Master] Reduce the refresh resource interval of LowerWeight

* add server node update event handle

Co-authored-by: caishunfeng <534328519@qq.com>
@caishunfeng caishunfeng deleted the reduce_refresh_resource_interval branch December 15, 2021 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Master] Reduce the refresh resource interval of LowerWeight

3 participants