Skip to content

Balanced thread pulling#451

Merged
sanderpick merged 3 commits intotextileio:masterfrom
anyproto:improvement/periodic-pulls
Oct 14, 2020
Merged

Balanced thread pulling#451
sanderpick merged 3 commits intotextileio:masterfrom
anyproto:improvement/periodic-pulls

Conversation

@dgtony
Copy link
Copy Markdown
Contributor

@dgtony dgtony commented Oct 14, 2020

Here is a small improvement essential for the ThreadsDB nodes serving a lot of threads. Currently threads are pulled as follows: once in a period goroutines with pullThread procedure for all known threads start simultaneously, and it leads to substantial spikes in CPU/memory/network usage. We can pretty easily get rid of the spikes and equalize resource consumption by allocating pull operations evenly during the specified period.
balanced-pulling

dgtony added 3 commits October 1, 2020 18:56
Signed-off-by: Anton Dort-Golts <dortgolts@gmail.com>
Signed-off-by: Anton Dort-Golts <dortgolts@gmail.com>
Signed-off-by: Anton Dort-Golts <dortgolts@gmail.com>
Copy link
Copy Markdown
Contributor

@sanderpick sanderpick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, LGTM! Thanks for the great contributions.

@sanderpick sanderpick merged commit 2a367ca into textileio:master Oct 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants