Skip to content

Use condition variable to wake stats collector.#40481

Merged
AkihiroSuda merged 1 commit intomoby:masterfrom
cpuguy83:stats_use_cond_var
Feb 21, 2020
Merged

Use condition variable to wake stats collector.#40481
AkihiroSuda merged 1 commit intomoby:masterfrom
cpuguy83:stats_use_cond_var

Conversation

@cpuguy83
Copy link
Copy Markdown
Member

@cpuguy83 cpuguy83 commented Feb 8, 2020

Before the collection goroutine wakes up every 1 second (as configured).
This sleep interval is in case there are no stats to collect we don't
end up in a tight loop.

Instead use a condition variable to signal that a collection is needed.
This prevents us from waking the goroutine needlessly when there is no
one looking for stats.

For now I've kept the sleep just moved it to the end of the loop, which
gives some space between collections.


This significantly speeds up the first stat collection.
Example, docker stats --no-stream goes from about 1.8s+- to 1.03s+-

Before the collection goroutine wakes up every 1 second (as configured).
This sleep interval is in case there are no stats to collect we don't
end up in a tight loop.

Instead use a condition variable to signal that a collection is needed.
This prevents us from waking the goroutine needlessly when there is no
one looking for stats.

For now I've kept the sleep just moved it to the end of the loop, which
gives some space between collections.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
@cpuguy83
Copy link
Copy Markdown
Member Author

cpuguy83 commented Feb 8, 2020

Coupled with #40478, you can get a single stat in ~0.02s-0.03s

@thaJeztah
Copy link
Copy Markdown
Member

ping @tiborvass @AkihiroSuda PTAL

@cpuguy83
Copy link
Copy Markdown
Member Author

Issue this is improving is that the kubelet hits the stats endpoint to collect stats and it takes almost 2s per container. This changes it to just over 1s per container.

Plus it's just not great to wake the daemon every second for no reason.

@AkihiroSuda AkihiroSuda merged commit 0c6b857 into moby:master Feb 21, 2020
@cpuguy83 cpuguy83 deleted the stats_use_cond_var branch February 21, 2020 05:27
@thaJeztah thaJeztah added this to the 20.03.0 milestone Apr 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants