-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Closed
Labels
questionQuestions that are neither investigations, bugs, nor enhancementsQuestions that are neither investigations, bugs, nor enhancementsstalestalebot believes this issue/PR has not been touched recentlystalebot believes this issue/PR has not been touched recently
Description
We are heavy users of endpoints with metadata and after we started running a build that includes 09c5d35 we are seeing EdfLoadBalancerBase::refresh() consume > 10% of CPU on a totally idle envoy instance. From perf top:
13.59% envoy [.] Envoy::Upstream::EdfLoadBalancerBase::refresh(unsigned int)::{lambda(Envoy::Upstream::ZoneAwareLoadBalancerBase::HostsSourc
8.63% envoy [.] Envoy::Config::Metadata::metadataValue
4.11% envoy [.] std::vector<std::shared_ptr<Envoy::Upstream::Host>, std::allocator<std::shared_ptr<Envoy::Upstream::Host> > >::emplace_back
3.83% [kernel] [k] do_syscall_64
On a production instance, we've seen this going > 80% when endpoints were churning (our biggest cluster has > 2000 endpoints, each with 2 or 3 metadata strings).
If we revert 09c5d35, the calls to EdfLoadBalancerBase::refresh() aren't noticeable anymore.
@mattklein123 have you seen anything similar?
cc: @derekargueta
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
questionQuestions that are neither investigations, bugs, nor enhancementsQuestions that are neither investigations, bugs, nor enhancementsstalestalebot believes this issue/PR has not been touched recentlystalebot believes this issue/PR has not been touched recently