Skip to content

perf regression  #3790

@rgs1

Description

@rgs1

We are heavy users of endpoints with metadata and after we started running a build that includes 09c5d35 we are seeing EdfLoadBalancerBase::refresh() consume > 10% of CPU on a totally idle envoy instance. From perf top:

  13.59%  envoy                                    [.] Envoy::Upstream::EdfLoadBalancerBase::refresh(unsigned int)::{lambda(Envoy::Upstream::ZoneAwareLoadBalancerBase::HostsSourc
   8.63%  envoy                                    [.] Envoy::Config::Metadata::metadataValue
   4.11%  envoy                                    [.] std::vector<std::shared_ptr<Envoy::Upstream::Host>, std::allocator<std::shared_ptr<Envoy::Upstream::Host> > >::emplace_back
   3.83%  [kernel]                                 [k] do_syscall_64

On a production instance, we've seen this going > 80% when endpoints were churning (our biggest cluster has > 2000 endpoints, each with 2 or 3 metadata strings).

If we revert 09c5d35, the calls to EdfLoadBalancerBase::refresh() aren't noticeable anymore.

@mattklein123 have you seen anything similar?

cc: @derekargueta

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionQuestions that are neither investigations, bugs, nor enhancementsstalestalebot believes this issue/PR has not been touched recently

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions