Skip to content

Lower allocation overhead of metrics in peer.Send #2840

New issue
New issue
Closed
#3411
#2851
Closed
Lower allocation overhead of metrics in peer.Send#2840
#3411
#2851
Labels
enhancementNew feature or requestmetricsp2p
@ValarDragon

Description

@ValarDragon
ValarDragon
opened on Apr 18, 2024

Feature Request

Summary

In Memory profiles of full nodes, we see large amounts of heap allocation in p2p.send coming from Prometheus logging, that is ~entirely duplicate work.
image

This is coming from these lines:

cometbft/p2p/peer.go

Lines 284 to 292 in b846639

if res {
labels := []string{
"peer_id", string(p.ID()),
"chID", fmt.Sprintf("%#x", chID),
}
p.metrics.PeerSendBytesTotal.With(labels...).Add(float64(len(msgBytes)))
p.metrics.MessageSendBytesTotal.With("message_type", metricLabelValue).Add(float64(len(msgBytes)))
}
return res

We should be able to have the labels already cached (or at minimum the peer ID label cached) in the existing peer.Metrics instance.

Problem Definition

Lower heap allocation and time overhead of sending something to peers. I have no idea how much this is a dominant term in wall clock time, though I do see recv time and send time routinely in CPU profiles.

Proposal

Cache the peer ID label in p.metrics.PeerSendBytesTotal, so this is re-done on every packet send

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestmetricsp2p

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions