Skip to content

[pubsub] there is too highly memory usage when creating many events subscriptions #6439

@banishee

Description

@banishee

Tendermint version (use tendermint version or git rev-parse --verify HEAD if installed from source):
before a91680e

What happened:
When I use a bot to make lots of events subscriptions to a node, I noticed that there is a huge increasing memory usage.
Then I checked the code, and I found there is an unreasonable initial slice with the math.MaxInt16 length.

tree := tokens32{tree: make([]token32, math.MaxInt16)}

For almost over ten thousand subscription clients, the node will take over 25 GB memory
image

What you expected to happen:
I think it is better to lower the initial length, such as 16 (the length of most subscription types I tested are only using less than 16 )

Have you tried the latest version: yes/no
no

How to reproduce it (as minimally and precisely as possible):

  • run a local node
  • make many subscriptions

Metadata

Metadata

Assignees

No one assigned

    Labels

    stalefor use by stalebot

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions