Skip to content

Publish rate limit on broker to avoid OOM #5513

@jiazhai

Description

@jiazhai

Out Of Memory is seen frequently while client produced message too quickly and used all the direct memory.
This happens like this:

  1. Client send message to broker;
  2. Broker call bookkeeper client to write message to Bookie, and hold the direct memory until write success;
    If in the 2nd step the direct memory not get released, then the Broker will get OOM.

It would be great to have a way to detect the memory pressure, and do the publish rate limit if lack of memory; once the memory get back, cancel the rate limit.

Metadata

Metadata

Assignees

Labels

type/featureThe PR added a new feature or issue requested a new feature

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions