Skip to content

[NEW] Client "eviction" - drop clients when their total buffer overhead is over a limit #7676

@oranagra

Description

@oranagra

The problem

Redis has a well known issue of mixing both dataset memory and other memory overheads in it's memory accounting so that both trigger eviction.

One very common case is that client buffers (usually output buffers) eat a lot of memory and cause massive eviction.
This is mitigated by the client-output-buffer-limit config which disconnects a client if a single client consumed too much. but a common case is where many clients are connected at the same time, each consuming a big chunk of memory, which together makes a lot of memory and that can induce massive key eviction.

Description of the feature

Redis already counts (most of) the client buffers (the argv buffer is missing, see #5159).
What can be done is to match the total sum of all client related buffers, if it reaches over a limit, find a (random / oldest / newest) client, and disconnect it (keep doing that in a loop until we're under the limit).

Additional information

Another common case that causes eviction is dict rehashing, but we can't really solve that since redis needs to respect the memory limit and avoid eating far more memory than it is allowed to.
Related issues: #1880, #5613, #4213, #5975, #4496

Note that we should not attempt to exclude the client memory from the key eviction. if the clients are consuming less than the threshold, then that's an acceptable (steady) state, and redis still needs to respect it's maxmemory limit.
Related issues: #1618, #4668, #4583

Metadata

Metadata

Labels

state:major-decisionRequires core team consensusstate:needs-designthe solution is not obvious and some effort should be made to design it

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions