-
Notifications
You must be signed in to change notification settings - Fork 24.4k
Description
The problem
Redis has a well known issue of mixing both dataset memory and other memory overheads in it's memory accounting so that both trigger eviction.
One very common case is that client buffers (usually output buffers) eat a lot of memory and cause massive eviction.
This is mitigated by the client-output-buffer-limit config which disconnects a client if a single client consumed too much. but a common case is where many clients are connected at the same time, each consuming a big chunk of memory, which together makes a lot of memory and that can induce massive key eviction.
Description of the feature
Redis already counts (most of) the client buffers (the argv buffer is missing, see #5159).
What can be done is to match the total sum of all client related buffers, if it reaches over a limit, find a (random / oldest / newest) client, and disconnect it (keep doing that in a loop until we're under the limit).
Additional information
Another common case that causes eviction is dict rehashing, but we can't really solve that since redis needs to respect the memory limit and avoid eating far more memory than it is allowed to.
Related issues: #1880, #5613, #4213, #5975, #4496
Note that we should not attempt to exclude the client memory from the key eviction. if the clients are consuming less than the threshold, then that's an acceptable (steady) state, and redis still needs to respect it's maxmemory limit.
Related issues: #1618, #4668, #4583