Skip to content

Conversation

@sundb
Copy link
Collaborator

@sundb sundb commented Dec 26, 2022

This call is introduced in #8687, but became irrelevant in #11348, and is currently a no-op.
The fact is that #11348 an unintended side effect, which is that even if the client eviction config
is enabled, there are certain types of clients for which memory consumption is not accurately
tracked, and so unlike normal clients, their memory isn't reported correctly in INFO.

while((ln = listNext(&li))) {
client *monitor = ln->value;
addReply(monitor,cmdobj);
updateClientMemUsageAndBucket(c);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we meant to update the memory of the client monitor, not c (in which case this line should be modified, not deleted).
the thing is that we normally update client memory in processCommandAndResetClient (after processCommand), so there's no need to call it per addReply, but in this case (feeding monitors), much like in pubsubPublishMessageInternal, we add a reply for the non-current client, so we need to call updateClientMemUsageAndBucket.

Copy link
Collaborator Author

@sundb sundb Dec 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But monitors are never evicted (pubsub clients can be evicted), so updateClientMemUsageAndBucket() will always be NOP.

    int allow_eviction = clientEvictionAllowed(c);
    removeClientFromMemUsageBucket(c, allow_eviction);

    if (!allow_eviction) {
        return 0;    <- exit here
    }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting, that seems like a side effect of #11348.
in the original #8687, calling updateClientMemUsage used to update the client memory usage metrics.
now with #11348, it'll never do that for a monitor (and master / slave), even if client eviction is enabled.
considering the accurate memory accounting was mainly intended for the eviction, i suppose we're ok with this change.
@hpatro @yoav-steinberg what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the statement is pretty much a no-op right now, I think we can safely clean it up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hpatro it is a no-op right now, but the question was if it should be.
i.e. i realized that your PR had implications on accurate memory reporting of certain types of clients even when client eviction is enabled.

i.e. it means that when enabling client eviction, the user enjoys accurate memory tracking of most clients, but monitor/master/slave remain inaccurate.
we can fix that if we want, but i'm leaning towards accepting it.

i wonder if we'll remember to re-introduce that line if we ever decide to change that..

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hpatro I see that you've handled this in #11348 by calling updateClientMemoryUsage() in clientCron() to update the client's memory usage even when eviction is disabled.

redis/src/server.c

Lines 1014 to 1016 in 7379d22

* If client eviction is enabled, update the bucket as well. */
if (!updateClientMemUsageAndBucket(c))
updateClientMemoryUsage(c);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hpatro to be clear, what i meant is that there's a benefit of accurate client memory tracking, not only for the purpose of client eviction.
i.e. when someone calls INFO, and sees a certain amount of memory being used, they'll know how much of it belongs to clients rather than having an outdated value.
in my previous understanding your PR meant that we have inaccurate reporting while client eviction is disabled, and we have accurate reporting when it is enabled.
but now i realize that for certain types of clients, it'll remain inaccurate regardless of the client eviction config.
i'm trying to decide i we wanna fix it.

@sundb yes, that's what he meant, that all we need to do is remove the line you removed in this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when someone calls INFO, and sees a certain amount of memory being used, they'll know how much of it belongs to clients rather than having an outdated value.

I don't see much value of accurate memory usage date for just observability. @oranagra Are there any other benefits you can think of?
I think we should continue with our current solution and accept this PR to clean up the unnecessary call.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i can't think of other benefits, in some cases, when looking at INFO to diagnose an issue, it is important to have correct metrics.
however, the fact is that by default they're not correct, and people need to enable client-eviction to have them correct means that in most cases it's not gonna serve us.

so let's just add a comment in updateClientMemUsageAndBucket that explains that currently these types of clients are not accurately reported in INFO regardless of the config, and that if we someday want to improve that we need to re-add a call in replicationFeedMonitors.
@sundb can you do that? or do you want me to edit this PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oranagra Please look at the last commit.
Not sure it's accurate, comment is much harder than code...

@oranagra oranagra merged commit af0a4fe into redis:unstable Dec 28, 2022
@sundb sundb deleted the remove_unnecessary_updateClientMemUsageAndBucket branch December 29, 2022 01:33
oranagra pushed a commit that referenced this pull request Jan 16, 2023
…ors (#11657)

This call is introduced in #8687, but became irrelevant in #11348, and is currently a no-op.
The fact is that #11348 an unintended side effect, which is that even if the client eviction config
is enabled, there are certain types of clients for which memory consumption is not accurately
tracked, and so unlike normal clients, their memory isn't reported correctly in INFO.

(cherry picked from commit af0a4fe)
oranagra pushed a commit that referenced this pull request Jul 25, 2023
A bug introduced in #11657 (7.2 RC1), causes client-eviction (#8687)
and INFO to have inaccurate memory usage metrics of MONITOR clients.

Because the type in `c->type` and the type in `getClientType()` are confusing
(in the later, `CLIENT_TYPE_NORMAL` not `CLIENT_TYPE_SLAVE`), the comment
we wrote in `updateClientMemUsageAndBucket` was wrong, and in fact that function
didn't skip monitor clients.
And since it doesn't skip monitor clients, it was wrong to delete the call for it from
`replicationFeedMonitors` (it wasn't a NOP).
That deletion could mean that the monitor client memory usage is not always up to
date (updated less frequently, but still a candidate for client eviction).
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…ors (redis#11657)

This call is introduced in redis#8687, but became irrelevant in redis#11348, and is currently a no-op.
The fact is that redis#11348 an unintended side effect, which is that even if the client eviction config
is enabled, there are certain types of clients for which memory consumption is not accurately
tracked, and so unlike normal clients, their memory isn't reported correctly in INFO.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Unreleased

Development

Successfully merging this pull request may close these issues.

3 participants