-
Notifications
You must be signed in to change notification settings - Fork 24.4k
Don't run command filter on blocked command reprocessing #11895
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Previously we would run the module command filters even upon blocked command reprocessing. This could modify the command, and it's args. This is irrelevant in the context of a command being reprocessed (it already went through the filters), as well as breaks the cahced command lookup that exists in the case of a reprocessed command. fixes redis#11894.
|
trying to work on it right now, trying to figure out, how to get it to work in tcl, my command filter (added to commandfilter.c) is |
|
actually, blpop isn't a good case, as it always gets timeout from last arg. need to use brpoplpush as it gets it from a fixed position arg. |
you can also use BLMPOP... easier still |
|
added a test, but while my new test passes, older existing tests are seeing failures, unsure why at the moment. |
tests/modules/commandfilter.c
Outdated
|
|
||
| brpoplpush_count++; | ||
|
|
||
| if (brpoplpush_count % 2 != 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure about this methodology. This is for tests purpose so there is no real harm, but in case the same function will be used in cases of timeout brpoplpush the counter will be messed up. can you just do an RM_CALL to incr some variable and check it's value after the client is unblocked?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so I agree that its not great for general purpose (heck, 2 blockers, the 2nd wouldn't necc execute to perhaps the 3rd pass through). The thought was just making some easiy predictable for testing purposes. anything else just seems to complicate it beyond what's needed for testing purposes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can make one that reverses the direction.
i.e. changes BLMOVE RIGHT LEFT to BLMOVE LEFT RIGHT (swap the two arguments).
then to wake up the command we push two elements to the list, and watch which one is moved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@oranagra , I'll try that.
|
@sjpotter I wonder if we can conclude that it's always wrong to call the filter on reprocessing, as it depends on what the filter tries to achieve. Some ideas:
|
|
@yossigo, don't necessarily disagree with your post, some other things that could be added.
With all that said, what is the logic for running it again on reprocess? i.e. on first time through, the cmd did or didn't get modified, "reprocess" is an internal redis implementation detail, why couldn't it be implemented differently. Lets say blocking instead was handled at the blpopCommand level, it wouldn't ever reprocess. I'm trying to come up with a use case where the user would want to do that. With that said, I'm not against the idea of being overly more flexible (and as noted, give ideas of more than can be done). |
|
i personally agree with the above. while we can do all these things, we don't have a need for that yet, and we can keep it working the same as it was before we change the internals. |
|
I've discussed this with Yossi, we agree that the extensions described above can / should be postponed to some future date, and for now we just need to make sure the internals change doesn't break previous behavior. |
tests/modules/commandfilter.c
Outdated
|
|
||
| brpoplpush_count++; | ||
|
|
||
| if (brpoplpush_count % 2 != 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can make one that reverses the direction.
i.e. changes BLMOVE RIGHT LEFT to BLMOVE LEFT RIGHT (swap the two arguments).
then to wake up the command we push two elements to the list, and watch which one is moved.
| RedisModuleString *dir1 = RedisModule_HoldString(NULL, RedisModule_CommandFilterArgGet(filter, 3)); | ||
| RedisModuleString *dir2 = RedisModule_HoldString(NULL, RedisModule_CommandFilterArgGet(filter, 4)); | ||
| RedisModule_CommandFilterArgReplace(filter, 3, dir2); | ||
| RedisModule_CommandFilterArgReplace(filter, 4, dir1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you need to explain what you're doing here and why, not just the concern about freeing
|
@sjpotter i saw some missing comments in your command filter PR and decided to edit them instead of bothering you. |
Previously we would run the module command filters even upon blocked command reprocessing. This could modify the command, and it's args. This is irrelevant in the context of a command being reprocessed (it already went through the filters), as well as breaks the crashed command lookup that exists in the case of a reprocessed command.
fixes #11894.