Before evicted and before expired server events are not executed inside an execution unit. #12733
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…de an execution unit. This [PR](redis#9406) introduces new server event, `RedisModuleEvent_Key`. This new event allows to read a key data just before it removed from the database (either deleted, expired, evicted, or overwritten). When the key is removed from the database, either by active expire or eviction. The new event was not called as part of an execution unit. This can cause an issue if the module registers a post notification job inside the event. This job will not be executed atomically with the expiration/eviction operation and will not replicated inside a Multi/Exec. Moreover, the post notification job will be executed right after the event where it is still not safe to perform any write operation, this will violate the promise that post notification job will be called atomically with the operation that triggered it and **only when it is safe to write**. The PR fixes the issue by wrapping each expiration/eviction of a key with an execution unit. This make sure the entire operation will run atomically and all the post notification jobs will be executed at the end where it safe to write. Tests were modified to verify the fix.
oranagra
reviewed
Nov 6, 2023
added 3 commits
November 7, 2023 11:04
oranagra
approved these changes
Nov 8, 2023
enjoy-binbin
reviewed
Nov 8, 2023
Member
|
@enjoy-binbin I think you're right. please make a PR. |
enjoy-binbin
added a commit
to enjoy-binbin/redis
that referenced
this pull request
Nov 9, 2023
…unit This is a follow-up fix to redis#12733. We need to apply the same changes to delKeysInSlot. Refer to redis#12733 for more details.
oranagra
pushed a commit
that referenced
this pull request
Dec 11, 2023
…unit (#12745) This is a follow-up fix to #12733. We need to apply the same changes to delKeysInSlot. Refer to #12733 for more details. This PR contains some other minor cleanups / improvements to the test suite and docs. It uses the postnotifications test module in a cluster mode test which revealed a leak in the test module (fixed).
Merged
oranagra
pushed a commit
that referenced
this pull request
Jan 9, 2024
…de an execution unit. (#12733) Redis 7.2 (#9406) introduced a new modules event, `RedisModuleEvent_Key`. This new event allows the module to read the key data just before it is removed from the database (either deleted, expired, evicted, or overwritten). When the key is removed from the database, either by active expire or eviction. The new event was not called as part of an execution unit. This can cause an issue if the module registers a post notification job inside the event. This job will not be executed atomically with the expiration/eviction operation and will not replicated inside a Multi/Exec. Moreover, the post notification job will be executed right after the event where it is still not safe to perform any write operation, this will violate the promise that post notification job will be called atomically with the operation that triggered it and **only when it is safe to write**. This PR fixes the issue by wrapping each expiration/eviction of a key with an execution unit. This makes sure the entire operation will run atomically and all the post notification jobs will be executed at the end where it is safe to write. Tests were modified to verify the fix. (cherry picked from commit 0ffb9d2)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Redis 7.2 (#9406) introduced a new modules event,
RedisModuleEvent_Key. This new event allows the module to read the key data just before it is removed from the database (either deleted, expired, evicted, or overwritten).When the key is removed from the database, either by active expire or eviction. The new event was not called as part of an execution unit. This can cause an issue if the module registers a post notification job inside the event. This job will not be executed atomically with the expiration/eviction operation and will not replicated inside a Multi/Exec. Moreover, the post notification job will be executed right after the event where it is still not safe to perform any write operation, this will violate the promise that post notification job will be called atomically with the operation that triggered it and only when it is safe to write.
This PR fixes the issue by wrapping each expiration/eviction of a key with an execution unit. This makes sure the entire operation will run atomically and all the post notification jobs will be executed at the end where it is safe to write.
Tests were modified to verify the fix.