Add FUNCTION DUMP and RESTORE. #9938
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added 2 new FUNCTION sub-commands: 1. FUNCTION DUMP - dump a binary blob representation of all the functions. 2. FUNCTION RESTORE - give the binary blob extracted using FUNCTION DUMP, restore all the functions on the given blob. Drop existing functions if exists. Modify `redis-cli --cluster add-node` to use FUNCTION DUMP to get existing functions from one of the nodes in the cluster, and FUNCTION RESTORE to load the same set of functions to the new node. redis-cli will execute this step before sending the `CLUSTER MEET` command to the new node. If FUNCTION DUMP returns an error, assume the current Redis version do not support functions and skip FUNCTION RESTORE. If FUNCTION RESTORE fails, abort and do not send the `CLUSTER MEET` command.
oranagra
reviewed
Dec 14, 2021
sundb
reviewed
Dec 15, 2021
sundb
reviewed
Dec 16, 2021
madolson
reviewed
Dec 16, 2021
Merged
3 tasks
added 5 commits
December 19, 2021 12:01
* Added restore policy argument to FUNCTION RESTORE commands (either FLUSH, APPEND or REPLACE). * Small consmetic and bug fixes.
oranagra
reviewed
Dec 19, 2021
added 2 commits
December 19, 2021 19:00
* Do not count on 'err' out parameter. * Change 'err' out parameter on 'rdbFunctionLoad' to be optional. * Replace line comment on redis-cli.
oranagra
reviewed
Dec 19, 2021
Changed restore policy to be the last argument on 'FUNCTION RESTORE'.
oranagra
reviewed
Dec 19, 2021
oranagra
reviewed
Dec 19, 2021
* Change default restore policy to APPEND. * Added restore policy argument to function-restore.json * documentation fixes.
oranagra
approved these changes
Dec 19, 2021
Member
|
@redis/core-team please approve |
sundb
reviewed
Dec 20, 2021
sundb
reviewed
Dec 20, 2021
Co-authored-by: sundb <sundbcn@gmail.com>
yossigo
reviewed
Dec 20, 2021
yossigo
approved these changes
Dec 21, 2021
added 4 commits
December 21, 2021 22:10
* Changed FUNCTION RESTORE to be WRITE command instead of MAY_REPLICATE * Added command description to FUNCTION DUMP and RESTORE on command json * Added documentation to describe FUNCTION DUMP serialization format * Changed RESTORE_POLICY_* to enum * Change blob terminology to payload
* Added test to verify 'redis-cli --cluster add-node' uploads existing functions to the new node.
oranagra
approved these changes
Dec 22, 2021
Co-authored-by: Oran Agra <oran@redislabs.com>
madolson
approved these changes
Dec 23, 2021
Contributor
madolson
left a comment
There was a problem hiding this comment.
The new cluster test LGTM, also approving the two new API designs. Left some other minor comments.
1. redis-cli will verify there is no functions on the new node before adding it to the cluster (on redis-cli --cluster add-node) 2. Drop unneeded variables on the new test 3. Use redis_client instead of redis_deferring_client (on new test) 4. Better naming new test
soloestoy
approved these changes
Dec 24, 2021
Contributor
soloestoy
left a comment
There was a problem hiding this comment.
FUNCTION RESTORE [FLUSH|APPEND|REPLACE]
APPEND: appends the restored functions to the existing functions. On collision, abort.
just one question, does it satisfy atomicity? the abort means rollback or just abort?
Member
if any conflicts are detected, the operation is aborted without any modification to the server. |
oranagra
reviewed
Dec 24, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow the conclusions here: #9899
This PR handles point 1,2 on the action items list: #9899 (comment)
Added 2 new FUNCTION sub-commands:
FUNCTION DUMP- dump a binary payload representation of all the functions.FUNCTION RESTORE <PAYLOAD> [FLUSH|APPEND|REPLACE]- give the binary payload extracted usingFUNCTION DUMP, restore all the functions on the given payload. Restore policy can be given to control how to handle existing functions (default is APPEND):Modify
redis-cli --cluster add-nodeto useFUNCTION DUMPto get existing functions from one of the nodes in the cluster, andFUNCTION RESTOREto load the same set of functions to the new node.redis-cliwill execute this step before sending theCLUSTER MEETcommand to the new node. IfFUNCTION DUMPreturns an error, assume the current Redis version do not support functions and skipFUNCTION RESTORE. IfFUNCTION RESTOREfails, abort and do not send theCLUSTER MEETcommand. If the new node already contains functions (before theFUNCTION RESTOREis sent), abort and do not add the node to the cluster. Test was added to verifyredis-cli --cluster add-nodeworks as expected.todo: