Redis Functions - Libraries

Redis Functions PR introduces a new scripting approach. One of the common requests that came from the community is the ability to share code between functions, the following is a proposal of how users can achieve code sharing between different functions.

Before describing the solution we will point out some issues that lead to the idea we will describe later.

1. Code sharing can raise issues like version incompatibility, Assuming function 1 uses code from function 2. Function 2 might update the code in a way that will break function 1 invocation.
2. Code sharing between engines is problematic and can only be done using the FCALL command that will call another function. This raises (again) the topic of nesting function calls which are currently not supported. We keep it out of the scope of this proposal and will be discussed on another issue.
3. Name-spacing, 2 functions might declare same function names internally and this requires name-spacing to identify between them.

those points led us to these conclusions:

1. Code sharing between functions on the same engine should be done only by calling the other function entry point. The assumption is that entry points are less likely to introduce a drastic behaviour change because it is also exposed to clients. This proposal does not cover this case and keep it for future improvements.
2. Exposing one function entry point to another function is not enough. What if we want 2 functions to have a shared code which is not a function by itself (and one that uses a richer interface, like passing native language objects as argument)? To solve it, we introduce a new ability that allows batch loading of functions.

The Idea

Instead of allowing a single function creation per FUNCTION CREATE command, we suggest the option to create multiple functions in a single command (for example look at Lua library API section below). Functions that were created together can safely share code between each other without worrying about compatibility issues and versioning. We call this group of functions that was uploaded together a library. Library code is loaded into Redis using the FUNCTION LOAD command. When an engine receives the code given on FUNCTION LOAD command it will be able to register one or more functions. Each engine can decide how to expose the registration capability to the user, some engines will need to run parts of the code in order to know which functions it creates while others will only need to introspect the code.
This new concept also changes the way FUNCTIONS are deleted or updated. Libraries are immutable so it is not possible to delete or update a single function inside a library, the entire library needs to be updated/deleted together.

API Changes (compare to unstable)

The following API changes/additions will be made to support library:

New API

• FUNCTION LOAD <ENGINE NAME> <LIBRARY NAME> [DESCRIPTION <LIBRARY DESCRIPTION>] [REPLACE] <LIBRARY CODE>
  Load a new library into Redis. Arguments:
  • ENGINE NAME - the name of the engine to load the library
  • LIBRARY NAME - the name of the library
  • DESCRIPTION - optional description of the library
  • REPLACE - optional, indicate to replace the library with the same name if exists
  • LIBRARY CODE - library code

Changed API

• FUNCTION DELETE <LIBRARY NAME>
  Function delete will get the library name to delete instead of the function name.

• FUNCTION INFO <Library name> [WITHCODE]
  FUNCTION INFO returns information about libraries and not a single function. If the optional WITHCODE argument is given, the library code is also returned. The information given:

  • NAME
  • ENGINE
  • DESCRIPTION
  • FUNCTIONS LIST

• FUNCTION LIST
  Lists all the libraries. For each library list also the functions it created.

Deleted API

• FUNCTION CREATE

Notice: For easier migration from eval to functions, we can decide to keep the FUNCTION CREATE and treat it as a library with one function. To avoid confusion we believe it's better to drop it but it's debatable.

Lua Library API

When the Lua engine will get the code given to FUNCTIONS LOAD command it will immediately run it. In this run context it will be possible to call redis.register_function. This API will register a new function (with the given arguments that we will describe shortly) and link it to the current library. The arguments given to redis.register_function are:

• Function name - the name of the new function to register
• Function object - a Lua function object to call when the new function is invoke
• Function flags - optional flags as describe here
• Function description - optional text describe the function

The redis.register_function will only be available on FUNCTIONS LOAD invocation and will raise an error if used anywhere else. Also, during this function's registration phase, it will not be impossible to call Redis commands (redis.call) or interact with the key space in any way.

Lua library Example

The following example register a library with 2 functions test1 and test2 that simply return 2 and 3 respectively

> FUNCTIONS LOAD LUA lib1 “
    local function add1(x)
        return x + 1
    end

    local function test1()
        return add1(1)
    end

    local function test2()
        return add1(2)
    end

    redis.register_function(‘test1’, test1)
    redis.register_function(‘test2’, test2)
”
  OK
> FCALL test1 0
  2
> FCALL test2 0
  3

Library Functions Flags

We suggest support flags that can be given to Redis when the library creates a function. Those flags are used to tell Redis when it is OK to invoke the function. For example, one flag might indicate to disallow running the function if Redis reached its memory limit. Here is the list of flags we plan to support first (other flags might be added in the future):

• DENY_OOM - do not allow the function to be invoked if the memory limit is reached.
• BREAK_ATOMICITY - indicate that it's OK to kill the function even if it already performed a write command or disallow write * commands if OOM reached.
• NO_REPLICATE - indicate that all the data created by the function is temporary and should not be replicated to replicas/aof. Equivalent to redis.set_repl.

Note: DENY_OOM can solve this issue: #8478