Skip to content

Conversation

@nichchun
Copy link
Contributor

@nichchun nichchun commented Feb 11, 2022

Implementation based off of #9437

Implementation details:

This feature adds the ability to add four different types (Bool, Numeric, String, Enum) of configurations to a module to be accessed via the redis config file, and the CONFIG command.

Configuration Names:

We impose a restriction that a module configuration always starts with the module name and contains a '.' followed by the config name. If a module passes "config1" as the name to a register function, it will be registered as MODULENAME.config1.

Configuration Persistence:

Module Configurations exist only as long as a module is loaded. If a module is unloaded, the configurations are removed.
There is now also a minimal core API for removal of standardConfig objects from configs by name.

Get and Set Callbacks:

Storage of config values is owned by the module that registers them, and provides callbacks for Redis to access and manipulate the values. This is exposed through a GET and SET callback.

The get callback returns a typed value of the config to redis. The callback takes the name of the configuration, and also a privdata pointer. Note that these only take the CONFIGNAME portion of the config, not the entire MODULENAME.CONFIGNAME.

 typedef RedisModuleString * (*RedisModuleConfigGetStringFunc)(const char *name, void *privdata);
 typedef long long (*RedisModuleConfigGetNumericFunc)(const char *name, void *privdata);
 typedef int (*RedisModuleConfigGetBoolFunc)(const char *name, void *privdata);
 typedef int (*RedisModuleConfigGetEnumFunc)(const char *name, void *privdata);

Configs must also must specify a set callback, i.e. what to do on a CONFIG SET XYZ 123 or when loading configurations from cli/.conf file matching these typedefs. name is again just the CONFIGNAME portion, val is the parsed value from the core, privdata is the registration time privdata pointer, and err is for providing errors to a client.

typedef int (*RedisModuleConfigSetStringFunc)(const char *name, RedisModuleString *val, void *privdata, RedisModuleString **err);
typedef int (*RedisModuleConfigSetNumericFunc)(const char *name, long long val, void *privdata, RedisModuleString **err);
typedef int (*RedisModuleConfigSetBoolFunc)(const char *name, int val, void *privdata, RedisModuleString **err);
typedef int (*RedisModuleConfigSetEnumFunc)(const char *name, int val, void *privdata, RedisModuleString **err);

Modules can also specify an optional apply callback that will be called after value(s) have been set via CONFIG SET:

typedef int (*RedisModuleConfigApplyFunc)(RedisModuleCtx *ctx, void *privdata, RedisModuleString **err);

Flags:
We expose 7 new flags to the module, which are used as part of the config registration.

#define REDISMODULE_CONFIG_MODIFIABLE 0 /* This is the default for a module config. */
#define REDISMODULE_CONFIG_IMMUTABLE (1ULL<<0) /* Can this value only be set at startup? */
#define REDISMODULE_CONFIG_SENSITIVE (1ULL<<1) /* Does this value contain sensitive information */
#define REDISMODULE_CONFIG_HIDDEN (1ULL<<4) /* This config is hidden in `config get <pattern>` (used for tests/debugging) */
#define REDISMODULE_CONFIG_PROTECTED (1ULL<<5) /* Becomes immutable if enable-protected-configs is enabled. */
#define REDISMODULE_CONFIG_DENY_LOADING (1ULL<<6) /* This config is forbidden during loading. */
/* Numeric Specific Configs */
#define REDISMODULE_CONFIG_MEMORY (1ULL<<7) /* Indicates if this value can be set as a memory value */

Module Registration APIs:

int (*RedisModule_RegisterBoolConfig)(RedisModuleCtx *ctx, char *name, int default_val, unsigned int flags, RedisModuleConfigGetBoolFunc getfn, RedisModuleConfigSetBoolFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
int (*RedisModule_RegisterNumericConfig)(RedisModuleCtx *ctx, const char *name, long long default_val, unsigned int flags, long long min, long long max, RedisModuleConfigGetNumericFunc getfn, RedisModuleConfigSetNumericFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
int (*RedisModule_RegisterStringConfig)(RedisModuleCtx *ctx, const char *name, const char *default_val, unsigned int flags, RedisModuleConfigGetStringFunc getfn, RedisModuleConfigSetStringFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
int (*RedisModule_RegisterEnumConfig)(RedisModuleCtx *ctx, const char *name, int default_val, unsigned int flags, const char **enum_values, const int *int_values, int num_enum_vals, RedisModuleConfigGetEnumFunc getfn, RedisModuleConfigSetEnumFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
int (*RedisModule_LoadConfigs)(RedisModuleCtx *ctx);

The module name will be auto appended along with a "." to the front of the name of the config.

What RM_Register[...]Config does:

A RedisModule struct now keeps a list of ModuleConfig objects which look like:

typedef struct ModuleConfig {
    sds name; /* Name of config without the module name appended to the front */
    void *privdata; /* Optional data passed into the module config callbacks */
    union get_fn { /* The get callback specificed by the module */
        RedisModuleConfigGetStringFunc get_string;
        RedisModuleConfigGetNumericFunc get_numeric;
        RedisModuleConfigGetBoolFunc get_bool;
        RedisModuleConfigGetEnumFunc get_enum;
    } get_fn;
    union set_fn { /* The set callback specified by the module */
        RedisModuleConfigSetStringFunc set_string;
        RedisModuleConfigSetNumericFunc set_numeric;
        RedisModuleConfigSetBoolFunc set_bool;
        RedisModuleConfigSetEnumFunc set_enum;
    } set_fn;
    RedisModuleConfigApplyFunc apply_fn;
    RedisModule *module;
} ModuleConfig;

It also registers a standardConfig in the configs array, with a pointer to the ModuleConfig object associated with it.

What happens on a CONFIG GET/SET MODULENAME.MODULECONFIG:

For CONFIG SET, we do the same parsing as is done in config.c and pass that as the argument to the module set callback. For CONFIG GET, we call the module get callback and return that value to config.c to return to a client.

CONFIG REWRITE:

Starting up a server with module configurations in a .conf file but no module load directive will fail. The flip side is also true, specifying a module load and a bunch of module configurations will load those configurations in using the module defined set callbacks on a RM_LoadConfigs call. Configs being rewritten works the same way as it does for standard configs, as the module has the ability to specify a default value. If a module is unloaded with configurations specified in the .conf file those configurations will be commented out from the .conf file on the next config rewrite.

RM_LoadConfigs:

RedisModule_LoadConfigs(RedisModuleCtx *ctx);

This last API is used to make configs available within the onLoad() after they have been registered. The expected usage is that a module will register all of its configs, then call LoadConfigs to trigger all of the set callbacks, and then can error out if any of them were malformed. LoadConfigs will attempt to set all configs registered to either a .conf file argument/loadex argument or their default value if an argument is not specified. LoadConfigs is a required function if configs are registered. Also note that LoadConfigs does not call the apply callbacks, but a module can do that directly after the LoadConfigs call.

New Command: MODULE LOADEX [CONFIG NAME VALUE] [ARGS ...]:

This command provides the ability to provide startup context information to a module. LOADEX stands for "load extended" similar to GETEX. Note that provided config names need the full MODULENAME.MODULECONFIG name. Any additional arguments a module might want are intended to be specified after ARGS. Everything after ARGS is passed to onLoad as RedisModuleString **argv.

Notes:
There was an ask for double, my suggestion is that you should store a double as just a string and do conversion within the module. You get into weird cases like precision with floating points that vary between systems.

@chenyang8094
Copy link
Contributor

I think Module developers will all love this feature, thanks.

@madolson madolson added 7.0-must-have state:major-decision Requires core team consensus release-notes indication that this issue needs to be mentioned in the release notes labels Feb 12, 2022
@madolson
Copy link
Contributor

@redis/core-team Please take a look at the top comment and provide feedback about the implementation. There are still a couple of checks failing in the CI, so once those are cleaned up and we have consensus on the finalized implementation I'll ping the group again.

@oranagra oranagra linked an issue Feb 12, 2022 that may be closed by this pull request
Copy link
Member

@oranagra oranagra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't reviewed the code yet, just responding to the top comment of the PR, the discussion points, and a few other things i noticed.

I decided to find places to post these as code-comments so that separate discussion threads can be handled in parallel.

Copy link
Collaborator

@yossigo yossigo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nichchun I've just had a quick look at this, so posting some initial feedback mostly based on @oranagra's and the top comment.

A config rewrite will rewrite all module configurations to the .conf file. Presently, there is no way to make assumptions about default values, so they are ALWAYS written on a config rewrite.

Not in favor of that. The problem is we lose track of explicit config vs default config, which tends to become an issue over time when defaults change. Is there a problem to expose something similar to what Redis already does, and let the module control if a value is to be rewritten or not?

Alternatives to the ApplyConfigs() API is to call the setCallback immediately after the config is registered, this was rejected because you might have dependencies between config values, and it might be useful to load them all together first.

I very briefly proposed an alternative. I am sure we need to support multi-param set as a basic feature of the API. Possibly, also support both push and pull (where pull is done by the module at OnLoad time).

@madolson
Copy link
Contributor

madolson commented Feb 15, 2022

Core group meeting decisions:

  1. We should add a new API to register an apply callback that is executed after all of values have been set during config set. This allows simple modules to define just set callbacks, but more advanced used cases should be able to register this apply config. The apply callback should provide at least a list of config names.
  2. The current Apply API is likely okay, it makes it straight forward to reason about the values.
  3. We didn't fully align on default values. The major points are:
  • We would prefer defining default values once, perhaps during registration. This value is what is compared against when doing rewrites.
  • We aren't sure if configs should be able to force being included or excluded from a rewrite.

@oranagra
Copy link
Member

few notes:

i'm not certain the new apply callback must get names of all configs that were changed (it's a complexity i think we can leave out, and let modules that care handle on their own.
i see two options here: one is to have just one global apply callback per module, and the other is to have an optional apply callback per config (in which case redis can detect that several modified configs had the same apply callback, and call it only once, like we do for our internal configs).

regarding defaults, the important requirements in my eyes are: first that we avoid making something explicit in rewrite so that we can later change default and still distinguish between explicit changes and implicit ones.
and secondly that if we do let the module provide a default value, we must avoid a case where the real default (for startup) could be different than the one used for rewrite, so i think it must be at registration time, and then the module will always get a callback at startup, and can't distinguish between an implicit callback, and a case where the user actually provided a config (i think this is acceptable)

just to mention that we agree that enums need better handling, not just an array of valid strings.

@sjpotter
Copy link
Contributor

@madolson @oranagra one thing that has come up with our work on RedisRaft is that we want to base some RedisRaft configuration based on redis configuration (tls configuration in this case). It could be nice if one could get the callback as well on general redis "config set" commands.

thoughts?

@oranagra
Copy link
Member

oranagra commented Feb 17, 2022

@sjpotter i don't think this should be part of the configuration infrastructure. i think that maybe similarly to RM_GetServerInfo and RM_GetCommandKeysWithFlags, we should add some API to access configuration (i.e. as an alternative to getting it with RM_Call("config").

And we could add add an event / hook like RedisModuleEvent_ReplicationRoleChanged, notifying modules about configuration changes.
i.e. not an event per config, but rather a generic RedisModuleEvent_ConfigChanged that carries the config name.

nichchun and others added 2 commits February 17, 2022 09:58
Co-authored-by: sundb <sundbcn@gmail.com>
Co-authored-by: sundb <sundbcn@gmail.com>
@nichchun
Copy link
Contributor Author

nichchun commented Feb 18, 2022

Few updates here. ARGS is no longer required as an argument to loadex. There's now also an optional apply callback, and a default value that can be specified during registration. Enums now work based off ints (I went with the implicit index implementation.) Renamed RM_ApplyConfigs to RM_LoadConfigs to avoid confusion with new apply callbacks. Also a few documentation updates, and removed '.' restriction in config names. module configs always have a '.', but standardConfigs in the future can as well.

…ime, ability to specify an apply function added
@nichchun
Copy link
Contributor Author

Note the forcepush was to clean up a bunch of small commits that made up the new features

Copy link
Member

@oranagra oranagra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the module API is rather solid, other than maybe:

  1. a few more flags we could expose (like PERCENT_CONFIG)
  2. improved enums configs (not relying on implicit int values).
  3. improved apply callback that takes a RedisModuleCtx and / or privdata

I posted quite a few comments about the implementation, but most importantly there's a lot of logic copied from config.c into module.c, and i think we could maybe find a way to avoid that (see my comments in parseAndSetNumericConfig).

so maybe we need to look into a POC for that approach before dealing with the test of the comments?

p.s. haven't reviewed the test code.

Copy link
Contributor

@madolson madolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I still prefer the char *, but it's not something I honestly feel that strongly about.

@oranagra
Copy link
Member

I also prefer char*, using a heap allocated string is more burden for the common use case (static errors).
if we stick with char* only the ones that need to format a string have more burden.
@yossigo please look at the last commit and state if you agree to revert it.
(if not, we should at least unify some common code)

yossigo added 2 commits March 30, 2022 11:50
The test assumed the default maxclients is always 10000, but running
under valgrind with a smaller fileno results with that value decremented
by Redis on startup.
@yossigo
Copy link
Collaborator

yossigo commented Mar 30, 2022

@oranagra As discussed, I don't feel strongly about it but do think the RedisModuleString option is uglier/more boiler plate but also more correct from an API perspective. Pushed a commit to reduce duplication and a small test fix.

@oranagra oranagra merged commit bda9d74 into redis:unstable Mar 30, 2022
@oranagra
Copy link
Member

merged 🎉
Thank you @nichchun and everyone involved.

@sundb
Copy link
Collaborator

sundb commented Mar 31, 2022

@madolson
Copy link
Contributor

@nichchun ^


int RedisModule_OnUnload(RedisModuleCtx *ctx) {
REDISMODULE_NOT_USED(ctx);
if (strval) RedisModule_FreeString(ctx, strval);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reproduced the fail test on alpine, when I comment out this line test is OK.
But I didn't find out the real reason why, no stack on alpine is too painful.

Copy link
Member

@oranagra oranagra Mar 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing I can think of is that on alpine the dynamic library loader doesn't properly inits the global (maybe when the module is unloaded and reloaded).
@sundb can you try to init that global to null from the onload function and see if it helps?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oranagra You guessed it right. strval needs to be reset to NULL after free.
Change code to following will be OK:

    if (strval) {
        RedisModule_FreeString(ctx, strval);
        strval = NULL;
    }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make a PR to fix both issues.

*err = RedisModule_CreateString(NULL, "Cannot set string to 'rejectisfreed'", 36);
return REDISMODULE_ERR;
}
RedisModule_Free(strval);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line should be wrong, it should be RedisModule_FreeString, but it's not the cause of the test failure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's interesting that valgrind doesn't report that leak (leaking the actual string).
i suppose the tcl test code doesn't set the string twice (replacing a previous one).

it's also odd that the alpine code crash on some corruption, but valgrind doesn't report anything.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i see we do have a test that does it.
so i don't know why valgrind doesn't report the leak.

        r config set moduleconfigs.string wafflewednesdays
        assert_equal [r config get moduleconfigs.string] "moduleconfigs.string wafflewednesdays"
        r config set moduleconfigs.string \x73\x75\x70\x65\x72\x20\x00\x73\x65\x63\x72\x65\x74\x20\x70\x61\x73\x73\x77\x6f\x72\x64
        assert_equal [r config get moduleconfigs.string] "moduleconfigs.string {super \0secret password}"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yoav helped me figure this out.
It must be an embedded string, so being short the robj pointer holds the sds in the same allocattion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did test and confirm this was the case, I specifically asked for the case to make sure we weren't mismanaging the allocation, forgetting about embedded strings.

@oranagra oranagra mentioned this pull request Apr 5, 2022
oranagra added a commit that referenced this pull request Jun 2, 2022
…, and inserting comments around module and acl configs (#10761)

A regression from #10285 (redis 7.0).
CONFIG REWRITE would put lines with: `include`, `rename-command`,
`user`,  `loadmodule`, and any module specific config in a comment.

For ACL `user`, `loadmodule` and module specific configs would be
re-inserted at the end (instead of updating existing lines), so the only
implication is a messy config file full of comments.

But for `rename-command` and `include`, the implication would be that
they're now missing, so a server restart would lose them.

Co-authored-by: Oran Agra <oran@redislabs.com>
tezc added a commit to RedisLabs/redisraft that referenced this pull request Jun 30, 2022
Use new module configuration API

RedisRaft adopts new configuration API: redis/redis#10285

After this commit, `RAFT.CONFIG` will not exist anymore. `CONFIG` command will be used instead. 

Usage: 

- Command: `config set raft.addr 127.0.0.1:8000` and `config get raft.id`
- CLI:   `./redis-server --port 5001 --loadmodule redisraft.so --raft.id 3 --raft.addr 127.0.0.1:8000`
- Config file: 
  ```
  ...
  port 5000
  dbfilename redis.rdb
  loadmodule /home/ozan/Desktop/redisraft/redisraft.so
  raft.addr 10.0.3.69:5022
  raft.slot-config 0:5460
  ```

------

**Breaking changes**

  1. `CONFIG` command will be used instead of `RAFT.CONFIG`  
  2. Configuration parameters will require `raft.` or `--raft.` prefix. 
 
  3. Renamed some configuration parameters:  

     | Old name  | New name |
     | ------------- | ------------- |
     | raft-log-filename  | log-filename |
     | raft-interval | periodic-interval |
     | raft-response-timeout | response-timeout |
     | raft-log-max-cache-size | log-max-cache-size |
     | raft-log-max-file-size | log-max-file-size |
     | raft-log-fsync | log-fsync | 

  4.  `join-timeout` config was in seconds. Now, it is in milliseconds.
       `connection-timeout` config was in microseconds. Now, it is in milliseconds.
MeirShpilraien pushed a commit to RedisLabsModules/redismodule-rs that referenced this pull request Mar 26, 2023
The PR adds the ability to add module configuration as using configuration API introduced on Redis 7: redis/redis#10285

The configuration is added optionally on `redis_module!` macro under `configurations` list that can contains the following subsections:

* `i64` configuration:
A list of `i64` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <min value>, <max value>, <flags>, <optional on update callback>]`

* `string` configuration:
A list of `string` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <flags>, <optional on update callback>]`

* `bool` configuration:
A list of `bool` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <flags>, <optional on update callback>]`

* `enum` configuration:
A list of `enum` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <flags>, <optional on update callback>]`

An example of all the 4 options can be found under `example/configuration.rs`. Notice that `enum` configuration is of special type and require provide an `enum` that implements the following traits: `TryFrom<i32>`, `From<$name>`, `EnumConfigurationValue`, `Clone`. User can use `enum_configuration` macro to easily added those implementation on a given `enum`.

In addition, it is also possible to tell redismodule-rs to look at the module arguments as if they were configuration using `module_args_as_configuration: true/false` option.

Usage examample:

```rust
/// A macro to easily creating an enum that can be used as an enum configuration
enum_configuration! {
    enum EnumConfiguration {
        Val1 = 1,
        Val2 = 2,
    }
}

/// Static variable that can be used as configuration and will be automatically set when `config set` command is used.
/// All the variables must be somehow thread safe protected, either as atomic variable, `RedisGILGuard` or `Mutex`.
lazy_static! {
    static ref CONFIGURATION_I64: RedisGILGuard<i64> = RedisGILGuard::default();
    static ref CONFIGURATION_ATOMIC_I64: AtomicI64 = AtomicI64::new(1);
    static ref CONFIGURATION_REDIS_STRING: RedisGILGuard<RedisString> =
        RedisGILGuard::new(RedisString::create(None, "default"));
    static ref CONFIGURATION_STRING: RedisGILGuard<String> = RedisGILGuard::new("default".into());
    static ref CONFIGURATION_MUTEX_STRING: Mutex<String> = Mutex::new("default".into());
    static ref CONFIGURATION_ATOMIC_BOOL: AtomicBool = AtomicBool::default();
    static ref CONFIGURATION_BOOL: RedisGILGuard<bool> = RedisGILGuard::default();
    static ref CONFIGURATION_ENUM: RedisGILGuard<EnumConfiguration> =
        RedisGILGuard::new(EnumConfiguration::Val1);
    static ref CONFIGURATION_MUTEX_ENUM: Mutex<EnumConfiguration> =
        Mutex::new(EnumConfiguration::Val1);
}

/// This function will be called when a configuration change is done and will count the total number of changes.
fn num_changes(ctx: &Context, _: Vec<RedisString>) -> RedisResult {
    let val = NUM_OF_CONFIGERATION_CHANGES.lock(ctx);
    Ok(RedisValue::Integer(*val))
}

/// The module initialisation macro, gets the configuration variable and some extra data (such as configuration name and
/// default value) and pass it to Redis so Redis will set them and return their values on `config set` and `config get` respectively.
redis_module! {
    name: "configuration",
    version: 1,
    data_types: [],
    commands: [
        ["configuration.num_changes", num_changes, "", 0, 0, 0],
    ],
    configurations: [
        i64: [
            ["i64", &*CONFIGURATION_I64, 10, 0, 1000, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
            ["atomic_i64", &*CONFIGURATION_ATOMIC_I64, 10, 0, 1000, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
        ],
        string: [
            ["redis_string", &*CONFIGURATION_REDIS_STRING, "default", ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
            ["string", &*CONFIGURATION_STRING, "default", ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed::<String, _>))],
            ["mutex_string", &*CONFIGURATION_MUTEX_STRING, "default", ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed::<String, _>))],
        ],
        bool: [
            ["atomic_bool", &*CONFIGURATION_ATOMIC_BOOL, true, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
            ["bool", &*CONFIGURATION_BOOL, true, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
        ],
        enum: [
            ["enum", &*CONFIGURATION_ENUM, EnumConfiguration::Val1, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
            ["enum_mutex", &*CONFIGURATION_MUTEX_ENUM, EnumConfiguration::Val1, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))],
        ],
        module_args_as_configuration: true, // Indication that we also want to read module arguments as configuration.
    ]
}
```
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…, and inserting comments around module and acl configs (redis#10761)

A regression from redis#10285 (redis 7.0).
CONFIG REWRITE would put lines with: `include`, `rename-command`,
`user`,  `loadmodule`, and any module specific config in a comment.

For ACL `user`, `loadmodule` and module specific configs would be
re-inserted at the end (instead of updating existing lines), so the only
implication is a messy config file full of comments.

But for `rename-command` and `include`, the implication would be that
they're now missing, so a server restart would lose them.

Co-authored-by: Oran Agra <oran@redislabs.com>
moticless added a commit that referenced this pull request Nov 21, 2024
PR #10285 introduced support for modules to register four types of
configurations — Bool, Numeric, String, and Enum. Accessible through the
Redis config file and the CONFIG command.

With this PR, it will be possible to register configuration parameters
without automatically prefixing the parameter names. This provides
greater flexibility in configuration naming, enabling, for instance,
both `bf-initial-size` or `initial-size` to be defined in the module
without automatically prefixing with `<MODULE-NAME>.`. In addition it
will also be possible to create a single additional alias via the same
API. This brings us another step closer to integrate modules into redis
core.

**Example:** Register a configuration parameter `bf-initial-size` with
an alias `initial-size` without the automatic module name prefix, set
with new `REDISMODULE_CONFIG_UNPREFIXED` flag:
```
RedisModule_RegisterBoolConfig(ctx, "bf-initial-size|initial-size", default_val, optflags | REDISMODULE_CONFIG_UNPREFIXED, getfn, setfn, applyfn, privdata);
```
# API changes
Related functions that now support unprefixed configuration flag
(`REDISMODULE_CONFIG_UNPREFIXED`) along with optional alias:
```
RedisModule_RegisterBoolConfig
RedisModule_RegisterEnumConfig
RedisModule_RegisterNumericConfig
RedisModule_RegisterStringConfig
```

# Implementation Details:
`config.c`: On load server configuration, at function
`loadServerConfigFromString()`, it collects all unknown configurations
into `module_configs_queue` dictionary. These may include valid module
configurations or invalid ones. They will be validated later by
`loadModuleConfigs()` against the configurations declared by the loaded
module(s).
`Module.c:` The `ModuleConfig` structure has been modified to store now:
(1) Full configuration name (2) Alias (3) Unprefixed flag status -
ensuring that configurations retain their original registration format
when triggered in notifications.

Added error printout:
This change introduces an error printout for unresolved configurations,
detailing each unresolved parameter detected during startup. The last
line in the output existed prior to this change and has been retained to
systems relies on it:
```
595011:M 18 Nov 2024 08:26:23.616 # Unresolved Configuration(s) Detected:
595011:M 18 Nov 2024 08:26:23.616 #  >>> 'bf-initiel-size 8'
595011:M 18 Nov 2024 08:26:23.616 #  >>> 'search-sizex 32'
595011:M 18 Nov 2024 08:26:23.616 # Module Configuration detected without loadmodule directive or no ApplyConfig call: aborting
```

# Backward Compatibility:
Existing modules will function without modification, as the new
functionality only applies if REDISMODULE_CONFIG_UNPREFIXED is
explicitly set.

# Module vs. Core API Conflict Behavior
The new API allows to modules loading duplication of same configuration
name or same configuration alias, just like redis core configuration
allows (i.e. the users sets two configs with a different value, but
these two configs are actually the same one). Unlike redis core, given a
name and its alias, it doesn't allow have both configuration on load. To
implement it, it is required to modify DS `module_configs_queue` to
reflect the order of their loading and later on, during
`loadModuleConfigs()`, resolve pairs of names and aliases and which one
is the last one to apply. "Relaxing" this limitation can be deferred to
a future update if necessary, but for now, we error in this case.
funny-dog pushed a commit to funny-dog/redis that referenced this pull request Sep 17, 2025
PR redis#10285 introduced support for modules to register four types of
configurations — Bool, Numeric, String, and Enum. Accessible through the
Redis config file and the CONFIG command.

With this PR, it will be possible to register configuration parameters
without automatically prefixing the parameter names. This provides
greater flexibility in configuration naming, enabling, for instance,
both `bf-initial-size` or `initial-size` to be defined in the module
without automatically prefixing with `<MODULE-NAME>.`. In addition it
will also be possible to create a single additional alias via the same
API. This brings us another step closer to integrate modules into redis
core.

**Example:** Register a configuration parameter `bf-initial-size` with
an alias `initial-size` without the automatic module name prefix, set
with new `REDISMODULE_CONFIG_UNPREFIXED` flag:
```
RedisModule_RegisterBoolConfig(ctx, "bf-initial-size|initial-size", default_val, optflags | REDISMODULE_CONFIG_UNPREFIXED, getfn, setfn, applyfn, privdata);
```
# API changes
Related functions that now support unprefixed configuration flag
(`REDISMODULE_CONFIG_UNPREFIXED`) along with optional alias:
```
RedisModule_RegisterBoolConfig
RedisModule_RegisterEnumConfig
RedisModule_RegisterNumericConfig
RedisModule_RegisterStringConfig
```

# Implementation Details:
`config.c`: On load server configuration, at function
`loadServerConfigFromString()`, it collects all unknown configurations
into `module_configs_queue` dictionary. These may include valid module
configurations or invalid ones. They will be validated later by
`loadModuleConfigs()` against the configurations declared by the loaded
module(s).
`Module.c:` The `ModuleConfig` structure has been modified to store now:
(1) Full configuration name (2) Alias (3) Unprefixed flag status -
ensuring that configurations retain their original registration format
when triggered in notifications.

Added error printout:
This change introduces an error printout for unresolved configurations,
detailing each unresolved parameter detected during startup. The last
line in the output existed prior to this change and has been retained to
systems relies on it:
```
595011:M 18 Nov 2024 08:26:23.616 # Unresolved Configuration(s) Detected:
595011:M 18 Nov 2024 08:26:23.616 #  >>> 'bf-initiel-size 8'
595011:M 18 Nov 2024 08:26:23.616 #  >>> 'search-sizex 32'
595011:M 18 Nov 2024 08:26:23.616 # Module Configuration detected without loadmodule directive or no ApplyConfig call: aborting
```

# Backward Compatibility:
Existing modules will function without modification, as the new
functionality only applies if REDISMODULE_CONFIG_UNPREFIXED is
explicitly set.

# Module vs. Core API Conflict Behavior
The new API allows to modules loading duplication of same configuration
name or same configuration alias, just like redis core configuration
allows (i.e. the users sets two configs with a different value, but
these two configs are actually the same one). Unlike redis core, given a
name and its alias, it doesn't allow have both configuration on load. To
implement it, it is required to modify DS `module_configs_queue` to
reflect the order of their loading and later on, during
`loadModuleConfigs()`, resolve pairs of names and aliases and which one
is the last one to apply. "Relaxing" this limitation can be deferred to
a future update if necessary, but for now, we error in this case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes indication that this issue needs to be mentioned in the release notes state:major-decision Requires core team consensus

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

[NEW] Redis module configuration

10 participants