-
Notifications
You must be signed in to change notification settings - Fork 24.4k
Module Configurations #10285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Module Configurations #10285
Conversation
|
I think Module developers will all love this feature, thanks. |
|
@redis/core-team Please take a look at the top comment and provide feedback about the implementation. There are still a couple of checks failing in the CI, so once those are cleaned up and we have consensus on the finalized implementation I'll ping the group again. |
oranagra
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't reviewed the code yet, just responding to the top comment of the PR, the discussion points, and a few other things i noticed.
I decided to find places to post these as code-comments so that separate discussion threads can be handled in parallel.
yossigo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nichchun I've just had a quick look at this, so posting some initial feedback mostly based on @oranagra's and the top comment.
A config rewrite will rewrite all module configurations to the .conf file. Presently, there is no way to make assumptions about default values, so they are ALWAYS written on a config rewrite.
Not in favor of that. The problem is we lose track of explicit config vs default config, which tends to become an issue over time when defaults change. Is there a problem to expose something similar to what Redis already does, and let the module control if a value is to be rewritten or not?
Alternatives to the ApplyConfigs() API is to call the setCallback immediately after the config is registered, this was rejected because you might have dependencies between config values, and it might be useful to load them all together first.
I very briefly proposed an alternative. I am sure we need to support multi-param set as a basic feature of the API. Possibly, also support both push and pull (where pull is done by the module at OnLoad time).
|
Core group meeting decisions:
|
|
few notes: i'm not certain the new apply callback must get names of all configs that were changed (it's a complexity i think we can leave out, and let modules that care handle on their own. regarding defaults, the important requirements in my eyes are: first that we avoid making something explicit in rewrite so that we can later change default and still distinguish between explicit changes and implicit ones. just to mention that we agree that enums need better handling, not just an array of valid strings. |
|
@sjpotter i don't think this should be part of the configuration infrastructure. i think that maybe similarly to And we could add add an event / hook like |
Co-authored-by: sundb <sundbcn@gmail.com>
Co-authored-by: sundb <sundbcn@gmail.com>
|
Few updates here. ARGS is no longer required as an argument to loadex. There's now also an optional apply callback, and a default value that can be specified during registration. Enums now work based off ints (I went with the implicit index implementation.) Renamed RM_ApplyConfigs to RM_LoadConfigs to avoid confusion with new apply callbacks. Also a few documentation updates, and removed '.' restriction in config names. module configs always have a '.', but standardConfigs in the future can as well. |
…ime, ability to specify an apply function added
|
Note the forcepush was to clean up a bunch of small commits that made up the new features |
oranagra
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think the module API is rather solid, other than maybe:
- a few more flags we could expose (like PERCENT_CONFIG)
- improved enums configs (not relying on implicit int values).
- improved apply callback that takes a RedisModuleCtx and / or privdata
I posted quite a few comments about the implementation, but most importantly there's a lot of logic copied from config.c into module.c, and i think we could maybe find a way to avoid that (see my comments in parseAndSetNumericConfig).
so maybe we need to look into a POC for that approach before dealing with the test of the comments?
p.s. haven't reviewed the test code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I still prefer the char *, but it's not something I honestly feel that strongly about.
|
I also prefer |
The test assumed the default maxclients is always 10000, but running under valgrind with a smaller fileno results with that value decremented by Redis on startup.
|
@oranagra As discussed, I don't feel strongly about it but do think the |
|
merged 🎉 |
|
|
||
| int RedisModule_OnUnload(RedisModuleCtx *ctx) { | ||
| REDISMODULE_NOT_USED(ctx); | ||
| if (strval) RedisModule_FreeString(ctx, strval); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reproduced the fail test on alpine, when I comment out this line test is OK.
But I didn't find out the real reason why, no stack on alpine is too painful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only thing I can think of is that on alpine the dynamic library loader doesn't properly inits the global (maybe when the module is unloaded and reloaded).
@sundb can you try to init that global to null from the onload function and see if it helps?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@oranagra You guessed it right. strval needs to be reset to NULL after free.
Change code to following will be OK:
if (strval) {
RedisModule_FreeString(ctx, strval);
strval = NULL;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please make a PR to fix both issues.
| *err = RedisModule_CreateString(NULL, "Cannot set string to 'rejectisfreed'", 36); | ||
| return REDISMODULE_ERR; | ||
| } | ||
| RedisModule_Free(strval); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line should be wrong, it should be RedisModule_FreeString, but it's not the cause of the test failure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's interesting that valgrind doesn't report that leak (leaking the actual string).
i suppose the tcl test code doesn't set the string twice (replacing a previous one).
it's also odd that the alpine code crash on some corruption, but valgrind doesn't report anything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually i see we do have a test that does it.
so i don't know why valgrind doesn't report the leak.
r config set moduleconfigs.string wafflewednesdays
assert_equal [r config get moduleconfigs.string] "moduleconfigs.string wafflewednesdays"
r config set moduleconfigs.string \x73\x75\x70\x65\x72\x20\x00\x73\x65\x63\x72\x65\x74\x20\x70\x61\x73\x73\x77\x6f\x72\x64
assert_equal [r config get moduleconfigs.string] "moduleconfigs.string {super \0secret password}"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yoav helped me figure this out.
It must be an embedded string, so being short the robj pointer holds the sds in the same allocattion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did test and confirm this was the case, I specifically asked for the case to make sure we weren't mismanaging the allocation, forgetting about embedded strings.
…, and inserting comments around module and acl configs (#10761) A regression from #10285 (redis 7.0). CONFIG REWRITE would put lines with: `include`, `rename-command`, `user`, `loadmodule`, and any module specific config in a comment. For ACL `user`, `loadmodule` and module specific configs would be re-inserted at the end (instead of updating existing lines), so the only implication is a messy config file full of comments. But for `rename-command` and `include`, the implication would be that they're now missing, so a server restart would lose them. Co-authored-by: Oran Agra <oran@redislabs.com>
Use new module configuration API RedisRaft adopts new configuration API: redis/redis#10285 After this commit, `RAFT.CONFIG` will not exist anymore. `CONFIG` command will be used instead. Usage: - Command: `config set raft.addr 127.0.0.1:8000` and `config get raft.id` - CLI: `./redis-server --port 5001 --loadmodule redisraft.so --raft.id 3 --raft.addr 127.0.0.1:8000` - Config file: ``` ... port 5000 dbfilename redis.rdb loadmodule /home/ozan/Desktop/redisraft/redisraft.so raft.addr 10.0.3.69:5022 raft.slot-config 0:5460 ``` ------ **Breaking changes** 1. `CONFIG` command will be used instead of `RAFT.CONFIG` 2. Configuration parameters will require `raft.` or `--raft.` prefix. 3. Renamed some configuration parameters: | Old name | New name | | ------------- | ------------- | | raft-log-filename | log-filename | | raft-interval | periodic-interval | | raft-response-timeout | response-timeout | | raft-log-max-cache-size | log-max-cache-size | | raft-log-max-file-size | log-max-file-size | | raft-log-fsync | log-fsync | 4. `join-timeout` config was in seconds. Now, it is in milliseconds. `connection-timeout` config was in microseconds. Now, it is in milliseconds.
The PR adds the ability to add module configuration as using configuration API introduced on Redis 7: redis/redis#10285 The configuration is added optionally on `redis_module!` macro under `configurations` list that can contains the following subsections: * `i64` configuration: A list of `i64` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <min value>, <max value>, <flags>, <optional on update callback>]` * `string` configuration: A list of `string` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <flags>, <optional on update callback>]` * `bool` configuration: A list of `bool` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <flags>, <optional on update callback>]` * `enum` configuration: A list of `enum` configuration in the following format: `[<name>, <reference to the configuration object>, <default value>, <flags>, <optional on update callback>]` An example of all the 4 options can be found under `example/configuration.rs`. Notice that `enum` configuration is of special type and require provide an `enum` that implements the following traits: `TryFrom<i32>`, `From<$name>`, `EnumConfigurationValue`, `Clone`. User can use `enum_configuration` macro to easily added those implementation on a given `enum`. In addition, it is also possible to tell redismodule-rs to look at the module arguments as if they were configuration using `module_args_as_configuration: true/false` option. Usage examample: ```rust /// A macro to easily creating an enum that can be used as an enum configuration enum_configuration! { enum EnumConfiguration { Val1 = 1, Val2 = 2, } } /// Static variable that can be used as configuration and will be automatically set when `config set` command is used. /// All the variables must be somehow thread safe protected, either as atomic variable, `RedisGILGuard` or `Mutex`. lazy_static! { static ref CONFIGURATION_I64: RedisGILGuard<i64> = RedisGILGuard::default(); static ref CONFIGURATION_ATOMIC_I64: AtomicI64 = AtomicI64::new(1); static ref CONFIGURATION_REDIS_STRING: RedisGILGuard<RedisString> = RedisGILGuard::new(RedisString::create(None, "default")); static ref CONFIGURATION_STRING: RedisGILGuard<String> = RedisGILGuard::new("default".into()); static ref CONFIGURATION_MUTEX_STRING: Mutex<String> = Mutex::new("default".into()); static ref CONFIGURATION_ATOMIC_BOOL: AtomicBool = AtomicBool::default(); static ref CONFIGURATION_BOOL: RedisGILGuard<bool> = RedisGILGuard::default(); static ref CONFIGURATION_ENUM: RedisGILGuard<EnumConfiguration> = RedisGILGuard::new(EnumConfiguration::Val1); static ref CONFIGURATION_MUTEX_ENUM: Mutex<EnumConfiguration> = Mutex::new(EnumConfiguration::Val1); } /// This function will be called when a configuration change is done and will count the total number of changes. fn num_changes(ctx: &Context, _: Vec<RedisString>) -> RedisResult { let val = NUM_OF_CONFIGERATION_CHANGES.lock(ctx); Ok(RedisValue::Integer(*val)) } /// The module initialisation macro, gets the configuration variable and some extra data (such as configuration name and /// default value) and pass it to Redis so Redis will set them and return their values on `config set` and `config get` respectively. redis_module! { name: "configuration", version: 1, data_types: [], commands: [ ["configuration.num_changes", num_changes, "", 0, 0, 0], ], configurations: [ i64: [ ["i64", &*CONFIGURATION_I64, 10, 0, 1000, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ["atomic_i64", &*CONFIGURATION_ATOMIC_I64, 10, 0, 1000, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ], string: [ ["redis_string", &*CONFIGURATION_REDIS_STRING, "default", ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ["string", &*CONFIGURATION_STRING, "default", ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed::<String, _>))], ["mutex_string", &*CONFIGURATION_MUTEX_STRING, "default", ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed::<String, _>))], ], bool: [ ["atomic_bool", &*CONFIGURATION_ATOMIC_BOOL, true, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ["bool", &*CONFIGURATION_BOOL, true, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ], enum: [ ["enum", &*CONFIGURATION_ENUM, EnumConfiguration::Val1, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ["enum_mutex", &*CONFIGURATION_MUTEX_ENUM, EnumConfiguration::Val1, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed))], ], module_args_as_configuration: true, // Indication that we also want to read module arguments as configuration. ] } ```
…, and inserting comments around module and acl configs (redis#10761) A regression from redis#10285 (redis 7.0). CONFIG REWRITE would put lines with: `include`, `rename-command`, `user`, `loadmodule`, and any module specific config in a comment. For ACL `user`, `loadmodule` and module specific configs would be re-inserted at the end (instead of updating existing lines), so the only implication is a messy config file full of comments. But for `rename-command` and `include`, the implication would be that they're now missing, so a server restart would lose them. Co-authored-by: Oran Agra <oran@redislabs.com>
PR #10285 introduced support for modules to register four types of configurations — Bool, Numeric, String, and Enum. Accessible through the Redis config file and the CONFIG command. With this PR, it will be possible to register configuration parameters without automatically prefixing the parameter names. This provides greater flexibility in configuration naming, enabling, for instance, both `bf-initial-size` or `initial-size` to be defined in the module without automatically prefixing with `<MODULE-NAME>.`. In addition it will also be possible to create a single additional alias via the same API. This brings us another step closer to integrate modules into redis core. **Example:** Register a configuration parameter `bf-initial-size` with an alias `initial-size` without the automatic module name prefix, set with new `REDISMODULE_CONFIG_UNPREFIXED` flag: ``` RedisModule_RegisterBoolConfig(ctx, "bf-initial-size|initial-size", default_val, optflags | REDISMODULE_CONFIG_UNPREFIXED, getfn, setfn, applyfn, privdata); ``` # API changes Related functions that now support unprefixed configuration flag (`REDISMODULE_CONFIG_UNPREFIXED`) along with optional alias: ``` RedisModule_RegisterBoolConfig RedisModule_RegisterEnumConfig RedisModule_RegisterNumericConfig RedisModule_RegisterStringConfig ``` # Implementation Details: `config.c`: On load server configuration, at function `loadServerConfigFromString()`, it collects all unknown configurations into `module_configs_queue` dictionary. These may include valid module configurations or invalid ones. They will be validated later by `loadModuleConfigs()` against the configurations declared by the loaded module(s). `Module.c:` The `ModuleConfig` structure has been modified to store now: (1) Full configuration name (2) Alias (3) Unprefixed flag status - ensuring that configurations retain their original registration format when triggered in notifications. Added error printout: This change introduces an error printout for unresolved configurations, detailing each unresolved parameter detected during startup. The last line in the output existed prior to this change and has been retained to systems relies on it: ``` 595011:M 18 Nov 2024 08:26:23.616 # Unresolved Configuration(s) Detected: 595011:M 18 Nov 2024 08:26:23.616 # >>> 'bf-initiel-size 8' 595011:M 18 Nov 2024 08:26:23.616 # >>> 'search-sizex 32' 595011:M 18 Nov 2024 08:26:23.616 # Module Configuration detected without loadmodule directive or no ApplyConfig call: aborting ``` # Backward Compatibility: Existing modules will function without modification, as the new functionality only applies if REDISMODULE_CONFIG_UNPREFIXED is explicitly set. # Module vs. Core API Conflict Behavior The new API allows to modules loading duplication of same configuration name or same configuration alias, just like redis core configuration allows (i.e. the users sets two configs with a different value, but these two configs are actually the same one). Unlike redis core, given a name and its alias, it doesn't allow have both configuration on load. To implement it, it is required to modify DS `module_configs_queue` to reflect the order of their loading and later on, during `loadModuleConfigs()`, resolve pairs of names and aliases and which one is the last one to apply. "Relaxing" this limitation can be deferred to a future update if necessary, but for now, we error in this case.
PR redis#10285 introduced support for modules to register four types of configurations — Bool, Numeric, String, and Enum. Accessible through the Redis config file and the CONFIG command. With this PR, it will be possible to register configuration parameters without automatically prefixing the parameter names. This provides greater flexibility in configuration naming, enabling, for instance, both `bf-initial-size` or `initial-size` to be defined in the module without automatically prefixing with `<MODULE-NAME>.`. In addition it will also be possible to create a single additional alias via the same API. This brings us another step closer to integrate modules into redis core. **Example:** Register a configuration parameter `bf-initial-size` with an alias `initial-size` without the automatic module name prefix, set with new `REDISMODULE_CONFIG_UNPREFIXED` flag: ``` RedisModule_RegisterBoolConfig(ctx, "bf-initial-size|initial-size", default_val, optflags | REDISMODULE_CONFIG_UNPREFIXED, getfn, setfn, applyfn, privdata); ``` # API changes Related functions that now support unprefixed configuration flag (`REDISMODULE_CONFIG_UNPREFIXED`) along with optional alias: ``` RedisModule_RegisterBoolConfig RedisModule_RegisterEnumConfig RedisModule_RegisterNumericConfig RedisModule_RegisterStringConfig ``` # Implementation Details: `config.c`: On load server configuration, at function `loadServerConfigFromString()`, it collects all unknown configurations into `module_configs_queue` dictionary. These may include valid module configurations or invalid ones. They will be validated later by `loadModuleConfigs()` against the configurations declared by the loaded module(s). `Module.c:` The `ModuleConfig` structure has been modified to store now: (1) Full configuration name (2) Alias (3) Unprefixed flag status - ensuring that configurations retain their original registration format when triggered in notifications. Added error printout: This change introduces an error printout for unresolved configurations, detailing each unresolved parameter detected during startup. The last line in the output existed prior to this change and has been retained to systems relies on it: ``` 595011:M 18 Nov 2024 08:26:23.616 # Unresolved Configuration(s) Detected: 595011:M 18 Nov 2024 08:26:23.616 # >>> 'bf-initiel-size 8' 595011:M 18 Nov 2024 08:26:23.616 # >>> 'search-sizex 32' 595011:M 18 Nov 2024 08:26:23.616 # Module Configuration detected without loadmodule directive or no ApplyConfig call: aborting ``` # Backward Compatibility: Existing modules will function without modification, as the new functionality only applies if REDISMODULE_CONFIG_UNPREFIXED is explicitly set. # Module vs. Core API Conflict Behavior The new API allows to modules loading duplication of same configuration name or same configuration alias, just like redis core configuration allows (i.e. the users sets two configs with a different value, but these two configs are actually the same one). Unlike redis core, given a name and its alias, it doesn't allow have both configuration on load. To implement it, it is required to modify DS `module_configs_queue` to reflect the order of their loading and later on, during `loadModuleConfigs()`, resolve pairs of names and aliases and which one is the last one to apply. "Relaxing" this limitation can be deferred to a future update if necessary, but for now, we error in this case.
Implementation based off of #9437
Implementation details:
This feature adds the ability to add four different types (Bool, Numeric, String, Enum) of configurations to a module to be accessed via the redis config file, and the CONFIG command.
Configuration Names:
We impose a restriction that a module configuration always starts with the module name and contains a '.' followed by the config name. If a module passes "config1" as the name to a register function, it will be registered as MODULENAME.config1.
Configuration Persistence:
Module Configurations exist only as long as a module is loaded. If a module is unloaded, the configurations are removed.
There is now also a minimal core API for removal of standardConfig objects from configs by name.
Get and Set Callbacks:
Storage of config values is owned by the module that registers them, and provides callbacks for Redis to access and manipulate the values. This is exposed through a GET and SET callback.
The get callback returns a typed value of the config to redis. The callback takes the name of the configuration, and also a privdata pointer. Note that these only take the CONFIGNAME portion of the config, not the entire MODULENAME.CONFIGNAME.
Configs must also must specify a set callback, i.e. what to do on a CONFIG SET XYZ 123 or when loading configurations from cli/.conf file matching these typedefs. name is again just the CONFIGNAME portion, val is the parsed value from the core, privdata is the registration time privdata pointer, and err is for providing errors to a client.
Modules can also specify an optional apply callback that will be called after value(s) have been set via CONFIG SET:
Flags:
We expose 7 new flags to the module, which are used as part of the config registration.
Module Registration APIs:
The module name will be auto appended along with a "." to the front of the name of the config.
What RM_Register[...]Config does:
A RedisModule struct now keeps a list of ModuleConfig objects which look like:
It also registers a standardConfig in the configs array, with a pointer to the ModuleConfig object associated with it.
What happens on a CONFIG GET/SET MODULENAME.MODULECONFIG:
For CONFIG SET, we do the same parsing as is done in config.c and pass that as the argument to the module set callback. For CONFIG GET, we call the module get callback and return that value to config.c to return to a client.
CONFIG REWRITE:
Starting up a server with module configurations in a .conf file but no module load directive will fail. The flip side is also true, specifying a module load and a bunch of module configurations will load those configurations in using the module defined set callbacks on a RM_LoadConfigs call. Configs being rewritten works the same way as it does for standard configs, as the module has the ability to specify a default value. If a module is unloaded with configurations specified in the .conf file those configurations will be commented out from the .conf file on the next config rewrite.
RM_LoadConfigs:
RedisModule_LoadConfigs(RedisModuleCtx *ctx);This last API is used to make configs available within the onLoad() after they have been registered. The expected usage is that a module will register all of its configs, then call LoadConfigs to trigger all of the set callbacks, and then can error out if any of them were malformed. LoadConfigs will attempt to set all configs registered to either a .conf file argument/loadex argument or their default value if an argument is not specified. LoadConfigs is a required function if configs are registered. Also note that LoadConfigs does not call the apply callbacks, but a module can do that directly after the LoadConfigs call.
New Command: MODULE LOADEX [CONFIG NAME VALUE] [ARGS ...]:
This command provides the ability to provide startup context information to a module. LOADEX stands for "load extended" similar to GETEX. Note that provided config names need the full MODULENAME.MODULECONFIG name. Any additional arguments a module might want are intended to be specified after ARGS. Everything after ARGS is passed to onLoad as RedisModuleString **argv.
Notes:
There was an ask for double, my suggestion is that you should store a double as just a string and do conversion within the module. You get into weird cases like precision with floating points that vary between systems.