Improve python plugin configuration by arogge · Pull Request #1605 · bareos/bareos

arogge · 2023-11-27T12:02:17Z

add support for configuration files

by setting one or both of the options defaults_file or overrides_file you can now tell every plugin using the baseclass to read a configuration file.
While defaults_file will add the values before the plugin definition (i.e. plugin defintion will override the values from that file), using overrides_files will add the values after defaults and plugin definition (i.e. will override defaults and plugin definition).

All paths are also treated relative to the FD's configuration directory to make life easier.

add support for encoded configuration values

if you need to encode a value to make it unreadable (e.g. an api-key), you can now do that. If you - for example - want to set option password to an encoded value, you instead set the option password#enc to the encoded value and the baseclass will take care of decoding the value for the plugin.

TODO

provide a script to actually encode values
package everything correctly
add/update documentation

Please check

Short description and the purpose of this PR is present above this paragraph
Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General

Is the PR title usable as CHANGELOG entry?
Purpose of the PR is understood
Commit descriptions are understandable and well formatted
~~Check backport line~~
Required backport PRs have been created

Source code quality

Source code changes are understandable
Variable and function names are meaningful
Code comments are correct (logically and spelling)
Required documentation changes are present and part of the PR

Tests

Decision taken that a test is required (if not, then remove this paragraph)
The choice of the type of test (unit test or systemtest) is reasonable
Testname matches exactly what is being tested
On a fail, output of the test leads quickly to the origin of the fault

sebsura

This is pretty cool, but i do have some questions.

Is the idea that each plugin overwrites config_path so that each of them has a unique config file ? If not, maybe the core should just load the file into a string and pass that to python so that we do not have to bother with the path stuff inside python.
If each plugin is supposed to share the same config file, should we use the sections to tell which option belongs to which plugin ? I.e.

[Base] # shared by every plugin
...
[vmware] # only for vmware plugins
...
[my-plugin]
...

This would obviously only work if the plugin knows its name.

What do you think of using https://docs.python.org/3/library/configparser.html for parsing ? Looks like it has a similar goal and its a standard library (?). That way we can say that we follow a/the standard.

core/src/filed/fd_plugins.cc

core/src/plugins/filed/python/module/bareosfd.h

arogge · 2023-11-28T09:25:24Z

This is pretty cool, but i do have some questions.

* Is the idea that each plugin overwrites `config_path` so that each of them has a unique config file ?  If not, maybe the core should just load the file into a string and pass that to python so that we do not have to bother with the path stuff inside python.

No, that's not the idea.

* If each plugin is supposed to share the same config file, should we use the sections to tell which option belongs to which plugin ? I.e.

We're not sharing the configuration files. Retrieving the config-path from core is just so you can configure with relative paths.

* What do you think of using https://docs.python.org/3/library/configparser.html for parsing ? Looks like it has a similar goal and its a standard library (?).  That way we can say that we follow a/the standard.

Thought about it (especially as it is used in the VMWare plugin), but the mapping code would be more complicated than the "parser" we have for the configuration files now.
Also, I didn't want to use a different approach than the plugindef parser, so you don't get strange effects when moving a setting from plugindef to configuration file or back.

sebsura

I added some additional comments

systemtests/tests/py3plug-fd-basic/python-modules/config-test-module.py