Skip to content

Bareos Python Plugins: fix handling of extra pluginoptions#1177

Merged
pstorz merged 7 commits intobareos:masterfrom
joergsteffens:dev/joergs/master/pluginoptions
May 18, 2022
Merged

Bareos Python Plugins: fix handling of extra pluginoptions#1177
pstorz merged 7 commits intobareos:masterfrom
joergsteffens:dev/joergs/master/pluginoptions

Conversation

@joergsteffens
Copy link
Member

@joergsteffens joergsteffens commented May 6, 2022 •
edited by pstorz
Loading

This PR fixes a problem with handling additional pluginoptions, provided during a restore. When these are set, the plugin "forgets" about the Python module path.

Thank you for contributing to the Bareos Project!

Please check

  • Short description and the purpose of this PR is present above this paragraph
  • Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

General
  • PR name is meaningful
  • Purpose of the PR is understood
  • Separate commit for this PR in the CHANGELOG.md, PR number referenced is same
  • Commit descriptions are understandable and well formatted
  • If backport: add original PR number and target branch at top of this file: Backport of PR#000 to bareos-2x
Source code quality
  • Source code changes are understandable
  • Variable and function names are meaningful
  • Code comments are correct (logically and spelling)
  • Required documentation changes are present and part of the PR
  • bareos-check-sources --since-merge does not report any problems
  • git status should not report modifications in the source tree after building and testing
Tests
  • Decision taken that a test is required (if not, then remove this paragraph)
  • The choice of the type of test (unit test or systemtest) is reasonable
  • Testname matches exactly what is being tested
  • On a fail, output of the test leads quickly to the origin of the fault

@joergsteffens joergsteffens force-pushed the dev/joergs/master/pluginoptions branch 2 times, most recently from 98a87ba to ddec5cf Compare May 9, 2022 14:22
@joergsteffens joergsteffens requested a review from pstorz May 9, 2022 14:26
@joergsteffens joergsteffens marked this pull request as ready for review May 9, 2022 14:26
@joergsteffens joergsteffens force-pushed the dev/joergs/master/pluginoptions branch from ddec5cf to 015a4b1 Compare May 9, 2022 14:35
pstorz
pstorz requested changes May 16, 2022
View reviewed changes
Copy link
Member

@pstorz pstorz left a comment •
edited by joergsteffens
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the following change requests:

  • The problem that is fixed with this PR is not described
  • Variable name enhancements
  • changelog.md entry is missing
  • bareos-check-sources is not happy about systemtests/python-modules/bareos_unittest/json.py, needs to be formatted
  • system:python-bareos:python_fd_plugins does not pass:
361: === python-bareos/test_python_fd_plugins: starting at 12:05:14 ===
361: =
361: =
361: /home/pstorz/git/PR-REVIEW/python-bareos/bareos/bsock/lowlevel.py:57: UserWarning: Connection encryption via TLS-PSK is not available, as the module sslpsk is not installed.
361:   warnings.warn(
361: test_additional_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions) ... FAIL
361: test_overwrite_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions)
361: Overwrites a plugin option ... expected failure
361: test_without_extra_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions) ... FAIL
361: 
361: ======================================================================
361: FAIL: test_additional_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions)
361: ----------------------------------------------------------------------
361: Traceback (most recent call last):
361:   File "/home/pstorz/git/PR-REVIEW/b/systemtests/tests/python-bareos/test_python_fd_plugins.py", line 89, in test_additional_pluginoptions
361:     jobid = self.run_job(
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 103, in run_job
361:     self.wait_job(director, jobId)
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 91, in wait_job
361:     self.assertEqual(result["job"]["jobstatuslong"], expected_status)
361: AssertionError: 'Fatal Error' != 'OK'
361: - Fatal Error
361: + OK
361: 
361: 
361: ======================================================================
361: FAIL: test_without_extra_pluginoptions (test_python_fd_plugins.BareosFdPythonPluginOptions)
361: ----------------------------------------------------------------------
361: Traceback (most recent call last):
361:   File "/home/pstorz/git/PR-REVIEW/b/systemtests/tests/python-bareos/test_python_fd_plugins.py", line 55, in test_without_extra_pluginoptions
361:     jobid = self.run_job(director, jobname, level="Full", wait=True)
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 103, in run_job
361:     self.wait_job(director, jobId)
361:   File "/home/pstorz/git/PR-REVIEW/systemtests/python-modules/bareos_unittest/json.py", line 91, in wait_job
361:     self.assertEqual(result["job"]["jobstatuslong"], expected_status)
361: AssertionError: 'Fatal Error' != 'OK'
361: - Fatal Error
361: + OK
361: 
361: 
361: ----------------------------------------------------------------------
361: Ran 3 tests in 6.685s
361: 
361: FAILED (failures=2, expected failures=1)
361: ERROR: 
361: ERROR: 
361:  
361:   !!!!! python-bareos/test_python_fd_plugins failed!!! 12:05:21 !!!!! 
361:    Status: estat=9 zombie=0 backup=0 restore=0 diff=0
361:  
2/3 Test #361: system:python-bareos:python_fd_plugins ...***Failed    6.94 sec

@joergsteffens joergsteffens force-pushed the dev/joergs/master/pluginoptions branch from 05842a0 to 870ad84 Compare May 16, 2022 15:21
@joergsteffens joergsteffens requested a review from pstorz May 16, 2022 17:37
pstorz
pstorz requested changes May 17, 2022
View reviewed changes
Copy link
Member

@pstorz pstorz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the following remarks:

  • The Pluginoptions are in one line in the docs, while the multiline is much better readable.
  • Last build failed, now waiting for next build.

docs/manuals/source/TasksAndConcepts/Plugins/FileDaemonPlugins/VMwarePlugin.rst.inc
":vcuser=bakadm@vsphere.local"
":vcpass=Bak.Adm-1234"
":localvmdk=yes"
"python:module_name=bareos-fd-vmware:dc=dass5:folder=/:vmname=stephand-test02:vcserver=virtualcenter5.dass-it:vcuser=bakadm@vsphere.local:vcpass=Bak.Adm-1234:localvmdk=yes"
Copy link
Member

@pstorz pstorz May 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want to break the plugin options at the ":" as before? That makes things much better readable.

Copy link
Member Author

@joergsteffens joergsteffens May 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a string that has to be typed during the restore command. I assume, splitting it over multiple lines is not an option, especially not in Bareos <= 15.2.3, as stated above.
But now that you point to it, I think, also the quoted around this string have to be removed.

@pstorz
py3plug-fd-postgres systemtest: remove "BROKEN" label
9df098f 
The test was marked broken when the plugin was using psycopg2, but now
we use pg8000 so we want to enable it again if all requirements are met.

Also we allow python >= 3.10 to be used as we now avoid the problems
arising from the fact that the stat time filds are floats instead of
ints:

> mystatp.st_atime = statp.st_atime
> TypeError: 'float' object cannot be interpreted as an integer
@pstorz pstorz mentioned this pull request May 17, 2022
Closed
12 tasks
@joergsteffens joergsteffens requested a review from pstorz May 17, 2022 18:27
@pstorz pstorz merged commit db1af02 into bareos:master May 18, 2022
@joergsteffens joergsteffens deleted the dev/joergs/master/pluginoptions branch June 22, 2022 14:47
bruno-at-bareos pushed a commit to tuxmaster5000/bareos that referenced this pull request Jul 4, 2022
@joergsteffens @bruno-at-bareos
updated CHANGELOG for bareos#1177
8134647
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pstorz pstorz pstorz approved these changes

Assignees

@pstorz pstorz

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@joergsteffens @pstorz