Adapted the graphite sender plugin for new Python plugin API#877
Closed
dmitrkul wants to merge 1 commit intobareos:masterfrom
Closed
Adapted the graphite sender plugin for new Python plugin API#877dmitrkul wants to merge 1 commit intobareos:masterfrom
dmitrkul wants to merge 1 commit intobareos:masterfrom
Conversation
… 20 documentation
Contributor
|
Hi @dmitrkul - thanks for contributing. just for completeness: can you confirm that you've tested the patch and that it works with graphite? (sorry, I have no graphite environment ready to quickly test it). This patch needs to be aligned with #PR881, as both affect the base-class. |
Author
|
Hi, Maik.
No, I have not tested this patch. I will try to test and return with the
result.
Thank you.
чт, 8 июл. 2021 г. в 11:43, Maik Aussendorf ***@***.***>:
… Hi @dmitrkul <https://github.com/dmitrkul> - thanks for contributing.
just for completeness: can you confirm that you've tested the patch and
that it works with graphite? (sorry, I have no graphite environment ready
to quickly test it).
This patch needs to be aligned with #PR881, as both affect the base-class.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#877 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIET35HNT4J5P4XIH4BX3ULTWVQJZANCNFSM47R3F2MQ>
.
--
С уважением, Куликов Дмитрий.
|
Author
|
I tested the changes from this #PR881 and the result is the same. Context
not set by Bareos Python API.
"bDirEventJobRun event triggered at Unix time %s\n" % (self.jobRunTime),
RuntimeError:
../../../../../../core/src/plugins/dird/python/module/bareosdir.cc:357
:plugin_ctx is unset
The problem is not in the functionality of sending to graphite, but in the
logic of the API.
In the documentation for the transition to the new API bareos 20, it is
necessary to remove the context from all sections of the custom plug-in
code, which was done.
And after that, such an error began to appear. On bareos version 19 and
below, where the context was passed between methods and classes of custom
plugins, there were no such errors.
чт, 8 июл. 2021 г. в 11:59, Дмитрий Куликов ***@***.***>:
… Hi, Maik.
No, I have not tested this patch. I will try to test and return with the
result.
Thank you.
чт, 8 июл. 2021 г. в 11:43, Maik Aussendorf ***@***.***>:
> Hi @dmitrkul <https://github.com/dmitrkul> - thanks for contributing.
>
> just for completeness: can you confirm that you've tested the patch and
> that it works with graphite? (sorry, I have no graphite environment ready
> to quickly test it).
>
> This patch needs to be aligned with #PR881, as both affect the base-class.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#877 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AIET35HNT4J5P4XIH4BX3ULTWVQJZANCNFSM47R3F2MQ>
> .
>
--
С уважением, Куликов Дмитрий.
--
С уважением, Куликов Дмитрий.
|
16 tasks
Contributor
|
I get the same error in #881 with another plugin I wrote: Seems that the Error lies somewhere in the Python plugin, right?! |
Author
|
Hi.
I think the context is lost when switching between methods in bareosdir.cc
пн, 19 июл. 2021 г. в 16:48, Benibr ***@***.***>:
… I get the same error in #881 <#881>
with another plugin I wrote:
File "/usr/lib/bareos/plugins/BareosDirPluginPrometheusExporter.py", line 190, in push_job_information
JobMessage(M_INFO, "---------> {}\n".format(self.jobStatus))
RuntimeError: ../../../../../../core/src/plugins/dird/python/module/bareosdir.cc:378 :plugin_ctx is unset
Seems that the Error lies somewhere in the Python plugin, right?!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#877 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIET35GVODYGP2EKECLIFDTTYQUK5ANCNFSM47R3F2MQ>
.
--
С уважением, Куликов Дмитрий.
|
Author
|
Hi. |
Member
Member
|
Regarding the graphite plugin, I'd like to propose that you don't use and furthermore please replace as well as to adapt the code properly the modernized python plugin API. |
Author
|
Hi, I tried this and result is the same.
call last):
File "/usr/lib/bareos/plugins/BareosDirWrapper.py", line 40, in
handle_plugin_event
return bareos_dir_plugin_object.handle_plugin_event(event)
File "/usr/lib/bareos/plugins/BareosDirPluginGraphiteSender.py", line 73,
in handle_plugin_event
super(BareosDirPluginGraphiteSender, self).handle_plugin_event(event)
File "/usr/lib/bareos/plugins/BareosDirPluginBaseclass.py", line 130, in
handle_plugin_event
"bDirEventJobRun event triggered at Unix time %s\n" % (self.jobRunTime),
RuntimeError:
../../../../../../core/src/plugins/dird/python/module/bareosdir.cc:357
:plugin_ctx is unset
вт, 14 сент. 2021 г. в 11:52, sduehr ***@***.***>:
… Regarding the graphite plugin, I'd like to propose that you don't use
from bareosdir import *
but instead only
import bareosdir
and furthermore please replace
return bRCs[b'bRC_OK']
by
return bareosdir.bRC_OK
as well as
bDirEventType['bDirEventJobEnd']
by
bareosdir.bDirEventJobEnd
to adapt the code properly the modernized python plugin API.
See also
https://docs.bareos.org/TasksAndConcepts/Plugins.html#modernization-of-the-python-plugin-api
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#877 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIET35A3HTLXYJASNWPTQYDUB4EL7ANCNFSM47R3F2MQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
С уважением, Куликов Дмитрий.
|
Member
|
Ok, how did you test that? Only replacing BareosDirPluginBaseclass.py does not fix the plugin_ctx is unset error. |
Author
|
Sorry, I didnt see this PR. I will try to build plugin with this fixes.
ср, 15 сент. 2021 г. в 17:12, sduehr ***@***.***>:
… Ok, how did you test that?
The fix for the plugin_ctx is unset error is in the bareos-dir C code, so
you must either locally build the branch
https://github.com/bareos/bareos/tree/dev/sduehr/master/fix-python-dir-sd-plugin
or wait for the packages from the PR #923
<#923> to appear at
https://download.bareos.org/bareos/experimental/CD/PR-923/
Only replacing BareosDirPluginBaseclass.py does not fix the plugin_ctx is
unset error.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#877 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIET35DVW57GWX5POGM5CH3UCCSUTANCNFSM47R3F2MQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
С уважением, Куликов Дмитрий.
|
sduehr
reviewed
Sep 30, 2021
| # we return OK in anyway, we do not want to produce Bareos errors just because of failing | ||
| # Nagios notifications | ||
| return bRCs['bRC_OK'] | ||
| return bRCs[b'bRC_OK'] |
Member
There was a problem hiding this comment.
Suggested change
| return bRCs[b'bRC_OK'] | |
| return bareosdir.bRC_OK |
| @@ -25,7 +25,6 @@ | |||
| # Functions taken and adapted from bareos-dir.py | |||
|
|
|||
| from bareosdir import * | |||
Member
There was a problem hiding this comment.
Suggested change
| from bareosdir import * | |
| import bareosdir |
| self.transmitResult() | ||
|
|
||
| return bRCs['bRC_OK'] | ||
| return bRCs[b'bRC_OK'] |
Member
There was a problem hiding this comment.
Suggested change
| return bRCs[b'bRC_OK'] | |
| return bareosdir.bRC_OK |
| self.metrics['bareos.jobs.{}.status.other'.format(job)] = 1 | ||
|
|
||
| DebugMessage(context, 100, "Graphite metrics: {}\n".format(self.metrics)) | ||
| DebugMessage(100, "Graphite metrics: {}\n".format(self.metrics)) |
Member
There was a problem hiding this comment.
Suggested change
| DebugMessage(100, "Graphite metrics: {}\n".format(self.metrics)) | |
| bareosdir.DebugMessage(100, "Graphite metrics: {}\n".format(self.metrics)) |
| Overload this method if you want ot submit your changes on a different way | ||
| ''' | ||
| DebugMessage(context, 100, "Submitting metrics to {}:{}".format(self.collectorHost, | ||
| DebugMessage(100, "Submitting metrics to {}:{}".format(self.collectorHost, |
Member
There was a problem hiding this comment.
Suggested change
| DebugMessage(100, "Submitting metrics to {}:{}".format(self.collectorHost, | |
| bareosdir.DebugMessage(100, "Submitting metrics to {}:{}".format(self.collectorHost, |
| sock.close() | ||
| except Exception as e: | ||
| JobMessage(context, bJobMessageType['M_WARNING'], | ||
| JobMessage(bJobMessageType['M_WARNING'], |
Member
There was a problem hiding this comment.
Suggested change
| JobMessage(bJobMessageType['M_WARNING'], | |
| bareosdir.JobMessage(bareosdir.M_WARNING, |
| context, plugindef) | ||
| return bareos_dir_consts.bRCs['bRC_OK'] | ||
| plugindef) | ||
| return bareosdir.bRCs[b'bRC_OK'] |
Member
There was a problem hiding this comment.
Suggested change
| return bareosdir.bRCs[b'bRC_OK'] | |
| return bareosdir.bRC_OK |
Member
|
Hi, meanwhile PR #923 has been merged into the master branch. |
Author
|
Hi,
thanks. I closed my PR and I will try to check later that problems was
solved.
пн, 18 окт. 2021 г. в 22:46, sduehr ***@***.***>:
… Hi,
meanwhile PR #923 <#923> has been
merged into the master branch.
Could you please rebase your fork/branch to master or at least remove
BareosDirPluginBaseclass.py and apply the suggested changes to resolve the
merge conflict? And for sure please verify if the plugin works now.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#877 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIET35CHFO2RIDA6CZZYHATUHR2RBANCNFSM47R3F2MQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
С уважением, Куликов Дмитрий.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I have adapted the plugin according to the recommendations in the documentation https://docs.bareos.org/TasksAndConcepts/Plugins.html#porting-existing-python-plugins.
But after the changes, all backup jobs crash with this problem:
30-Jun 10:43 bareos:director JobId 17: Fatal error: bareosdir: Traceback (most recent call last):
File "/usr/lib/bareos/plugins/BareosDirWrapper.py", line 40, in handle_plugin_event
return bareos_dir_plugin_object.handle_plugin_event(event)
File "/usr/lib/bareos/plugins/BareosDirPluginGraphiteSender.py", line 72, in handle_plugin_event
super(BareosDirPluginGraphiteSender, self).handle_plugin_event(event)
File "/usr/lib/bareos/plugins/BareosDirPluginBaseclass.py", line 130, in handle_plugin_event
"bDirEventJobRun event triggered at Unix time %s\n" % (self.jobRunTime),
RuntimeError: ../../../../../../core/src/plugins/dird/python/module/bareosdir.cc:357 :plugin_ctx is unset
In debug I see that context is set
bareos:director (150): dird/dir_plugins.cc:517-0 === enter NewPlugins ===
bareos:director (150): dird/dir_plugins.cc:525-0 dir-plugin-list size=1
bareos:director (150): dird/dir_plugins.cc:393-0 Instantiate dir-plugin_ctx_list=7fcd480d92e0 JobId=19
bareos:director (150): dird/dir_plugins.cc:789-0 dir-plugin: Plugin registered event=11
bareos:director (150): python/python-dir.cc:578-0 python3-dir: Trying to load module with name bareos-dir-graphite-sender
bareos:director (150): python/python-dir.cc:592-0 python3-dir: Successfully loaded module with name bareos-dir-graphite-sender
bareos:director (100): module/bareosdir.cc:359-0 python3-dir-mod: Constructor called in module BareosDirPluginBaseclass
bareos:director (150): module/bareosdir.cc:413-0 python3-dir-mod: PyBareosRegisterEvents registering event 1
bareos:director (150): dird/dir_plugins.cc:789-0 dir-plugin: Plugin registered event=1
bareos:director (150): module/bareosdir.cc:413-0 python3-dir-mod: PyBareosRegisterEvents registering event 2
bareos:director (150): dird/dir_plugins.cc:789-0 dir-plugin: Plugin registered event=2
bareos:director (150): module/bareosdir.cc:413-0 python3-dir-mod: PyBareosRegisterEvents registering event 3
bareos:director (150): dird/dir_plugins.cc:789-0 dir-plugin: Plugin registered event=3
bareos:director (150): module/bareosdir.cc:413-0 python3-dir-mod: PyBareosRegisterEvents registering event 4
bareos:director (150): dird/dir_plugins.cc:789-0 dir-plugin: Plugin registered event=4
bareos:director (150): dird/dir_plugins.cc:594-0 dir-plugin: return Job name=ldap01.msk.avito.ru-ldap-job.2021-06-30_12.17.13_10
bareos:director (150): dird/dir_plugins.cc:604-0 dir-plugin: return bDirVarLevel=F
bareos:director (150): dird/dir_plugins.cc:609-0 dir-plugin: return bDirVarType=B
bareos:director (150): dird/dir_plugins.cc:589-0 dir-plugin: return bDirVarJobId=19
bareos:director (150): dird/dir_plugins.cc:614-0 dir-plugin: return bDirVarClient=ldap01.msk.avito.ru-fd
bareos:director (150): dird/dir_plugins.cc:682-0 dir-plugin: return bDirVarJobStatus=C
bareos:director (150): dird/dir_plugins.cc:687-0 dir-plugin: return bDirVarPriority=10
bareos:director (100): module/bareosdir.cc:359-0 python3-dir-mod: JobName = ldap01.msk.avito.ru-ldap-job.2021-06-30_12.17.13_10 - level = F - type = 66 - Id = 19 - Client = ldap01.msk.avito.ru-fd - jobStatus = 67 - Priority = 10 - BareosDirPluginBaseclass
bareos:director (100): module/bareosdir.cc:359-0 python3-dir-mod: plugin def parser called with
bareos:director (100): module/bareosdir.cc:359-0 python3-dir-mod: key:val = :bareos:director (150): dird/dir_plugins.cc:227-0 dir-plugin_ctx_list=7fcd480d92e0 JobId=19
But
bareos:director (150): dird/dir_plugins.cc:227-19 dir-plugin_ctx_list=7fcd480d92e0 JobId=19
bareos:director (150): include/python_plugins_common.inc:124-19 bareosdir: Traceback (most recent call last):
File "/usr/lib/bareos/plugins/BareosDirWrapper.py", line 40, in handle_plugin_event
return bareos_dir_plugin_object.handle_plugin_event(event)
File "/usr/lib/bareos/plugins/BareosDirPluginGraphiteSender.py", line 72, in handle_plugin_event
super(BareosDirPluginGraphiteSender, self).handle_plugin_event(event)
File "/usr/lib/bareos/plugins/BareosDirPluginBaseclass.py", line 130, in handle_plugin_event
"bDirEventJobRun event triggered at Unix time %s\n" % (self.jobRunTime),
RuntimeError: ../../../../../../core/src/plugins/dird/python/module/bareosdir.cc:357 :plugin_ctx is unset
…s 20 documentation
Thank you for contributing to the Bareos Project!
Please check
If you have any questions or problems, please give a comment in the PR.
Helpful documentation and best practices
Checklist for the reviewer of the PR (will be processed by the Bareos team)
General
Source code quality
bareos-check-sources --since-mergedoes not report any problemsgit statusshould not report modifications in the source tree after building and testingTests