Use Python dbus-next D-Bus library by agners · Pull Request #3234 · home-assistant/supervisor

agners · 2021-10-15T16:20:14Z

Proposed change

Use dbus-next library to communicate with D-Bus. dbus-next is a pure Python 3 library, supporting the asyncio main loop.

Tested & working:

Supervisor startup, loading properties from all kinds of D-Bus services like Systemd, NetworkManager, OS Agent etc
Waiting for signal, tested with Rauc OS update installation
NetworkManager update interface configuration

Type of change

Dependency upgrade
Bugfix (non-breaking change which fixes an issue)
New feature (which adds functionality to the supervisor)
Breaking change (fix/feature causing existing functionality to break)
Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #
This PR is related to issue:
Link to documentation pull request:
Link to cli pull request:

Checklist

The code change is tested and works locally.
Local tests pass. Your PR cannot be merged unless tests pass
There is no commented out code in this PR.
I have followed the development checklist
The code has been formatted using Black (black --fast supervisor tests)
Tests have been added to verify that the new code works.

If API endpoints of add-on configuration are added/changed:

Documentation added/updated for developers.home-assistant.io

pvizeli

We need to make sure that the error handling is 100% good, otherwise, we can run into boot loops. I already tried to replace the gdbus with a different library as you now use, but I was forced to stop my work on the branch because of a bad design with error handling on that library. This looks better, we need just make sure it works stable on a system with issues. We have also to cleanup the docker file and not install gdbus, and make sure we don't check this on bootstrap

agners · 2021-10-18T10:21:58Z

We need to make sure that the error handling is 100% good, otherwise, we can run into boot loops. I already tried to replace the gdbus with a different library as you now use, but I was forced to stop my work on the branch because of a bad design with error handling on that library. This looks better, we need just make sure it works stable on a system with issues. We have also to cleanup the docker file and not install gdbus, and make sure we don't check this on bootstrap

I think it was #1248 which actually added dbus-next, this library. But dbus-next evolved quite a bit since then, from the changelog it seems that also error handling got improved since then. Thanks for the input, I'll test with that in mind.

pvizeli · 2021-10-18T11:33:12Z

gdbus.py -> dbus.py ?

agners · 2021-10-18T12:48:45Z

gdbus.py -> dbus.py ?

I named it after the implementation just like before, so dbus_next.py. But maybe a generic name makes more sense here?

pvizeli · 2021-10-18T12:56:05Z

gdbus.py -> dbus.py ?

I named it after the implementation just like before, so dbus_next.py. But maybe a generic name makes more sense here?

Yeah, I think the generic is better as we use it as middle layer

pvizeli · 2021-10-18T18:27:16Z

You need rebase it for CI

It seems that gdbus (or systemd) automatically pick the correct interface and return the properties. However, dbussy requires the correct interface name to get all properties.

The property returns a type "y" which equates to "guchar": https://developer-old.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.AccessPoint.html#gdbus-property-org-freedesktop-NetworkManager-AccessPoint.Strength It seems that the old D-Bus implementation returned an array. With dbussy a integer is returned, so no list indexing required.

That is what the interface is expecting, otherwise the new lib choks on the Pathlib type.

Assemble Python native objects and pass them to dbus-next. Use dbus-next specific Variant class where necessary.

org.freedesktop.NetworkManager.Connection.Active.PropertyChanged is depricated. Also it seems that StateChanged leads to fewer and more accurate signals.

RequestScan expects an option dictionary. Pass an empty option dictionary to it.

Replace gdbus specific fixtures with json files representing the return values. Those can be easily converted into native Python objects.

The key name where the auth algorithm is stored is "auth-alg" instead of "auth-algo".

homeassistant added the cla-signed label Oct 15, 2021

agners mentioned this pull request Oct 15, 2021

Use DBussy for D-Bus communication #3233

Closed

12 tasks

agners added the refactor A code change that neither fixes a bug nor adds a feature label Oct 15, 2021

agners commented Oct 15, 2021

View reviewed changes

Comment thread Dockerfile Outdated

agners requested a review from pvizeli October 15, 2021 20:48

pvizeli reviewed Oct 18, 2021

View reviewed changes

Comment thread supervisor/utils/gdbus.py

Comment thread supervisor/utils/gdbus.py Outdated

pvizeli reviewed Oct 18, 2021

View reviewed changes

Comment thread supervisor/utils/gdbus.py Outdated

agners force-pushed the use-python-dbus-next branch from d17a4b6 to 4c5ba9d Compare October 18, 2021 13:34

pvizeli approved these changes Oct 18, 2021

View reviewed changes

agners added 16 commits October 18, 2021 20:29

Use the correct interface name to get properties of systemd

e201f54

It seems that gdbus (or systemd) automatically pick the correct interface and return the properties. However, dbussy requires the correct interface name to get all properties.

Add dbus-next via pip

fac1860

Use dbus-next for introspection

23862e6

Use dbus-next for method calls

dd9baca

Support signals and remove no longer used tests and code

79d9118

Pass rauc update file path as string

8e8fe36

That is what the interface is expecting, otherwise the new lib choks on the Pathlib type.

Support Network configuration with dbus-next

d4cfcbe

Assemble Python native objects and pass them to dbus-next. Use dbus-next specific Variant class where necessary.

Use org.freedesktop.NetworkManager.Connection.Active.StateChanged

d7fdc19

org.freedesktop.NetworkManager.Connection.Active.PropertyChanged is depricated. Also it seems that StateChanged leads to fewer and more accurate signals.

Remove template based network update code

da3d5b5

Pass correct data type to RequestScan.

d839368

RequestScan expects an option dictionary. Pass an empty option dictionary to it.

Address black

c3179b5

Add docstring

de99c35

Address isort

cfd8a3f

Add type to exception if unsupported type is passed to call_dbus()

1375eca

Address pylint errors

aa9741d

agners added 19 commits October 18, 2021 20:29

Update unit tests

2586b38

Replace gdbus specific fixtures with json files representing the return values. Those can be easily converted into native Python objects.

[TEMPORARY] Add dbus-next to requirements_tests.txt

e30a2be

Fix pylint

482cda2

Fix flake8/isort for tests

1b3e64a

Add basic test for connection object build

c8edd02

Fix auth algorithm key name

093f750

The key name where the auth algorithm is stored is "auth-alg" instead of "auth-algo".

Fix isort/flake8 errors

197c752

Sort for old isort

219ef27

Use dbus-next from our wheels repo

a53da14

Remove gdbus from Supervisor container image

d3b5342

Rename D-Bus utils module gdbus to dbus_next

32fe8c8

Improve Exception handling

8993fd8

Remove gdbus from Supervisor container image merge

ebb2bc7

Improve dbus_next, add unit tests

4e33f44

Address pyupgrade checks

8a16c92

Rename from dbus_next to dbus

89f4ae4

Fix argument passing

c40f1cc

Make pylint happy

fc2abb2

Reformat tests/dbus/test_systemd.py

3f3b48d

agners force-pushed the use-python-dbus-next branch from 5bb715b to 3f3b48d Compare October 18, 2021 18:29

agners merged commit 7a6663b into home-assistant:main Oct 18, 2021

github-actions Bot locked and limited conversation to collaborators Oct 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Python dbus-next D-Bus library#3234

Use Python dbus-next D-Bus library#3234
agners merged 35 commits into
home-assistant:mainfrom
agners:use-python-dbus-next

agners commented Oct 15, 2021

Uh oh!

Uh oh!

pvizeli left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

agners commented Oct 18, 2021

Uh oh!

pvizeli commented Oct 18, 2021

Uh oh!

Uh oh!

agners commented Oct 18, 2021

Uh oh!

pvizeli commented Oct 18, 2021

Uh oh!

pvizeli commented Oct 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

agners commented Oct 15, 2021

Proposed change

Type of change

Additional information

Checklist

Uh oh!

Uh oh!

pvizeli left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

agners commented Oct 18, 2021

Uh oh!

pvizeli commented Oct 18, 2021

Uh oh!

Uh oh!

agners commented Oct 18, 2021

Uh oh!

pvizeli commented Oct 18, 2021

Uh oh!

pvizeli commented Oct 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pvizeli left a comment •

edited

Loading