python-common/cryptotools: avoid interpreter issues of pyo3 workaround by benaryorg · Pull Request #67904 · ceph/ceph

benaryorg · 2026-03-20T04:49:12Z

This code has been tested on tentacle (20.2.0, Python 3.12, with the PyO3 patches).
Specifically this is needed with nixpkgs/NixOS since the Python interpreter in this instance is not on the PATH as it is provided as an absolute path.
However as the commit messages detail this is not the only situation in which issues can arise.
Without this patch the dashboard will not function as it will fail to find python3.
Other components using the cryptotools may be affected too.

This PR is split into two parts; the first part being the minimum fix for the code to call the correct interpreter.
The second part is a larger rewrite which removes the custom JSON error passing and the subprocess invocation, restores regular exception handling and generally handles things via provided Python facilities.

The second part is technically optional, but I was already rewriting the code to use multiprocessing and only found the actual bug preventing the interpreter from being properly accessible to the code through that detour.
Both parts have been tested individually, so the first part will run without the second.
If the project decides to only merge the first part that's fine by me, however I encourage you to consider the surface of the currently-merged-but-not-released code given, just as an example, the incredibly broad way exceptions have to be caught or the security concerns raised by the code in question.
Both of these issues are already solved by multiprocessing, along with some other features which may benefit further development or maintenance in the future.

The clearest indicator of why this is a good idea is probably the diff stat of the last commit:

src/python-common/ceph/cryptotools/cryptotools.py | 135 --------------
src/python-common/ceph/cryptotools/remote.py      | 210 ++++++----------------
src/python-common/ceph/cryptotools/select.py      |   4 +-
3 files changed, 55 insertions(+), 294 deletions(-)

Either way, backporting this to tentacle before the next release would be great, given that the current PyO3 workarounds are not actually contained in the latest release, this'd avoid anyone running into these issues in the first place (other than those who backported the patches on their own of course).

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug: 84710f9 as part of mgr:python: avoid pyo3 errors by running certain cryptographic functions in a child process #62951 (not very recent, but has not hit a release yet)
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate (as per the prior PR)
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins test classic perf Jenkins Job | Jenkins Job Definition
jenkins test crimson perf Jenkins Job | Jenkins Job Definition
jenkins test signed Jenkins Job | Jenkins Job Definition
jenkins test make check Jenkins Job | Jenkins Job Definition
jenkins test make check arm64 Jenkins Job | Jenkins Job Definition
jenkins test submodules Jenkins Job | Jenkins Job Definition
jenkins test dashboard Jenkins Job | Jenkins Job Definition
jenkins test dashboard cephadm Jenkins Job | Jenkins Job Definition
jenkins test api Jenkins Job | Jenkins Job Definition
jenkins test docs ReadTheDocs | Github Workflow Definition
jenkins test ceph-volume all Jenkins Jobs | Jenkins Jobs Definition
jenkins test windows Jenkins Job | Jenkins Job Definition
jenkins test rook e2e Jenkins Job | Jenkins Job Definition

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

ceph-jenkins · 2026-03-20T04:49:15Z