Skip to content

tentacle: mgr:python: avoid pyo3 errors by running certain cryptographic functions in a child process#66794

Merged
nizamial09 merged 25 commits intotentaclefrom
pyo3-tentacle
Jan 16, 2026
Merged

tentacle: mgr:python: avoid pyo3 errors by running certain cryptographic functions in a child process#66794
nizamial09 merged 25 commits intotentaclefrom
pyo3-tentacle

Conversation

@nizamial09
Copy link
Copy Markdown
Member

@nizamial09 nizamial09 commented Jan 5, 2026

backport of #62951, #66143

self.fail('Invalid private key: %s' % str(e))
_crt = self._load_cert(crt)
try:
context = SSL.Context(SSL.TLSv1_METHOD)

Check failure

Code scanning / CodeQL

Use of insecure SSL/TLS version High

Insecure SSL/TLS protocol version TLSv1 specified by
call to SSL.Context
.
Copy link
Copy Markdown
Member

@epuertat epuertat Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While reviewing this (again 😅 ) I realized that we're still using PyOpenSSL... Based on the warning these folks posted:

Note

Note: The Python Cryptographic Authority strongly suggests the use of pyca/cryptography where possible. If you are using pyOpenSSL for anything other than making a TLS connection you should move to cryptography and drop your pyOpenSSL dependency.

context = SSL.Context(SSL.TLSv1_METHOD)

could be easily replace with CPython's built-in ssl solution:

context = ssl.create_default_context() # TLS >=1.2

Same for Private Key/Certificate generation:

    def create_private_key(self) -> str:
        pkey = crypto.PKey()
        pkey.generate_key(crypto.TYPE_RSA, 2048)
        return crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey).decode()

into

from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import serialization

def create_private_key(self) -> str:
    key = rsa.generate_private_key(
        public_exponent=65537,
        key_size=2048,
    )

    pem = key.private_bytes(
        encoding=serialization.Encoding.PEM,
        format=serialization.PrivateFormat.TraditionalOpenSSL,
        encryption_algorithm=serialization.NoEncryption(),
    )

    return pem.decode("utf-8")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we have to "suffer" from the cryptography PyO3 side-effects, let's fully use it 😅

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good. but this needs to come in from the main branch to the tentacle so I'll open a separate PR for it and test it out..

pecastro and others added 22 commits January 5, 2026 16:14
…itialized once per interpreter process' issue.

Fixes: https://tracker.ceph.com/issues/64213
Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 717d0a6)
Where possible try to use structured output in JSON for easier parsing
and interaction with the parent process.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 6f2d92c)
Create a class to act as a common shim between the cryptotools external
functions and the mgr. It provides common conversion mechanisms and
could possibly act as an abstraction in case we decide to make
the external function calls in different ways in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 84710f9)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit a5861c1)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 9dcfde7)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 2b9cf24)
Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 4bcab13)
…he exit.

Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 56d508f)
Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 218d84f)
The remote verify_tls function was not raising errors when it should.
Fix the function so that it always returns an object when it succeeds or
fails gracefully. Always parse that function in the crypto caller class.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 96a7a72)
…ode/decode.

Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit e364df3)
Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 21d6e1d)
Name the parser objects after their functions and not `foo` and `bar`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 5d4eeff)
Replace a direct usage of bycrypt with our cryptocaller wrapper.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d2fd81e)
Why violate the typing in a test? mypy never noticed this because tests
are not type checked but there seems to be no need to turn a str into
bytes to pass to a function that is typed only as taking str!

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 955143d)
The functions now handle the i/o but allow the crypto function class
to centralize the functions that actually use the crypto libs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 4e4cfa5)
Use a main function to encapsulate the cli parsing rather than a block
of code in module scope.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit c98e53f)
Lightly reorganize and make the "endpoint" functions in cryptotools.py more
consistent and uniform. Use small functions for input and output
handling so that the handling is done the same way throughout. Pass a
pre-constructed crypto caller via the args to then endpoint functions.
Make generating the private key it's own named function rather than
one single (and only) function with overloaded behavior controlled by
a cli switch.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 552d7b4)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit c3dc34a)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 0c774d5)
Add a module to select a desired crypto caller. Update the callers
to use the crypto caller interface.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 0eb2f4b)
Previously, the internal crypto caller would catch (and convert) some
errors when reading the cert but not all cases. Move the logic to catch
the errors to a common location and do it once consistently.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit f6ab087)
The cephadm modules needs to use python cryptography module for ssh (via
asyncssh) and thus there's no need to use the remote crypto caller in
cephadm. Configure cephadm to always use the internal cryptocaller.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 2128ffa)
Add a mgr config option `crypto_caller` that lets a ceph user override
the default behavior of using the remote crypto caller. Supported
values are `internal` and `remote`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 27c2050)
@nizamial09 nizamial09 marked this pull request as ready for review January 5, 2026 11:12
@nizamial09 nizamial09 requested review from a team as code owners January 5, 2026 11:12
@nizamial09
Copy link
Copy Markdown
Member Author

jenkins test windows

@github-project-automation github-project-automation bot moved this from New to Reviewer approved in Ceph-Dashboard Jan 7, 2026
@afreen23
Copy link
Copy Markdown
Contributor

afreen23 commented Jan 7, 2026

@nizamial09 this would need teuthology and also ensuring ceph dash and adm are not releated

@nizamial09
Copy link
Copy Markdown
Member Author

@afreen23 yup. I'll wait for cephadm teams approval and probably @adk3798 or @phlogistonjohn can help me schedule a run.

@nizamial09
Copy link
Copy Markdown
Member Author

I'll also look athe dashboard e2e. I remember we need to fix something in main related to pyvenv. Probably we need to backport that too

in frontend e2e.sh file, we don't need to start the node venv early on
before the ceph cluster is started. we only need it for the `npm` or
`npx` commands. Starting node virtual env and then starting ceph will
cause the ceph cluster to assume the node-env python as the python
environment which breaks the cryptotools call.

So moving the node-env venv start after the ceph is created

Fixes: https://tracker.ceph.com/issues/73804
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a56ae5b)
@nizamial09
Copy link
Copy Markdown
Member Author

I'll also look athe dashboard e2e. I remember we need to fix something in main related to pyvenv. Probably we need to backport that too

done, backported #66143

@nizamial09
Copy link
Copy Markdown
Member Author

jenkins test api

@nizamial09
Copy link
Copy Markdown
Member Author

jenkins test dashboard

@nizamial09
Copy link
Copy Markdown
Member Author

jenkins test windows

@nizamial09
Copy link
Copy Markdown
Member Author

cephadm e2e failure

curl: (22) The requested URL returned error: 404 Not Found
Using url [https://download.fedoraproject.org/pub/fedora/linux/releases/40/Cloud/x86_64/images/Fedora-Cloud-Base-Generic.x86_64-40-1.14.qcow2...](https://download.fedoraproject.org/pub/fedora/linux/releases/40/Cloud/x86_64/images/Fedora-Cloud-Base-Generic.x86_64-40-1.14.qcow2...%1B[0m)
Grabbing image fedora40...
Image fedora40 not Added because Unable to download indicated image
++ kcli delete plan -y ceph
- Generating browser application bundles (phase: setup)...
Nothing to do for plan ceph

will need a backport of a different PR but I'll do that separately. Let's see if the dashboard e2e passes here.

@nizamial09
Copy link
Copy Markdown
Member Author

merging it now since tentacle is failing for a while

@nizamial09 nizamial09 merged commit c6f7860 into tentacle Jan 16, 2026
17 of 20 checks passed
@nizamial09 nizamial09 deleted the pyo3-tentacle branch January 16, 2026 08:46
@github-project-automation github-project-automation bot moved this from Reviewer approved to Done in Ceph-Dashboard Jan 16, 2026
James-1701 added a commit to James-1701/nixpkgs that referenced this pull request Feb 27, 2026
Backports upstream fix from ceph/ceph#66794 which runs cryptographic
functions in a child process to avoid the conflict.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

6 participants