mgr/dashboard: use system packages when running tox by nizamial09 · Pull Request #62239 · ceph/ceph

nizamial09 · 2025-03-12T05:36:20Z

This PR intends to configure the tox to run with system installed packages rather than getting a version from pip which may differ from the original ones that we support.

this will also unpin the grpc dependencies.

Fixes: https://tracker.ceph.com/issues/70411

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins test classic perf Jenkins Job | Jenkins Job Definition
jenkins test crimson perf Jenkins Job | Jenkins Job Definition
jenkins test signed Jenkins Job | Jenkins Job Definition
jenkins test make check Jenkins Job | Jenkins Job Definition
jenkins test make check arm64 Jenkins Job | Jenkins Job Definition
jenkins test submodules Jenkins Job | Jenkins Job Definition
jenkins test dashboard Jenkins Job | Jenkins Job Definition
jenkins test dashboard cephadm Jenkins Job | Jenkins Job Definition
jenkins test api Jenkins Job | Jenkins Job Definition
jenkins test docs ReadTheDocs | Github Workflow Definition
jenkins test ceph-volume all Jenkins Jobs | Jenkins Jobs Definition
jenkins test windows Jenkins Job | Jenkins Job Definition
jenkins test rook e2e Jenkins Job | Jenkins Job Definition

xmlsec had an upgrade yesterday night and python3-saml might need to adapt its library accordingly I suppose. Testing a fix by pinning lxml Another approach is being tried out separately ceph#62239, but that is failing with some other errors. Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com>

src/pybind/mgr/dashboard/tox.ini

nizamial09 · 2025-03-12T12:41:55Z

extending couple of stuffs @pecastro doing in https://github.com/ceph/ceph/pull/61967/files so that it applies for the dashboard tox in general. Might need to adapt few tests as stated by him in his original PR.

xmlsec had an upgrade yesterday night and python3-saml might need to adapt its library accordingly I suppose. Testing a fix by pinning lxml Another approach is being tried out separately ceph#62239, but that is failing with some other errors. Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com> (cherry picked from commit 1f84505) Conflicts: src/pybind/mgr/dashboard/requirements.txt - only kept the lxml pinning. didn't add the newer deps that are present in main

xmlsec had an upgrade yesterday night and python3-saml might need to adapt its library accordingly I suppose. Testing a fix by pinning lxml Another approach is being tried out separately ceph#62239, but that is failing with some other errors. Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com> (cherry picked from commit 1f84505) Conflicts: src/pybind/mgr/dashboard/requirements.txt - didn't add the dependency that are present in main and only added the lxml pinning

github-actions · 2025-03-17T08:49:41Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

src/pybind/mgr/dashboard/tests/test_sso.py

src/python-common/pyproject.toml

debian/control

Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com>

nizamial09 · 2025-06-27T05:27:15Z

src/pybind/mgr/dashboard/services/nvmeof_client.py

@@ -1,3 +1,5 @@
+# pylint: disable=unexpected-keyword-arg


I saw that pylint2.6.0 mishandles line-level disables which was true for me because it failed when I did

MessageToDict(message, including_default_value_fields=True, # pylint: disable=unexpected-keyword-arg preserving_proto_field_name=True) # type: ignore

so had to keep it here. this is only a lint error so I am suspecting our pylinter is a bit older and its way past time to update that.

nizamial09 · 2025-06-27T07:42:12Z

jenkins test dashboard

nizamial09 · 2025-06-27T07:42:20Z

jenkins test dashboard cephadm

nizamial09 · 2025-06-30T08:14:08Z

okay, I just saw that the ubuntu 24.04 is not compliant with the current version of flake8 and other test frameworks.. I am testing it with an ubuntu container. Will update this PR to reflect all those (otherwise its useless for the ubuntu migration).

The problem I saw was ubuntu has py3.12 which uses a newer version of importlib_metadata and that broke flake8.
PyCQA/flake8#1701

nizamial09 · 2025-06-30T09:29:42Z

Okay, as I started upgrading things, it requires a bigger migration so I am gonna open a separate PR just for those upgrades to make it easier for testing it and capturing issues..

src/pybind/mgr/dashboard/controllers/saml2.py

Signed-off-by: Nizamudeen A <nia@redhat.com>

…ailure" This reverts commit 1f84505. Signed-off-by: Nizamudeen A <nia@redhat.com>

Signed-off-by: Nizamudeen A <nia@redhat.com>

Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com>

Signed-off-by: Nizamudeen A <nia@redhat.com>

nizamial09 · 2025-07-01T14:26:45Z

jenkins test make check arm64

which brings in python3-saml and is not needed anymore after ceph#62239 Fixes: https://tracker.ceph.com/issues/70937 Signed-off-by: Nizamudeen A <nia@redhat.com>

which brings in python3-saml and is not needed anymore after ceph#62239 Fixes: https://tracker.ceph.com/issues/70937 Signed-off-by: Nizamudeen A <nia@redhat.com> (cherry picked from commit 7064294)

marcan · 2025-08-20T04:44:11Z

src/pybind/mgr/dashboard/services/nvmeof_client.py

+    # distros; centos and ubuntu.
+    import os
+    os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
+


This is not just due to a mismatch. Protobuf3 C++ is broken with subinterpreters, and this workaround is needed when running with that version (e.g. Fedora). The Protobuf4 C++ implementation should work properly.

See: protocolbuffers/protobuf#10143

which brings in python3-saml and is not needed anymore after ceph#62239 Fixes: https://tracker.ceph.com/issues/70937 Signed-off-by: Nizamudeen A <nia@redhat.com>

nizamial09 requested a review from a team as a code owner March 12, 2025 05:36

nizamial09 requested review from cloudbehl and pecastro and removed request for a team March 12, 2025 05:36

github-actions bot added dashboard pybind labels Mar 12, 2025

github-project-automation bot added this to Ceph-Dashboard Mar 12, 2025

github-project-automation bot moved this to New in Ceph-Dashboard Mar 12, 2025

nizamial09 force-pushed the xmlsec-fail-main branch from 8797936 to 4975988 Compare March 12, 2025 05:51

nizamial09 requested a review from a team as a code owner March 12, 2025 05:51

github-actions bot added the cephadm label Mar 12, 2025

nizamial09 mentioned this pull request Mar 12, 2025

mgr/dashboard: pin lxml to fix run-dashboard-tox-make-check failure #62242

Merged

14 tasks

nizamial09 force-pushed the xmlsec-fail-main branch from 4975988 to 1bdbc5d Compare March 12, 2025 07:13

nizamial09 commented Mar 12, 2025

View reviewed changes

src/pybind/mgr/dashboard/tox.ini Outdated Show resolved Hide resolved

nizamial09 force-pushed the xmlsec-fail-main branch from 40fcbc4 to 6d820eb Compare March 12, 2025 09:50

github-actions bot added the build/ops label Mar 12, 2025

nizamial09 force-pushed the xmlsec-fail-main branch from 6d820eb to ed21579 Compare March 13, 2025 06:48

nizamial09 changed the title ~~mgr/dashboard: fix xmlsec failure in run-tox-mgr-dashboard-py3~~ mgr/dashboard: use system packages when running tox Mar 13, 2025

nizamial09 force-pushed the xmlsec-fail-main branch from ed21579 to adaf388 Compare March 17, 2025 05:11

github-actions bot added the needs-rebase label Mar 17, 2025

nizamial09 force-pushed the xmlsec-fail-main branch from adaf388 to ad57a6b Compare March 17, 2025 14:08

github-actions bot removed the needs-rebase label Mar 17, 2025

nizamial09 force-pushed the xmlsec-fail-main branch from ad57a6b to 73e9995 Compare March 17, 2025 15:14

github-project-automation bot moved this from New to Reviewer approved in Ceph-Dashboard Jun 26, 2025

nizamial09 force-pushed the xmlsec-fail-main branch from 1db010a to 9922f04 Compare June 26, 2025 13:42

phlogistonjohn reviewed Jun 26, 2025

View reviewed changes

src/pybind/mgr/dashboard/tests/test_sso.py Outdated Show resolved Hide resolved

src/python-common/pyproject.toml Outdated Show resolved Hide resolved

debian/control Show resolved Hide resolved

mgr/dashboard: use system packages when running tox

ffde8d0

Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com>

nizamial09 force-pushed the xmlsec-fail-main branch from 9922f04 to 3606ee7 Compare June 27, 2025 04:59

nizamial09 commented Jun 27, 2025

View reviewed changes

tchaikov mentioned this pull request Jun 27, 2025

mgr/dashboard: Fix inline markup warning in API documentation #64164

Merged

14 tasks

nizamial09 requested a review from phlogistonjohn June 27, 2025 05:30

epuertat approved these changes Jun 30, 2025

View reviewed changes

src/pybind/mgr/dashboard/controllers/saml2.py Outdated Show resolved Hide resolved

nizamial09 added 5 commits July 1, 2025 13:29

mgr/dashboard: disable saml2 tests when the dep is not there

8acf8f7

Signed-off-by: Nizamudeen A <nia@redhat.com>

Revert "mgr/dashboard: pin lxml to fix run-dashboard-tox-make-check f…

8e6b6cd

…ailure" This reverts commit 1f84505. Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: sitepackages only for py3

6985ed1

Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: unpin grpcio deps

84922d1

Fixes: https://tracker.ceph.com/issues/70411 Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: catch protobuf error due to mismatch in version

acb0f19

Signed-off-by: Nizamudeen A <nia@redhat.com>

nizamial09 force-pushed the xmlsec-fail-main branch from 3606ee7 to acb0f19 Compare July 1, 2025 07:59

nizamial09 requested a review from Pegonzal July 1, 2025 09:30

nizamial09 merged commit 2dd02b5 into ceph:main Jul 2, 2025
15 checks passed

nizamial09 deleted the xmlsec-fail-main branch July 2, 2025 04:40

github-project-automation bot moved this from Reviewer approved to Done in Ceph-Dashboard Jul 2, 2025

nizamial09 mentioned this pull request Jul 2, 2025

tentacle: mgr/dashboard: use system packages when running tox #64295

Merged

nizamial09 mentioned this pull request Jul 8, 2025

mgr/dashboard: rm requirements-extra file #64380

Merged

14 tasks

marcan reviewed Aug 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/dashboard: use system packages when running tox#62239

mgr/dashboard: use system packages when running tox#62239
nizamial09 merged 6 commits intoceph:mainfrom
rhcs-dashboard:xmlsec-fail-main

nizamial09 commented Mar 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

nizamial09 commented Mar 12, 2025

Uh oh!

github-actions bot commented Mar 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nizamial09 Jun 27, 2025

Uh oh!

nizamial09 commented Jun 27, 2025

Uh oh!

nizamial09 commented Jun 27, 2025

Uh oh!

nizamial09 commented Jun 30, 2025

Uh oh!

nizamial09 commented Jun 30, 2025

Uh oh!

Uh oh!

nizamial09 commented Jul 1, 2025

Uh oh!

Uh oh!

marcan Aug 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

nizamial09 commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution Guidelines

Checklist

Uh oh!

Uh oh!

nizamial09 commented Mar 12, 2025

Uh oh!

github-actions bot commented Mar 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nizamial09 Jun 27, 2025

Choose a reason for hiding this comment

Uh oh!

nizamial09 commented Jun 27, 2025

Uh oh!

nizamial09 commented Jun 27, 2025

Uh oh!

nizamial09 commented Jun 30, 2025

Uh oh!

nizamial09 commented Jun 30, 2025

Uh oh!

Uh oh!

nizamial09 commented Jul 1, 2025

Uh oh!

Uh oh!

marcan Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

nizamial09 commented Mar 12, 2025 •

edited

Loading