Infrastructure

Not just one dependency:

No matching distribution found for parse>=1.8.2
ERROR: No matching distribution found for mypy
ERROR: No matching distribution found for jsonschema~=4.0
ERROR: No matching distribution found for cryptography

running some tests here: https://github.com/ceph/ceph/pull/52544

I think we need to..

Pin the following packages here: https://github.com/ceph/ceph/blob/main/src/pybind/mgr/requirements-required.txt
- pytest-cov
- cryptography

Possibly pin mypy here: https://github.com/ceph/ceph/blob/main/src/python-common/requirements-lint.txt

Pin parse to parse>=1.8.2 so it's compatible with behave in https://github.com/ceph/ceph/blob/main/monitoring/ceph-mixin/requirements-lint.txt

Verified this locally:

root@dc6ed7dc7f00:~/ceph/build# git diff
diff --git a/src/python-common/requirements-lint.txt b/src/python-common/requirements-lint.txt
index 2a7142182c2..821416a383e 100644
--- a/src/python-common/requirements-lint.txt
+++ b/src/python-common/requirements-lint.txt
@@ -1,2 +1,3 @@
 flake8==3.7.8
 rstcheck==3.3.1
+mypy

Also verified this locally:

diff --git a/src/pybind/mgr/dashboard/requirements-lint.txt b/src/pybind/mgr/dashboard/requirements-lint.txt
index 57e51915740..aa561911e8d 100644
--- a/src/pybind/mgr/dashboard/requirements-lint.txt
+++ b/src/pybind/mgr/dashboard/requirements-lint.txt
@@ -8,4 +8,4 @@ rstcheck==3.3.1
 autopep8==1.5.7
 pyfakefs==4.5.0
 isort==5.5.3
-jsonschema~=4.0
+jsonschema==4.18.4
diff --git a/src/pybind/mgr/dashboard/requirements-test.txt b/src/pybind/mgr/dashboard/requirements-test.txt
index da283d0b64a..5c52c1a4ac7 100644
--- a/src/pybind/mgr/dashboard/requirements-test.txt
+++ b/src/pybind/mgr/dashboard/requirements-test.txt
@@ -1,4 +1,4 @@
 pytest-cov
 pytest-instafail
 pyfakefs==4.5.0
-jsonschema~=4.0
+jsonschema==4.18.4
diff --git a/src/pybind/mgr/requirements-required.txt b/src/pybind/mgr/requirements-required.txt
index 76fef65dbe4..09389a9998b 100644
--- a/src/pybind/mgr/requirements-required.txt
+++ b/src/pybind/mgr/requirements-required.txt
@@ -1,7 +1,7 @@
 -e ../../python-common
 asyncmock
 cherrypy
-cryptography
+cryptography==41.0.2
 jsonpatch
 Jinja2
 pecan

(doing this all in a jammy container)

Trying https://github.com/ceph/ceph/pull/52558

Ugh, these adjustments don't play nice with the install-deps.sh script. Still working through it..

Ah, interesting.. this applies to mypy at least, but after running installl-deps.sh, a folder "src/python-common/wheelhouse" is created. These dependencies are used in the python-common test. Notice that mypy is not in the folder after running install-deps.sh.

ceph# ls src/python-common/wheelhouse
attrs-23.1.0-py3-none-any.whl        entrypoints-0.3-py2.py3-none-any.whl  md5                                     pluggy-1.2.0-py3-none-any.whl           rstcheck-3.3.1-py3-none-any.whl      wheel-0.40.0-py3-none-any.whl
cachetools-5.3.1-py3-none-any.whl    filelock-3.12.2-py3-none-any.whl      mypy_extensions-1.0.0-py3-none-any.whl  pycodestyle-2.5.0-py2.py3-none-any.whl  setuptools-68.0.0-py3-none-any.whl
chardet-5.1.0-py3-none-any.whl       flake8-3.7.8-py2.py3-none-any.whl     packaging-23.1-py3-none-any.whl         pyflakes-2.1.1-py2.py3-none-any.whl     tomli-2.0.1-py3-none-any.whl
colorama-0.4.6-py2.py3-none-any.whl  iniconfig-2.0.0-py3-none-any.whl      pip-23.2-py3-none-any.whl               pyproject_api-1.5.3-py3-none-any.whl    tox-4.6.4-py3-none-any.whl
distlib-0.3.7-py2.py3-none-any.whl   mccabe-0.6.1-py2.py3-none-any.whl     platformdirs-3.9.1-py3-none-any.whl     pytest_mypy-0.10.3-py3-none-any.whl     virtualenv-20.24.0-py3-none-any.whl

This step in the install-deps.sh script generates /wheelhouse folders in various locations:

function preload_wheels_for_tox() {
    ci_debug "Running preload_wheels_for_tox() in install-deps.sh" 
    local ini=$1
    shift
    pushd . > /dev/null
    cd $(dirname $ini)
    local require_files=$(ls *requirements*.txt 2>/dev/null) || true
    local constraint_files=$(ls *constraints*.txt 2>/dev/null) || true
    local require=$(echo -n "$require_files" | sed -e 's/^/-r /')
    local constraint=$(echo -n "$constraint_files" | sed -e 's/^/-c /')
    local md5=wheelhouse/md5
    if test "$require"; then
        if ! test -f $md5 || ! md5sum -c $md5 > /dev/null; then
            rm -rf wheelhouse
        fi
    fi
    if test "$require" && ! test -d wheelhouse ; then
        type python3 > /dev/null 2>&1 || continue
        activate_virtualenv $top_srcdir || exit 1
        python3 -m pip install --upgrade pip
        populate_wheelhouse "wheel -w $wip_wheelhouse" $require $constraint || exit 1
        mv $wip_wheelhouse wheelhouse
        md5sum $require_files $constraint_files > $md5
    fi
    popd > /dev/null
}

These generated folders are missing the packages we're getting errors about.

Think this came from the new release of pip on July 15th: https://pypi.org/project/pip/23.2/

Keeping pip at v22.0.2 is what is helping the tests pass for now.

reef backport: https://github.com/ceph/ceph/pull/52562