Project

General

Profile

Actions
Copy link

Bug #72500

open

ragweed: ModuleNotFoundError: No module named 'boto.vendored.six.moves' on ubuntu 24.04/python3.12

Added by Casey Bodley 9 months ago. Updated 6 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Mark Kogan
Target version:
-
% Done:

0%

Source:
Q/A
Backport:
squid tentacle
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
ragweed
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

https://qa-proxy.ceph.com/teuthology/cbodley-2025-07-30_19:08:51-rgw:verify-wip-main-ubuntu24-distro-default-smithi/8417451/teuthology.log

2025-07-30T21:21:59.640 INFO:teuthology.orchestra.run.smithi064.stdout:py: commands[0]> python -m pytest -v -m 'not fails_on_rgw'
2025-07-30T21:22:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:============================= test session starts ==============================
2025-07-30T21:22:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:platform linux -- Python 3.12.3, pytest-7.4.4, pluggy-1.4.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python
2025-07-30T21:22:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:cachedir: .tox/py/.pytest_cache
2025-07-30T21:22:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0
2025-07-30T21:22:00.003 INFO:teuthology.orchestra.run.smithi064.stdout:configfile: pytest.ini
2025-07-30T21:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout:collecting ... collected 0 items / 1 error
2025-07-30T21:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout:
2025-07-30T21:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout:==================================== ERRORS ====================================
2025-07-30T21:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout:___________________ ERROR collecting ragweed/tests/tests.py ____________________
2025-07-30T21:22:00.151 INFO:teuthology.orchestra.run.smithi064.stdout:ImportError while importing test module '/home/ubuntu/cephtest/ragweed.client.0/ragweed/tests/tests.py'.
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:Hint: make sure your test modules/packages have valid Python names.
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:Traceback:
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/lib/python3.12/importlib/__init__.py:90: in import_module
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:    return _bootstrap._gcd_import(name[level:], package, level)
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:ragweed/tests/tests.py:2: in <module>
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:    import ragweed.framework
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:ragweed/framework.py:3: in <module>
2025-07-30T21:22:00.152 INFO:teuthology.orchestra.run.smithi064.stdout:    import boto
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:.tox/py/lib/python3.12/site-packages/boto/__init__.py:27: in <module>
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:    from boto.pyami.config import Config, BotoConfigLocations
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:.tox/py/lib/python3.12/site-packages/boto/pyami/config.py:29: in <module>
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:    from boto.compat import expanduser, ConfigParser, NoOptionError, NoSectionError, StringIO
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:.tox/py/lib/python3.12/site-packages/boto/compat.py:52: in <module>
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:    from boto.vendored.six.moves import filter, http_client, map, _thread, \
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:E   ModuleNotFoundError: No module named 'boto.vendored.six.moves'
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:=========================== short test summary info ============================
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:ERROR ragweed/tests/tests.py
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
2025-07-30T21:22:00.153 INFO:teuthology.orchestra.run.smithi064.stdout:=============================== 1 error in 0.15s ===============================
2025-07-30T21:22:00.180 INFO:teuthology.orchestra.run.smithi064.stdout:py: exit 2 (0.54 seconds) /home/ubuntu/cephtest/ragweed.client.0> python -m pytest -v -m 'not fails_on_rgw' pid=42969
2025-07-30T21:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout:  py: FAIL code 2 (10.50=setup[9.96]+cmd[0.54] seconds)
2025-07-30T21:22:00.182 INFO:teuthology.orchestra.run.smithi064.stdout:  evaluation failed :( (10.57 seconds)

this will block https://github.com/ceph/ceph/pull/63912 which adds ubuntu 24 testing to teuthology

Actions
Copy link
 #1

Updated by Casey Bodley 9 months ago

  • Backport set to squid tentacle
Actions
Copy link
 #2

Updated by Casey Bodley 9 months ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by J. Eric Ivancich 9 months ago

  • Assignee set to Mark Kogan
Actions
Copy link
 #4

Updated by Mark Kogan 9 months ago

quick update - reproduced locally in a ubuntu podman container -

in ubuntu 24.04 the default python version was bumped to 3.12 under which the current boto version is no longer supported (only boto3 is supported)

for discussion in the bug-scrub - from the looks of it lets please discus few options to move forward:

1. refactor ragweed to use boto3

- OR -

2. install python 3.11 on ubuntu 24.04:

example would look something like:

podman run -it --rm --net=host --name=ubu2404 ubuntu:24.04

apt update

apt install -y python3 python3-pip tox
apt install -y virtualenv python3-virtualenv

apt install -y git sudo vim

sudo apt install -y software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa -y
sudo apt install -y python3.11 python3.11-venv python3.11-dev

root@silvertip:/# python3.11 --version
Python 3.11.13

cd ~
git clone -b ceph-master --progress https://github.com/ceph/ragweed 
cd ragweed

vim ./tox.ini
[tox]
envlist = py

[testenv]
deps = -rrequirements.txt
passenv =
  PYTHONPATH
  RAGWEED_CONF
  RAGWEED_STAGES
commands = python -m pytest {posargs}
basepython = python3.11  # <---
:x

tox
...
Actions
Copy link
 #5

Updated by Casey Bodley 8 months ago

  • Subject changed from ragweed: ModuleNotFoundError: No module named 'boto.vendored.six.moves' on ubuntu 24.04 to ragweed/s3tests: ModuleNotFoundError: No module named 'boto.vendored.six.moves' on ubuntu 24.04

i rescheduled a ubuntu 24.04 job with the ragweed task removed to work around this, but saw s3-tests fail in exactly the same way: from https://qa-proxy.ceph.com/teuthology/cbodley-2025-08-21_18:10:17-rgw-main-distro-default-smithi/8456905/teuthology.log

2025-08-21T18:28:35.335 DEBUG:teuthology.orchestra.run.smithi177:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and bucket_logging'
2025-08-21T18:28:35.937 INFO:teuthology.orchestra.run.smithi177.stdout:py: install_deps> python -I -m pip install -r requirements.txt
2025-08-21T18:28:55.215 INFO:teuthology.orchestra.run.smithi177.stdout:.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0'
2025-08-21T18:28:57.061 INFO:teuthology.orchestra.run.smithi177.stdout:.pkg: _optional_hooks> python /home/ubuntu/cephtest/tox-venv/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
2025-08-21T18:28:57.227 INFO:teuthology.orchestra.run.smithi177.stdout:.pkg: get_requires_for_build_sdist> python /home/ubuntu/cephtest/tox-venv/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
2025-08-21T18:28:57.420 INFO:teuthology.orchestra.run.smithi177.stdout:.pkg: get_requires_for_build_wheel> python /home/ubuntu/cephtest/tox-venv/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
2025-08-21T18:28:57.637 INFO:teuthology.orchestra.run.smithi177.stdout:.pkg: prepare_metadata_for_build_wheel> python /home/ubuntu/cephtest/tox-venv/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
2025-08-21T18:28:57.853 INFO:teuthology.orchestra.run.smithi177.stdout:.pkg: build_sdist> python /home/ubuntu/cephtest/tox-venv/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
2025-08-21T18:28:58.117 INFO:teuthology.orchestra.run.smithi177.stdout:py: install_package_deps> python -I -m pip install PyYAML 'boto3>=1.0.0' 'boto>=2.0b4' 'gevent>=1.0' 'isodate>=0.4.4' 'munch>=2.0.0'
2025-08-21T18:28:58.580 INFO:teuthology.orchestra.run.smithi177.stdout:py: install_package> python -I -m pip install --force-reinstall --no-deps /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.tar.gz
2025-08-21T18:29:01.108 INFO:teuthology.orchestra.run.smithi177.stdout:py: commands[0]> pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and bucket_logging'
2025-08-21T18:29:01.402 INFO:teuthology.orchestra.run.smithi177.stdout:============================= test session starts ==============================
2025-08-21T18:29:01.402 INFO:teuthology.orchestra.run.smithi177.stdout:platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python
2025-08-21T18:29:01.402 INFO:teuthology.orchestra.run.smithi177.stdout:cachedir: .tox/py/.pytest_cache
2025-08-21T18:29:01.402 INFO:teuthology.orchestra.run.smithi177.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0
2025-08-21T18:29:01.402 INFO:teuthology.orchestra.run.smithi177.stdout:configfile: pytest.ini
2025-08-21T18:29:03.874 INFO:teuthology.orchestra.run.smithi177.stdout:collecting ... collected 922 items / 4 errors / 832 deselected / 90 selected
2025-08-21T18:29:03.874 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.874 INFO:teuthology.orchestra.run.smithi177.stdout:==================================== ERRORS ====================================
2025-08-21T18:29:03.874 INFO:teuthology.orchestra.run.smithi177.stdout:_____________ ERROR collecting s3tests/functional/test_headers.py ______________
2025-08-21T18:29:03.874 INFO:teuthology.orchestra.run.smithi177.stdout:ImportError while importing test module '/home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py'.
2025-08-21T18:29:03.874 INFO:teuthology.orchestra.run.smithi177.stdout:Hint: make sure your test modules/packages have valid Python names.
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:Traceback:
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:/usr/lib/python3.12/importlib/__init__.py:90: in import_module
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:    return _bootstrap._gcd_import(name[level:], package, level)
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests/functional/__init__.py:3: in <module>
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:    import boto.exception
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/__init__.py:27: in <module>
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.pyami.config import Config, BotoConfigLocations
2025-08-21T18:29:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/pyami/config.py:29: in <module>
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.compat import expanduser, ConfigParser, NoOptionError, NoSectionError, StringIO
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/compat.py:52: in <module>
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.vendored.six.moves import filter, http_client, map, _thread, \
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:E   ModuleNotFoundError: No module named 'boto.vendored.six.moves'
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:________________ ERROR collecting s3tests/functional/test_s3.py ________________
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:ImportError while importing test module '/home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3.py'.
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:Hint: make sure your test modules/packages have valid Python names.
2025-08-21T18:29:03.876 INFO:teuthology.orchestra.run.smithi177.stdout:Traceback:
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:/usr/lib/python3.12/importlib/__init__.py:90: in import_module
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:    return _bootstrap._gcd_import(name[level:], package, level)
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests/functional/__init__.py:3: in <module>
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:    import boto.exception
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/__init__.py:27: in <module>
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.pyami.config import Config, BotoConfigLocations
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/pyami/config.py:29: in <module>
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.compat import expanduser, ConfigParser, NoOptionError, NoSectionError, StringIO
2025-08-21T18:29:03.877 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/compat.py:52: in <module>
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.vendored.six.moves import filter, http_client, map, _thread, \
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:E   ModuleNotFoundError: No module named 'boto.vendored.six.moves'
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:____________ ERROR collecting s3tests/functional/test_s3_website.py ____________
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:ImportError while importing test module '/home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py'.
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:Hint: make sure your test modules/packages have valid Python names.
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:Traceback:
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:/usr/lib/python3.12/importlib/__init__.py:90: in import_module
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:    return _bootstrap._gcd_import(name[level:], package, level)
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests/functional/__init__.py:3: in <module>
2025-08-21T18:29:03.878 INFO:teuthology.orchestra.run.smithi177.stdout:    import boto.exception
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/__init__.py:27: in <module>
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.pyami.config import Config, BotoConfigLocations
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/pyami/config.py:29: in <module>
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.compat import expanduser, ConfigParser, NoOptionError, NoSectionError, StringIO
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/compat.py:52: in <module>
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.vendored.six.moves import filter, http_client, map, _thread, \
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:E   ModuleNotFoundError: No module named 'boto.vendored.six.moves'
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:______________ ERROR collecting s3tests/functional/test_utils.py _______________
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:ImportError while importing test module '/home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_utils.py'.
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:Hint: make sure your test modules/packages have valid Python names.
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:Traceback:
2025-08-21T18:29:03.879 INFO:teuthology.orchestra.run.smithi177.stdout:/usr/lib/python3.12/importlib/__init__.py:90: in import_module
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:    return _bootstrap._gcd_import(name[level:], package, level)
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests/functional/__init__.py:3: in <module>
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:    import boto.exception
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/__init__.py:27: in <module>
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.pyami.config import Config, BotoConfigLocations
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/pyami/config.py:29: in <module>
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.compat import expanduser, ConfigParser, NoOptionError, NoSectionError, StringIO
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:.tox/py/lib/python3.12/site-packages/boto/compat.py:52: in <module>
2025-08-21T18:29:03.880 INFO:teuthology.orchestra.run.smithi177.stdout:    from boto.vendored.six.moves import filter, http_client, map, _thread, \
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:E   ModuleNotFoundError: No module named 'boto.vendored.six.moves'
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:=============================== warnings summary ===============================
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2244
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2244: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:    ["starts-with", "$key", "\$foo"],\
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2259
2025-08-21T18:29:03.881 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2259: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.882 INFO:teuthology.orchestra.run.smithi177.stdout:    payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\
2025-08-21T18:29:03.882 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.882 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2265
2025-08-21T18:29:03.882 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2265: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.882 INFO:teuthology.orchestra.run.smithi177.stdout:    response = client.get_object(Bucket=bucket_name, Key='\$foo.txt')
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2322
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2322: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:    ["starts-with", "$key", "\$foo"],\
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2337
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2337: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:    payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\
2025-08-21T18:29:03.883 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2355
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2355: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:    ["starts-with", "$key", "\$foo"],\
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2370
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2370: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:    payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id[::-1]),\
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2388
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2388: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.884 INFO:teuthology.orchestra.run.smithi177.stdout:    ["starts-with", "$key", "\$foo"],\
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2403
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2403: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:    payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2453
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2453: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:    ["starts-with", "$key", "\$foo"],\
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.885 INFO:teuthology.orchestra.run.smithi177.stdout:s3tests_boto3/functional/test_s3.py:2485
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:  /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2485: SyntaxWarning: invalid escape sequence '\$'
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:    ["starts-with", "$key", "\$foo"],\
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:=========================== short test summary info ============================
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR s3tests/functional/test_headers.py
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR s3tests/functional/test_s3.py
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR s3tests/functional/test_s3_website.py
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR s3tests/functional/test_utils.py
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!
2025-08-21T18:29:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:================ 832 deselected, 11 warnings, 4 errors in 2.40s ================

Actions
Copy link
 #6

Updated by Mark Kogan 8 months ago

reviewing work-in-progress branch/commit:

WIP branch: https://github.com/mkogan1/ragweed/commits/wip-ragweed-to-boto3/
-->
WIP commit: https://github.com/mkogan1/ragweed/commit/a4a005b9fc8a0bd922f91629614aa8430de89ab1

https://i.imgur.com/su0zwDF.png

❯ env RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check python3.12 -m tox -r -- -v
py: remove tox env folder /mnt/osd--host/src-git/ragweed--mk/.tox/py
.pkg: remove tox env folder /mnt/osd--host/src-git/ragweed--mk/.tox/.pkg
py: install_deps> python -I -m pip install -r requirements.txt
.pkg: install_requires> python -I -m pip install 'setuptools>=61.0'
.pkg: _optional_hooks> python /home/mkogan/.local/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /home/mkogan/.local/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_sdist> python /home/mkogan/.local/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py: install_package_deps> python -I -m pip install PyYAML boto3 gevent isodate munch
py: install_package> python -I -m pip install --force-reinstall --no-deps /mnt/osd--host/src-git/ragweed--mk/.tox/.tmp/package/12/ragweed-0.0.1.tar.gz
py: commands[0]> python -m pytest -v
======================================================================================= test session starts =======================================================================================
platform linux -- Python 3.12.8, pytest-8.4.1, pluggy-1.6.0 -- /mnt/osd--host/src-git/ragweed--mk/.tox/py/bin/python
cachedir: .tox/py/.pytest_cache
rootdir: /mnt/osd--host/src-git/ragweed--mk
configfile: pytest.ini
collected 9 items                                                                                                                                                                                 

ragweed/tests/tests.py::r_test_small_obj_data::test PASSED                                                                                                                                  [ 11%]
ragweed/tests/tests.py::r_test_multipart_simple::test PASSED                                                                                                                                [ 22%]
ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED                                                                                                                      [ 33%]
ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED                                                                                                                        [ 44%]
ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED                                                                                                                 [ 55%]
ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED                                                                                                                               [ 66%]
ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED                                                                                                                     [ 77%]
ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED                                                                                                                          [ 88%]
ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED                                                                                                                [100%]

======================================================================================== 9 passed in 8.30s ========================================================================================
  py: OK (23.16=setup[14.44]+cmd[8.72] seconds)
  congratulations :) (23.21 seconds)
Actions
Copy link
 #7

Updated by Casey Bodley 8 months ago

  • Project changed from rgw to rgw-testing
  • Subject changed from ragweed/s3tests: ModuleNotFoundError: No module named 'boto.vendored.six.moves' on ubuntu 24.04 to ragweed: ModuleNotFoundError: No module named 'boto.vendored.six.moves' on ubuntu 24.04/python3.12
  • Status changed from New to In Progress

split the s3tests part out to https://tracker.ceph.com/issues/72876

Actions
Copy link
 #8

Updated by Mark Kogan 6 months ago

there is a way to install python 3.11 on RHEL 10:

## https://github.com/pyenv/pyenv -- Simple Python Version Management: pyenv

sudo dnf groupinstall "Development Tools" 
sudo dnf install openssl-devel bzip2-devel libffi-devel zlib-devel readline-devel

# Install pyenv
curl -fsSL https://pyenv.run | bash

# Add to shell and restart terminal -- https://github.com/pyenv/pyenv?tab=readme-ov-file#zsh 
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(pyenv init - zsh)"' >> ~/.zshrc
exec zsh

# Build Python 3.11
pyenv install 3.11.11

# Enable Python 3.11
pyenv global 3.11.11

# Verify
python --version  # Should show 3.11.11

# Verify sudo
sudo -E env "PATH=$PATH" python --version  # Should show 3.11.11
Actions
Copy link

Also available in: Atom PDF