Skip to content

qa/rgw/keystone: bump repo again to stable/2025.2#67549

Closed
cbodley wants to merge 15 commits intoceph:mainfrom
cbodley:wip-75041
Closed

qa/rgw/keystone: bump repo again to stable/2025.2#67549
cbodley wants to merge 15 commits intoceph:mainfrom
cbodley:wip-75041

Conversation

@cbodley
Copy link
Contributor

@cbodley cbodley commented Feb 26, 2026

This reverts commit 75570ce.

the keystone task recently started failing with

ModuleNotFoundError: No module named 'pkg_resources' because of keystone dependency

this was fixed upstream in openstack/osc-lib@b6d1d53, but we've been pinned to an older osc-lib without this fix. the same commit appears to resolve issues with newer 'cliff' versions which this reverted commit also mentioned as a constraint

Fixes: https://tracker.ceph.com/issues/75041

Show available Jenkins commands

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

@github-actions github-actions bot added the tests label Feb 26, 2026
@cbodley cbodley added the rgw label Feb 26, 2026
@cbodley
qa/rgw/keystone: bump repo again to stable/2025.2
0e075db 
Fixes: https://tracker.ceph.com/issues/75041

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley
Revert "qa/keystone: pin python-openstackclient and osc-lib"
24344f2 
This reverts commit 75570ce.

the keystone task recently started failing with
> ModuleNotFoundError: No module named 'pkg_resources' because of keystone dependency

this was fixed upstream in openstack/osc-lib@b6d1d53,
but we've been pinned to an older osc-lib without this fix. the same
commit appears to resolve issues with newer 'cliff' versions which this
reverted commit also mentioned as a constraint

Fixes: https://tracker.ceph.com/issues/75041

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley cbodley changed the title Revert "qa/keystone: pin python-openstackclient and osc-lib" qa/rgw/keystone: bump repo again to stable/2025.2 Feb 26, 2026
@cbodley
Copy link
Contributor Author

cbodley commented Feb 26, 2026

this seems to get past the python issues, but it looks like the openstack project create command fails to authenticate with connection errors? from https://qa-proxy.ceph.com/teuthology/cbodley-2026-02-26_16:11:11-rgw:tempest-main-distro-default-trial/72568/teuthology.log

2026-02-26T16:24:13.005 INFO:teuthology.orchestra.run.trial145.stderr:Using auth plugin: password
2026-02-26T16:24:13.018 INFO:teuthology.orchestra.run.trial145.stderr:Using parameters {'auth_url': 'http://trial145.front.sepia.ceph.com:5000/v3', 'project_name': 'admin', 'project_domain_id': 'default', 'username': 'admin', 'user_domain_id': 'default', 'password': '***'}
2026-02-26T16:24:13.030 INFO:teuthology.orchestra.run.trial145.stderr:Get auth_ref
2026-02-26T16:24:13.043 INFO:teuthology.orchestra.run.trial145.stderr:REQ: curl -g -i -X GET http://trial145.front.sepia.ceph.com:5000/v3 -H "Accept: application/json" -H "User-Agent: openstacksdk/4.10.0 keystoneauth1/5.12.0 python-requests/2.32.4 CPython/3.10.12"
2026-02-26T16:24:13.054 INFO:teuthology.orchestra.run.trial145.stderr:Starting new HTTP connection (1): trial145.front.sepia.ceph.com:5000
2026-02-26T16:24:13.066 INFO:teuthology.orchestra.run.trial145.stderr:Failed to discover available identity versions when contacting http://trial145.front.sepia.ceph.com:5000/v3. Attempting to parse version from URL.
2026-02-26T16:24:13.079 INFO:teuthology.orchestra.run.trial145.stderr:Making authentication request to http://trial145.front.sepia.ceph.com:5000/v3/auth/tokens
2026-02-26T16:24:13.090 INFO:teuthology.orchestra.run.trial145.stderr:Starting new HTTP connection (2): trial145.front.sepia.ceph.com:5000
2026-02-26T16:24:13.102 INFO:teuthology.orchestra.run.trial145.stderr:Unable to establish connection to http://trial145.front.sepia.ceph.com:5000/v3/auth/tokens: HTTPConnectionPool(host='trial145.front.sepia.ceph.com', port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f06e8e1ac80>: Failed to establish a new connection: [Errno 111] Connection refused'))
2026-02-26T16:24:13.115 INFO:teuthology.orchestra.run.trial145.stderr:Traceback (most recent call last):
2026-02-26T16:24:13.129 INFO:teuthology.orchestra.run.trial145.stderr:  File "/home/ubuntu/cephtest/keystone/.tox/venv/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
2026-02-26T16:24:13.141 INFO:teuthology.orchestra.run.trial145.stderr:    conn = connection.create_connection(
2026-02-26T16:24:13.153 INFO:teuthology.orchestra.run.trial145.stderr:  File "/home/ubuntu/cephtest/keystone/.tox/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection
2026-02-26T16:24:13.164 INFO:teuthology.orchestra.run.trial145.stderr:    raise err
2026-02-26T16:24:13.179 INFO:teuthology.orchestra.run.trial145.stderr:  File "/home/ubuntu/cephtest/keystone/.tox/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
2026-02-26T16:24:13.191 INFO:teuthology.orchestra.run.trial145.stderr:    sock.connect(sa)
2026-02-26T16:24:13.202 INFO:teuthology.orchestra.run.trial145.stderr:ConnectionRefusedError: [Errno 111] Connection refused

@cbodley
Copy link
Contributor Author

cbodley commented Feb 26, 2026

^ cc @tobias-urdin, any idea what may have changed with 2025.2?

@cbodley
Copy link
Contributor Author

cbodley commented Feb 26, 2026

this error just before 'openstack project create' command looks relevant?

2026-02-26T17:06:39.125 INFO:tasks.keystone.client.0.trial173.stderr:/home/ubuntu/cephtest/keystone/.tox/venv/bin/python: can't open file '/home/ubuntu/cephtest/keystone/.tox/venv/bin/keystone-wsgi-public': [Errno 2] No such file or directory

@cbodley
Copy link
Contributor Author

cbodley commented Feb 26, 2026

ref https://docs.openstack.org/releasenotes/keystone/en_GB/2025.2.html

The WSGI scripts, keystone-wsgi-admin and keystone-wsgi-public, have been removed. Deployment tooling should instead reference the Python module paths for the service, keystone.wsgi.api, if their chosen WSGI server supports this (gunicorn, uWSGI) or implement a .wsgi script themselves if not (mod_wsgi).

@cbodley cbodley mentioned this pull request Feb 26, 2026
Closed
14 tasks
@cbodley
Copy link
Contributor Author

cbodley commented Feb 26, 2026

ref https://docs.openstack.org/releasenotes/keystone/en_GB/2025.2.html

The WSGI scripts, keystone-wsgi-admin and keystone-wsgi-public, have been removed. Deployment tooling should instead reference the Python module paths for the service, keystone.wsgi.api, if their chosen WSGI server supports this (gunicorn, uWSGI) or implement a .wsgi script themselves if not (mod_wsgi).

oops, i now realize this is directly related to #66517. i'll try to test the combined branches

@tobias-urdin @cbodley
qa/keystone: use uwsgi for daemon
ce414de 
Instead of using the builtin development server that
can only serve a single request and is discouraged
start Keystone with uwsgi instead.

Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
@tobias-urdin
Copy link
Contributor

tobias-urdin commented Feb 27, 2026

I updated #66517 so that it loads Keystone from module instead which is required for 2024.2 – you can try that again and see if it starts/works now. Let me know how it goes and I will have a look asap.

@kshtsk kshtsk mentioned this pull request Mar 5, 2026
Merged
14 tasks
@cbodley
Copy link
Contributor Author

cbodley commented Mar 5, 2026

closing for #67660

@cbodley cbodley closed this Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

rgw tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@cbodley @tobias-urdin