Project

General

Profile

Actions

Bug #70891

open

run-tox-mgr-dashboard-py3 - No socket could be created -- (('127.0.0.1', 54583): [Errno 98] Address already in use)

Added by Bill Scales 12 months ago. Updated 6 months ago.

Status:
New
Priority:
High
Assignee:
-
Category:
Build, CI, Dependencies & Tools
Target version:
-
% Done:

0%

Source:
Backport:
tentacle
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

https://jenkins.ceph.com/job/ceph-pull-requests-arm64/72074/

287/314 Test   #7: run-tox-mgr-dashboard-py3 ........................***Failed  157.01 sec
Requirement already satisfied: tox in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (4.25.0)
Requirement already satisfied: cachetools>=5.5.1 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (5.5.2)
Requirement already satisfied: chardet>=5.2 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (5.2.0)
Requirement already satisfied: colorama>=0.4.6 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (0.4.6)
Requirement already satisfied: filelock>=3.16.1 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (3.18.0)
Requirement already satisfied: packaging>=24.2 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (24.2)
Requirement already satisfied: platformdirs>=4.3.6 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (4.3.7)
Requirement already satisfied: pluggy>=1.5 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (1.5.0)
Requirement already satisfied: pyproject-api>=1.8 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (1.9.0)
Requirement already satisfied: tomli>=2.2.1 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (2.2.1)
Requirement already satisfied: typing-extensions>=4.12.2 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (4.13.2)
Requirement already satisfied: virtualenv>=20.29.1 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from tox) (20.30.0)
Requirement already satisfied: distlib<1,>=0.3.7 in /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/mgr-dashboard-py3-virtualenv/lib/python3.10/site-packages (from virtualenv>=20.29.1->tox) (0.3.9)
py3: install_deps /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard> python -I -m pip install -r requirements.txt -r requirements-test.txt -r requirements-extra.txt -c constraints.txt
py3: freeze /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard> python -m pip freeze --all
py3: attrs==25.3.0,bcrypt==3.2.2,-e git+https://github.com/ceph/ceph.git@01a4536f8fd6cbac419f749a145c037234c0e8af#egg=ceph&subdirectory=src/python-common,certifi==2025.1.31,cffi==1.17.1,charset-normalizer==3.4.1,cheroot==10.0.1,CherryPy==13.1.0,coverage==7.8.0,cryptography==44.0.2,grpcio==1.46.5,grpcio-tools==1.46.5,idna==3.10,iniconfig==2.1.0,isodate==0.7.2,jaraco.functools==4.1.0,jmespath==1.0.1,jsonpatch==1.33,jsonpointer==3.0.0,jsonschema==4.23.0,jsonschema-specifications==2024.10.1,lxml==4.8.0,more-itertools==8.14.0,natsort==8.4.0,packaging==24.2,pip==25.0.1,pluggy==1.5.0,portend==3.2.0,prettytable==3.16.0,protobuf==3.20.3,py==1.11.0,pycparser==2.22,pyfakefs==4.5.0,PyJWT==2.10.1,pyOpenSSL==25.0.0,pytest==7.0.1,pytest-cov==6.1.1,pytest-instafail==0.5.0,python-dateutil==2.9.0.post0,python3-saml==1.16.0,PyYAML==6.0.2,referencing==0.36.2,repoze.lru==0.7,requests==2.32.3,Routes==2.5.1,rpds-py==0.24.0,setuptools==78.1.0,six==1.17.0,tempora==5.8.0,tomli==2.2.1,typing_extensions==4.13.2,urllib3==2.4.0,wcwidth==0.2.13,wheel==0.45.1,xmlsec==1.3.15,xmltodict==0.14.2
py3: commands[0] /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard> pytest
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.0.1, pluggy-1.5.0
cachedir: .tox/py3/.pytest_cache
rootdir: /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard, configfile: tox.ini
plugins: instafail-0.5.0, cov-6.1.1, pyfakefs-4.5.0
collected 583 items

tools.py ....                                                            [  0%]
controllers/_version.py ......                                           [  1%]
controllers/rgw.py ..                                                    [  2%]
services/rgw_client.py .                                                 [  2%]
tests/helper.py .                                                        [  2%]
tests/test_access_control.py ........................................... [  9%]
..........................                                               [ 14%]
tests/test_api_auditing.py .E

_____________ ERROR at setup of ApiAuditingTest.test_audit_delete ______________

cls = <class 'dashboard.tests.test_api_auditing.ApiAuditingTest'>

    @classmethod
    def setup_class(cls):
        ''
        # Creates a server
        conf = {
            'scheme': 'http',
            'protocol': 'HTTP/1.1',
            'port': 54583,
            'host': '127.0.0.1',
            'validate': False,
            'server': 'wsgi',
        }
        supervisor_factory = cls.available_servers.get(
            conf.get('server', 'wsgi'))
        if supervisor_factory is None:
            raise RuntimeError('Unknown server in config: %s' % conf['server'])
        supervisor = supervisor_factory(**conf)

        # Copied from "run_test_suite" 
        cherrypy.config.reset()
        baseconf = cls._setup_server(supervisor, conf)
        cherrypy.config.update(baseconf)
        setup_client()

        if hasattr(cls, 'setup_server'):
            # Clear the cherrypy tree and clear the wsgi server so that
            # it can be updated with the new root
            cherrypy.tree = cherrypy._cptree.Tree()
            cherrypy.server.httpserver = None
            cls.setup_server()
            # Add a resource for verifying there are no refleaks
            # to *every* test class.
            cherrypy.tree.mount(gctools.GCRoot(), '/gc')
            cls.do_gc_test = True
>           supervisor.start(cls.__module__)

.tox/py3/lib/python3.10/site-packages/cherrypy/test/helper.py:276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py3/lib/python3.10/site-packages/cherrypy/test/helper.py:81: in start
    cherrypy.engine.start()
.tox/py3/lib/python3.10/site-packages/cherrypy/process/wspbus.py:283: in start
    raise e_info
.tox/py3/lib/python3.10/site-packages/cherrypy/process/wspbus.py:268: in start
    self.publish('start')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <cherrypy.process.wspbus.Bus object at 0xea7692c28040>, channel = 'start'
args = (), kwargs = {}
exc = OSError("No socket could be created -- (('127.0.0.1', 54583): [Errno 98] Address already in use)")
output = [None, None]
raw_items = <generator object Bus.publish.<locals>.<genexpr> at 0xea768d5cd150>
items = [(50, <bound method RequestCounter.start of <cherrypy.lib.gctools.RequestCounter object at 0xea768dfeeda0>>), (50, <ch... object at 0xea7692648820>), (75, <bound method Server.start of <cherrypy._cpserver.Server object at 0xea76927aa7a0>>)]
priority = 75
listener = <bound method Server.start of <cherrypy._cpserver.Server object at 0xea76927aa7a0>>

    def publish(self, channel, *args, **kwargs):
        """Return output of all subscribers for the given channel.""" 
        if channel not in self.listeners:
            return []

        exc = ChannelFailures()
        output = []

        raw_items = (
            (self._priorities[(channel, listener)], listener)
            for listener in self.listeners[channel]
        )
        items = sorted(raw_items, key=operator.itemgetter(0))
        for priority, listener in items:
            try:
                output.append(listener(*args, **kwargs))
            except KeyboardInterrupt:
                raise
            except SystemExit:
                e = sys.exc_info()[1]
                # If we have previous errors ensure the exit code is non-zero
                if exc and e.code == 0:
                    e.code = 1
                raise
            except Exception:
                exc.handle_exception()
                if channel == 'log':
                    # Assume any further messages to 'log' will fail.
                    pass
                else:
                    self.log('Error in %r listener %r' % (channel, listener),
                             level=40, traceback=True)
        if exc:
>           raise exc
E           cherrypy.process.wspbus.ChannelFailures: OSError("No socket could be created -- (('127.0.0.1', 54583): [Errno 98] Address already in use)")

.tox/py3/lib/python3.10/site-packages/cherrypy/process/wspbus.py:248: ChannelFailures
------------------------------ Captured log setup ------------------------------
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Listening for SIGTERM.
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Listening for SIGHUP.
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Listening for SIGUSR1.
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Bus STARTING
ERROR    cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Error in HTTP server: shutting down
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard/.tox/py3/lib/python3.10/site-packages/cherrypy/process/servers.py", line 225, in _start_http_thread
    self.httpserver.start()
  File "/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard/.tox/py3/lib/python3.10/site-packages/cheroot/server.py", line 1844, in start
    self.prepare()
  File "/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard/.tox/py3/lib/python3.10/site-packages/cheroot/server.py", line 1799, in prepare
    raise socket.error(msg)
OSError: No socket could be created -- (('127.0.0.1', 54583): [Errno 98] Address already in use)

INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Bus STOPPING
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 54583)) already shut down
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Bus STOPPED
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Bus EXITING
INFO     cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Bus EXITED
ERROR    cherrypy.error:_cplogging.py:220 [11/Apr/2025:06:39:56] ENGINE Error in 'start' listener <bound method Server.start of <cherrypy._cpserver.Server object at 0xea76927aa7a0>>
Actions #2

Updated by Casey Bodley 11 months ago

  • Category set to Build, CI, Dependencies & Tools
  • Priority changed from Normal to High

https://jenkins.ceph.com/job/ceph-pull-requests/158033/

The following tests FAILED:
7 - run-tox-mgr-dashboard-py3 (Failed)

log has 263 matches for "ERROR at setup of"

Actions #3

Updated by Casey Bodley 11 months ago

  • Backport set to tentacle
Actions #4

Updated by Casey Bodley 6 months ago

https://jenkins.ceph.com/job/ceph-pull-requests-arm64/81428/

The following tests FAILED:
      7 - run-tox-mgr-dashboard-py3 (Failed)
     31 - run-rbd-unit-tests-1.sh (Failed)


============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.0.1, pluggy-1.6.0
cachedir: .tox/py3/.pytest_cache
rootdir: /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard, configfile: tox.ini
plugins: pyfakefs-4.5.0, instafail-0.5.0, cov-7.0.0
collected 602 items

tools.py ....                                                            [  0%]
controllers/_version.py ......                                           [  1%]
controllers/rgw.py ..                                                    [  1%]
services/rgw_client.py .                                                 [  2%]
tests/helper.py .                                                        [  2%]
tests/test_access_control.py ........................................... [  9%]
..........................                                               [ 13%]
Exception in thread HTTPServer Thread-1 (_start_http_thread):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard/.tox/py3/lib/python3.10/site-packages/cherrypy/process/servers.py", line 225, in _start_http_thread
    self.httpserver.start()
  File "/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard/.tox/py3/lib/python3.10/site-packages/cheroot/server.py", line 1844, in start
    self.prepare()
  File "/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/pybind/mgr/dashboard/.tox/py3/lib/python3.10/site-packages/cheroot/server.py", line 1799, in prepare
    raise socket.error(msg)
OSError: No socket could be created -- (('127.0.0.1', 54583): [Errno 98] Address already in use)
tests/test_api_auditing.py .E

_____________ ERROR at setup of ApiAuditingTest.test_audit_delete ______________

Actions

Also available in: Atom PDF