Skip to content

Celery does not consider authMechanism on mongodb backend URLs #5734

New issue
New issue
Closed
#5795
Closed
Celery does not consider authMechanism on mongodb backend URLs#5734
#5795
@spengjie

Description

@spengjie
spengjie
opened on Sep 16, 2019

Checklist

  • I have read the relevant section in the
    contribution guide
    on reporting bugs.
  • I have checked the issues list
    for similar or identical bug reports.
  • I have checked the pull requests list
    for existing proposed fixes.
  • I have checked the commit log
    to find out if the bug was already fixed in the master branch.
  • I have included all related issues and possible duplicate issues
    in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

  • I have included the output of celery -A proj report in the issue.
    (if you are not able to do this, then at least specify the Celery
    version affected).
  • I have verified that the issue exists against the master branch of Celery.
  • I have included the contents of pip freeze in the issue.
  • I have included all the versions of all the external dependencies required
    to reproduce this bug.

Optional Debugging Information

  • I have tried reproducing the issue on more than one Python version
    and/or implementation.
  • I have tried reproducing the issue on more than one message broker and/or
    result backend.
  • I have tried reproducing the issue on more than one version of the message
    broker and/or result backend.
  • I have tried reproducing the issue on more than one operating system.
  • I have tried reproducing the issue on more than one workers pool.
  • I have tried reproducing the issue with autoscaling, retries,
    ETA/Countdown & rate limits disabled.
  • I have tried reproducing the issue after downgrading
    and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

Possible Duplicates

  • None

Environment & Settings

Celery version: 4.3.0 with fixes in PR #5527

celery report Output: 

software -> celery:4.3.0 (rhubarb) kombu:4.6.4 py:3.7.4
            billiard:3.6.1.0 py-amqp:2.5.1
platform -> system:Windows arch:64bit, WindowsPE
            kernel version:10 imp:CPython
loader   -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

Steps to Reproduce

  1. Give Celery a Backend URL pointing to a MongoDB like below:
    mongodb://admin:admin@192.168.0.1/task?authSource=admin&authMechanism=SCRAM-SHA-256
  2. Start Celery worker.
  3. Send any task.

Required Dependencies

  • Minimal Python Version: 3.0
  • Minimal Celery Version: 3.0
  • Minimal Kombu Version: Unknown
  • Minimal Broker Version: Unknown
  • Minimal Result Backend Version: MongoDB 4.0
  • Minimal OS and/or Kernel Version: N/A
  • Minimal Broker Client Version: Unknown
  • Minimal Result Backend Client Version: Unknown

Python Packages

pip freeze Output: 

amqp==2.5.1
asn1crypto==0.24.0
astroid==2.2.5
bcrypt==3.1.7
billiard==3.6.1.0
celery==4.3.0
cffi==1.12.3
Click==7.0
colorama==0.4.1
cryptography==2.7
irectory=backend
Flask==1.1.1
importlib-metadata==0.19
isort==4.3.21
itsdangerous==1.1.0
Jinja2==2.10.1
kombu==4.6.4
lazy-object-proxy==1.4.2
MarkupSafe==1.1.1
mccabe==0.6.1
more-itertools==7.2.0
paramiko==2.6.0
pycparser==2.19
pylint==2.3.1
pymodm==0.4.1
pymongo==3.9.0
PyNaCl==1.3.0
pytz==2019.2
six==1.12.0
typed-ast==1.4.0
vine==1.3.0
Werkzeug==0.15.5
wrapt==1.11.2
zipp==0.6.0

Other Dependencies

MongoDB:

Version: >= 3.0.0

Minimally Reproducible Test Case

Details 







Expected Behavior



Support all the authentication options listed at authentication-options, including authSource, authMechanism, authMechanismProperties and gssapiServiceName.


Actual Behavior



An error occurred as below.


Traceback (most recent call last):
  File "D:\GitHub\org\dev\.venv\lib\site-packages\kombu\utils\objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'collection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\GitHub\org\dev\.venv\lib\site-packages\kombu\utils\objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'database'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\user\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\ptvsd_launcher.py", line 43, in <module>
    main(ptvsdArgs)
  File "c:\Users\user\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd\__main__.py", line 432, in main
    run()
  File "c:\Users\user\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd\__main__.py", line 316, in run_file
    runpy.run_path(target, run_name='__main__')
  File "C:\Program Files\Python37\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Program Files\Python37\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Program Files\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "d:\GitHub\org\dev\backend\tests\workertest.py", line 22, in <module>
    print('Task finished? ', result.ready())
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\result.py", line 313, in ready
    return self.state in self.backend.READY_STATES
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\result.py", line 473, in state
    return self._get_task_meta()['status']
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\result.py", line 412, in _get_task_meta
    return self._maybe_set_cache(self.backend.get_task_meta(self.id))
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\backends\base.py", line 386, in get_task_meta
    meta = self._get_task_meta_for(task_id)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\backends\mongodb.py", line 206, in _get_task_meta_for
    obj = self.collection.find_one({'_id': task_id})
  File "D:\GitHub\org\dev\.venv\lib\site-packages\kombu\utils\objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\backends\mongodb.py", line 293, in collection
    collection = self.database[self.taskmeta_collection]
  File "D:\GitHub\org\dev\.venv\lib\site-packages\kombu\utils\objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\backends\mongodb.py", line 288, in database
    return self._get_database()
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\backends\mongodb.py", line 271, in _get_database
    conn = self._get_connection()
  File "D:\GitHub\org\dev\.venv\lib\site-packages\celery\backends\mongodb.py", line 161, in _get_connection
    self._connection = MongoClient(**conf)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\pymongo\mongo_client.py", line 668, in __init__
    username, password, dbase, opts)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\pymongo\client_options.py", line 151, in __init__
    username, password, database, options)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\pymongo\client_options.py", line 39, in _parse_credentials
    mechanism, source, username, password, options, database)
  File "D:\GitHub\org\dev\.venv\lib\site-packages\pymongo\auth.py", line 107, in _build_credentials_tuple
    raise ConfigurationError("%s requires a username." % (mech,))
pymongo.errors.ConfigurationError: SCRAM-SHA-256 requires a username.
Metadata
Metadata
Assignees
No one assigned
    Labels
    No labels
    No labels
    Type
    No type
    Projects
    No projects
    Milestone
    No milestone
    Relationships
    None yet
    Development
    No branches or pull requests
    Issue actions