[bug] pylint + pylint-django + django + pulp + 'dynaconf_merge' in INSTALLED_APPS fails #578
Description
Describe the bug
This is likely a case of https://www.dynaconf.com/django/#known-caveats but filing in case there is more to it.
Attempting to use pylint against default galaxy_ng settings throws exceptions if djangos settings INSTALLED_APPS
has a 'dynaconf_merge' item. If 'dynaconf_merge' is removed, it works.
In this case, the django settings file is galaxy_ng's galaxy_ng/app/settings.py which incudes:
INSTALLED_APPS = [
'rest_framework.authtoken',
'dynaconf_merge',
]With 'dynaconf_merge' in there, pylint throws a stack of exceptions (full traceback further below):
- django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
- ModuleNotFoundError: No module named 'dynaconf_merge'
- KeyError: None
The ImproperlyConfigured seems to be a result of any settings error that result in circular imports amongst other things.
The ModuleNotFoundError is from pylint->pylint-django->django->settings.configure() presumably because of https://www.dynaconf.com/django/#known-caveats . pylint-django calls settings.configure() explicitly at
https://github.com/PyCQA/pylint-django/blob/master/pylint_django/checkers/foreign_key_strings.py#L87-L135
To Reproduce
Steps to reproduce the behavior:
- Having the following config files:
Config files
https://github.com/pulp/pulpcore/blob/master/pulpcore/app/settings.py
https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/settings.py
/path/.env
No .env* or *.env files.
and
/path/settings.toml
No settings.toml
- Executing under the following environment
Execution
# ~/venvs/galaxy_ng is current activated python 3.9 virtualenv.
$ pylint --load-plugins pylint_django --django-settings-module=galaxy_ng.app.alikins galaxy_ng
Traceback (most recent call last):
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint_django/checkers/foreign_key_strings.py", line 90, in open
django.setup()
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/conf/__init__.py", line 79, in __getattr__
self._setup(name)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/conf/__init__.py", line 66, in _setup
self._wrapped = Settings(settings_module)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/conf/__init__.py", line 177, in __init__
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint_django/checkers/foreign_key_strings.py", line 114, in open
django.setup()
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/django/apps/config.py", line 90, in create
module = import_module(entry)
File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'dynaconf_merge'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/adrian/venvs/galaxy_ng/bin/pylint", line 8, in <module>
sys.exit(run_pylint())
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/__init__.py", line 24, in run_pylint
PylintRun(sys.argv[1:])
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/lint/run.py", line 381, in __init__
linter.check(args)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/lint/pylinter.py", line 873, in check
self._check_files(
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/lint/pylinter.py", line 905, in _check_files
with self._astroid_module_checker() as check_astroid_module:
File "/usr/lib64/python3.9/contextlib.py", line 117, in __enter__
return next(self.gen)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/lint/pylinter.py", line 1014, in _astroid_module_checker
checker.open()
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint_django/checkers/foreign_key_strings.py", line 118, in open
self.add_message(
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/checkers/base_checker.py", line 110, in add_message
self.linter.add_message(msgid, line, node, args, confidence, col_offset)
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/message/message_handler_mix_in.py", line 224, in add_message
self.add_one_message(
File "/home/adrian/venvs/galaxy_ng/lib64/python3.9/site-packages/pylint/message/message_handler_mix_in.py", line 278, in add_one_message
self.stats["by_module"][self.current_name][msg_cat] += 1
KeyError: None
With the 'dynaconf_merge' removed from galaxy_ng/app/settings.py:INSTALLED_APPS:
************* Module galaxy_ng.tests.functional.utils
galaxy_ng/tests/functional/utils.py:8:0: E0401: Unable to import 'pulp_smash' (import-error)
galaxy_ng/tests/functional/utils.py:9:0: E0401: Unable to import 'pulp_smash.pulp3.utils' (import-error)
galaxy_ng/tests/functional/utils.py:34:0: E0611: No name 'galaxy_ng' in module 'pulpcore.client' (no-name-in-module)
galaxy_ng/tests/functional/utils.py:34:0: E0401: Unable to import 'pulpcore.client.galaxy_ng' (import-error)
galaxy_ng/tests/functional/utils.py:131:4: W0621: Redefining name 'tasks' from outer scope (line 143) (redefined-outer-name)
galaxy_ng/tests/functional/utils.py:4:0: C0411: standard import "from unittest import SkipTest" should be placed before "import requests" (wrong-import-order)
galaxy_ng/tests/functional/utils.py:5:0: C0411: standard import "from time import sleep" should be placed before "import requests" (wrong-import-order)
... < lots more output but no tracebacks etc>Expected behavior
Basically the same behavior as the 'no dynaconf_merge' example above. But without having to remove
'dynaconf_merge' (and missing whatever config it pulls in).
Environment (please complete the following information):
- OS: Fedora 33
- Dynaconf Version: 3.1.4
- Frameworks in use: django, django-rest-framework, pulp, galaxy_ng, pylint
astroid 2.5.6
dynaconf 3.1.4
galaxy-ng 4.4.0.dev0 /home/adrian/src/galaxy_ng
pulp-ansible 0.7.1
pulp-container 2.5.2
pulpcore 3.11.0.post0
pulpcore-client 3.10.0
pylint 2.8.2
pylint-django 2.4.4
pylint-plugin-utils 0.6
Additional context
Add any other context about the problem here.