Airflow fails to initdb with cattrs 1.1.0 #11965
Description
Apache Airflow version:
1.10.12
Kubernetes version (if you are using kubernetes) (use kubectl version):
N/A
Environment:
- Cloud provider or hardware configuration:
- OS (e.g. from /etc/os-release): Centos 7
- Kernel (e.g.
uname -a): 3.10.0-229.el7.x86_64 - Install tools: pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
- Others:
What happened:
Following the instructions here, I encountered an issue at the airflow initdb stage:
[xxx@xxx ~]# airflow initdb
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 26, in <module>
from airflow.bin.cli import CLIFactory
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 94, in <module>
api_module = import_module(conf.get('cli', 'api_client')) # type: Any
File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.6/site-packages/airflow/api/client/local_client.py", line 24, in <module>
from airflow.api.common.experimental import delete_dag
File "/usr/local/lib/python3.6/site-packages/airflow/api/common/experimental/delete_dag.py", line 26, in <module>
from airflow.models.serialized_dag import SerializedDagModel
File "/usr/local/lib/python3.6/site-packages/airflow/models/serialized_dag.py", line 35, in <module>
from airflow.serialization.serialized_objects import SerializedDAG
File "/usr/local/lib/python3.6/site-packages/airflow/serialization/serialized_objects.py", line 28, in <module>
import cattr
File "/usr/local/lib/python3.6/site-packages/cattr/__init__.py", line 1, in <module>
from .converters import Converter, GenConverter, UnstructureStrategy
File "/usr/local/lib/python3.6/site-packages/cattr/converters.py", line 16, in <module>
from attr import fields, resolve_types
ImportError: cannot import name 'resolve_types'
What you expected to happen:
I expected the Airflow DB to be initialised as per the instructions.
How to reproduce it:
On a fresh installation, follow the Quick Start guide until the apache initdb stage.
Anything else we need to know:
Investigation suggests that this was caused by a Python dependency; specifically cattrs==1.1.0, which was released yesterday (2020-10-29). Downgrading cattrs manually to 1.0.0 does fix the issue and allows the Airflow database to be initialised:
[xxx@xxx ~]# pip3 install cattrs==1.0.0
Collecting cattrs==1.0.0
Downloading https://files.pythonhosted.org/packages/17/5b/6afbdaeb066ecf8ca28d85851048103ac80bb169491a54a14bd39823c422/cattrs-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: attrs>=17.3 in /usr/local/lib/python3.6/site-packages (from cattrs==1.0.0)
Installing collected packages: cattrs
Found existing installation: cattrs 1.1.0
Uninstalling cattrs-1.1.0:
Successfully uninstalled cattrs-1.1.0
Successfully installed cattrs-1.0.0
[xxx@xxx ~]# airflow initdb
DB: sqlite:////path/to/airflow.db
[2020-10-30 09:36:05,431] {db.py:378} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema
INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
/usr/local/lib/python3.6/site-packages/alembic/ddl/sqlite.py:44: UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
...
Done.