Apache Airflow version
3.0.6
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When using custom classes that subclass pathlib.PosixPath, Airflow 3's new SDK templating system calls .resolve(context) on these objects when rendering templates. If the subclass doesn't override resolve to accept a context argument, the method inherited from PosixPath is called, and the context (a dict) is interpreted as 'strict=True', causing Path.resolve(strict=True) and triggering FileNotFoundError if the file doesn't exist. This is a regression from Airflow 2.x and breaks compatibility with custom path-like objects in op_kwargs and other templated fields.
Example error:
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/some/file.txt'
File "/home/airflow/.local/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/templater.py", line 192 in render_template
File "/home/airflow/.local/lib/python3.11/site-packages/custom_file_lib/path.py", line 140 in resolve
File "/usr/local/lib/python3.11/pathlib.py", line 993 in resolve
This behavior can break any custom library that extends pathlib.Path and doesn't override resolve(context).
What you think should happen instead?
The templater should not call .resolve(context) on objects that inherit from pathlib.Path, or should handle cases where resolve is not designed for templating. Ideally, the templater should distinguish between 'templating resolution' and 'filesystem resolution'. Or, provide a clear interface so custom classes can be compatible without breaking existing path-like subclasses.
How to reproduce
- Create a custom class that subclasses pathlib.PosixPath and does not override resolve().
- Pass an instance of this class as a templated field/op_kwargs in Airflow 3.x SDK DAG.
- Trigger task rendering (e.g. via BashOperator or similar).
- Observe FileNotFoundError during template rendering if the file does not exist.
Operating System
Linux (Debian-based container)
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else?
Happens every time a custom Path subclass is passed to a templated field and the file does not exist yet.
Are you willing to submit PR?
Code of Conduct
Apache Airflow version
3.0.6
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When using custom classes that subclass pathlib.PosixPath, Airflow 3's new SDK templating system calls .resolve(context) on these objects when rendering templates. If the subclass doesn't override resolve to accept a context argument, the method inherited from PosixPath is called, and the context (a dict) is interpreted as 'strict=True', causing Path.resolve(strict=True) and triggering FileNotFoundError if the file doesn't exist. This is a regression from Airflow 2.x and breaks compatibility with custom path-like objects in op_kwargs and other templated fields.
Example error:
This behavior can break any custom library that extends pathlib.Path and doesn't override resolve(context).
What you think should happen instead?
The templater should not call .resolve(context) on objects that inherit from pathlib.Path, or should handle cases where resolve is not designed for templating. Ideally, the templater should distinguish between 'templating resolution' and 'filesystem resolution'. Or, provide a clear interface so custom classes can be compatible without breaking existing path-like subclasses.
How to reproduce
Operating System
Linux (Debian-based container)
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else?
Happens every time a custom Path subclass is passed to a templated field and the file does not exist yet.
Are you willing to submit PR?
Code of Conduct