Summary

As we prepare for the release of Airflow 3.0, one of the key areas that need significant enhancement is the debugging experience.

Current Challenges

• Insufficient Logging: Logs are often fragmented, in some cases overly verbose or non-existent and lack sufficient detail to easily trace issues. We should do an audit of the existing logs.
• Complex Tracebacks: Debugging stack traces can be difficult due to the complex nature of DAG (Directed Acyclic Graph) execution and requires a full-running Airflow. Airflow's dag.test and task.test does a good job already but we should see if we can do even better.
• Error Handling: Current error messages are not always informative or actionable, making it hard to understand the root cause of failures. We should do an audit of the existing errors.
• Tooling Integration: Lack of integration with modern debugging and observability tools hinders the debugging process. Can we create a listing tool or some capabilities in the Airflow CLI that catches obvious errors? airflow dags parse does a job at it, worth checking if it is sufficient or not.

Whoever takes on this task should conduct a user research on the mailing list, Slack, Meetup or Airflow Summit to identify other common debugging problems that can be fixed.