Rename execution_date to logical_date across codebase by sunank200 · Pull Request #43902 · apache/airflow

sunank200 · 2024-11-11T23:47:17Z

Motivation

This PR renames execution_date to logical_date across the codebase. The shift towards logical_date helps move away from the limitations of execution_date, particularly with dynamic DAG runs and cases where multiple runs occur at the same time.

Key Changes

Replaced all instances of execution_date with logical_date including models.
Updated references in database models, templates, and functions.
Removed unique constraints on execution_date in the database to allow multiple DAG runs with the same logical time.

How `execution_date` and `logical_date` Work

Logical date is equivalent to execution date: The two are just different names for the same value.
Timetable controls logical date: The logical date can be set to any value, not necessarily tied to the data interval's start or end.
Schedules dictate behavior: For value-based schedules (like cron), the logical date is set by the timetable class used.

Transitioning from `execution_date`

Airflow 3.0 will fully remove execution_date.
For uniquely identifying runs:
- Use run_id for human-readable identifiers.
- Use data_interval_start/data_interval_end for data partitioning in future.
- id (auto-incremented) is recommended for ordering DAG runs.
If your existing logic relies on execution_date, switch to using data_interval_start or data_interval_end for identifying time ranges.

Testing

Updated unit tests to reflect the changes.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

uranusjr · 2024-11-12T04:39:29Z

This needs a news fragment. Rewording the above summary would work fine. See existing files in newsfragments (the significant ones) for some examples.

uranusjr · 2024-11-12T04:40:43Z

For templates, replace {{ ds }} with {{ data_interval_start | ds }}.

~~This should probably use logical_date for max compatibility. Also should mention ts (same rewrite).~~

Oh wait, we’re not removing those in this PR yet, so this should not be mentioned at all.

airflow/api_connexion/openapi/v1.yaml

airflow/api_connexion/schemas/dag_run_schema.py

airflow/cli/cli_config.py

airflow/models/taskreschedule.py

airflow/models/xcom.py

airflow/utils/context.py

airflow/models/taskinstance.py

airflow/utils/context.pyi

docs/apache-airflow/templates-ref.rst

providers/src/airflow/providers/cncf/kubernetes/cli/kubernetes_command.py

Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>

We removed all the deprecated keys in apache#43902 so we no longer need this code. In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.

We removed all the deprecated keys in #43902 so we no longer need this code. In preparation of #45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.

We removed all the deprecated keys in apache#43902 so we no longer need this code. In preparation of apache#45583, I want to simplify this code. We can always revert/re-add this later when we need to deprecate a key.

boring-cyborg bot added area:API Airflow's REST/HTTP API area:CLI area:Scheduler including HA (high availability) scheduler provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels Nov 11, 2024

sunank200 mentioned this pull request Nov 11, 2024

Remove logical_date from DAG Run APIs and Functions, transition to run_id as sole identifier for Airflow 3.0 #42404

Merged

3 tasks

sunank200 added legacy api Whether legacy API changes should be allowed in PR legacy ui Whether legacy UI change should be allowed in PR labels Nov 11, 2024

sunank200 force-pushed the rename-execution-date-to-logical-date branch 9 times, most recently from ea36b6c to 65f9b8e Compare November 12, 2024 02:54

sunank200 requested review from Lee-W and phanikumv November 12, 2024 02:54