Skip to content

Port task.test to Task SDK#50827

Merged
kaxil merged 1 commit intoapache:mainfrom
astronomer:move-task-test
May 20, 2025
Merged

Port task.test to Task SDK#50827
kaxil merged 1 commit intoapache:mainfrom
astronomer:move-task-test

Conversation

@kaxil
Copy link
Member

@kaxil kaxil commented May 20, 2025

Follow-up of #50300 for running a single task.

Post this, the next is to modify / remove ti.run usage in tests

Dag used to test:

from airflow.sdk import Variable, dag, task

@dag
def ample_simplest_dag():

    @task
    def my_task():
        print("hellooo")
        import os
        os.environ["AIRFLOW_VAR_MY_VARIABLE"] = "my_value"
        assert Variable.get("my_variable") == "my_value"

    my_task()


d = ample_simplest_dag()

and then ran:

root@75043446567f:/opt/airflow# airflow tasks test ample_simplest_dag my_task

Logs:

root@75043446567f:/opt/airflow# airflow tasks test ample_simplest_dag my_task
[2025-05-20T18:00:14.130+0000] {manager.py:147} INFO - DAG bundles loaded: dags-folder, example_dags, example_standard_dags
[2025-05-20T18:00:14.996+0000] {dagbag.py:573} INFO - Filling up the DagBag from /files/dags
[2025-05-20T18:00:16.315+0000] {dag.py:2321} INFO - created dagrun <DagRun ample_simplest_dag @ None: __airflow_temporary_run_2025-05-20T18:00:16.238529+00:00__, state:running, queued_at: None. run_type: manual>
[2025-05-20T18:00:16.336+0000] {dag.py:1221} INFO - [DAG TEST] starting task_id=my_task map_index=-1
[2025-05-20T18:00:16.336+0000] {dag.py:1224} INFO - [DAG TEST] running task <TaskInstance: ample_simplest_dag.my_task __airflow_temporary_run_2025-05-20T18:00:16.238529+00:00__ [None]>
2025-05-20 18:00:17 [debug    ] Starting task instance run     hostname=75043446567f pid=147 ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4 unixname=root
2025-05-20 18:00:17 [debug    ] Retrieved task instance details dag_id=ample_simplest_dag state=queued task_id=my_task ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
2025-05-20 18:00:17 [info     ] Task started                   hostname=75043446567f previous_state=queued ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
2025-05-20 18:00:17 [info     ] Task instance state updated    rows_affected=1 ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
[2025-05-20T18:00:17.055+0000] {_client.py:1026} INFO - HTTP Request: PATCH http://in-process.invalid./task-instances/0196eeda-74bb-7bd0-8013-23f704571fd4/run "HTTP/1.1 200 OK"
2025-05-20 18:00:17 [debug    ] Sending request                [task] msg=SetRenderedFields(rendered_fields={'templates_dict': None, 'op_args': [], 'op_kwargs': {}}, type='SetRenderedFields')
2025-05-20 18:00:17 [debug    ] Received message from task runner [task] msg=SetRenderedFields(rendered_fields={'templates_dict': None, 'op_args': [], 'op_kwargs': {}}, type='SetRenderedFields')
2025-05-20 18:00:17 [info     ] Updating RenderedTaskInstanceFields field_count=3 ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
2025-05-20 18:00:17 [debug    ] RenderedTaskInstanceFields updated successfully ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
[2025-05-20T18:00:17.082+0000] {_client.py:1026} INFO - HTTP Request: PUT http://in-process.invalid./task-instances/0196eeda-74bb-7bd0-8013-23f704571fd4/rtif "HTTP/1.1 201 Created"
Task instance is in running state
 Previous state of the Task instance: TaskInstanceState.QUEUED
Current task name:my_task
Dag name:ample_simplest_dag
hellooo
[2025-05-20T18:00:17.086+0000] {python.py:219} INFO - Done. Returned value was: None
2025-05-20 18:00:17 [debug    ] Sending request                [task] msg=SucceedTask(state='success', end_date=datetime.datetime(2025, 5, 20, 18, 0, 17, 86745, tzinfo=datetime.timezone.utc), task_outlets=[], outlet_events=[], rendered_map_index=None, type='SucceedTask')
2025-05-20 18:00:17 [debug    ] Received message from task runner [task] msg=SucceedTask(state='success', end_date=datetime.datetime(2025, 5, 20, 18, 0, 17, 86745, tzinfo=datetime.timezone.utc), task_outlets=[], outlet_events=[], rendered_map_index=None, type='SucceedTask')
2025-05-20 18:00:17 [debug    ] Updating task instance state   new_state=success ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
2025-05-20 18:00:17 [debug    ] Retrieved current task instance state max_tries=0 previous_state=running ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4 try_number=0
2025-05-20 18:00:17 [info     ] Task instance state updated    new_state=success rows_affected=1 ti_id=0196eeda-74bb-7bd0-8013-23f704571fd4
[2025-05-20T18:00:17.106+0000] {_client.py:1026} INFO - HTTP Request: PATCH http://in-process.invalid./task-instances/0196eeda-74bb-7bd0-8013-23f704571fd4/state "HTTP/1.1 204 No Content"
2025-05-20 18:00:17 [debug    ] Running finalizers             [task] ti=RuntimeTaskInstance(id=UUID('0196eeda-74bb-7bd0-8013-23f704571fd4'), task_id='my_task', dag_id='ample_simplest_dag', run_id='__airflow_temporary_run_2025-05-20T18:00:16.238529+00:00__', try_number=0, map_index=-1, hostname='75043446567f', context_carrier=None, task=<Task(_PythonDecoratedOperator): my_task>, max_tries=0, start_date=datetime.datetime(2025, 5, 20, 18, 0, 16, 632534, tzinfo=datetime.timezone.utc), end_date=None, state=<TaskInstanceState.SUCCESS: 'success'>, is_mapped=False, rendered_map_index=None, log_url=None)
Task instance in success state
 Previous state of the Task instance: TaskInstanceState.RUNNING
Task operator:<Task(_PythonDecoratedOperator): my_task>
[2025-05-20T18:00:17.118+0000] {dag.py:1270} INFO - [DAG TEST] end task task_id=my_task map_index=-1
image

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@kaxil kaxil marked this pull request as ready for review May 20, 2025 15:13
@kaxil kaxil requested review from amoghrajesh and ashb as code owners May 20, 2025 15:13
Follow-up of apache#50300 for running a single task.
@kaxil kaxil force-pushed the move-task-test branch from 71c2401 to bc2c9be Compare May 20, 2025 18:11
@kaxil kaxil merged commit 3a9858e into apache:main May 20, 2025
68 of 69 checks passed
@kaxil kaxil deleted the move-task-test branch May 20, 2025 19:41
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really like how simple this makes it!

nailo2c pushed a commit to nailo2c/airflow that referenced this pull request May 21, 2025
Follow-up of apache#50300 for running a single task.
kaxil added a commit that referenced this pull request May 22, 2025
Follow-up of #50300 for running a single task.

(cherry picked from commit 3a9858e)
dadonnelly316 pushed a commit to dadonnelly316/airflow that referenced this pull request May 26, 2025
Follow-up of apache#50300 for running a single task.
kaxil added a commit that referenced this pull request Jun 3, 2025
Follow-up of #50300 for running a single task.

(cherry picked from commit 3a9858e)
sanederchik pushed a commit to sanederchik/airflow that referenced this pull request Jun 7, 2025
Follow-up of apache#50300 for running a single task.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

3 participants