Checklist

• I have checked the issues list
  for similar or identical bug reports.
• I have checked the pull requests list
  for existing proposed fixes.
• I have checked the commit log
  to find out if the bug was already fixed in the main branch.
• I have included all related issues and possible duplicate issues in this issue
  (If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

• we think that In the celery task chain, the next subtask does not wait for the previous on_success to complete #5905 was also raised because of the misunderstanding of the current documentation

Possible Duplicates

• None

Description

From the documentation alone it is unclear at what point in time Task handlers https://docs.celeryq.dev/en/stable/userguide/tasks.html#handlers are executed.

• before_start and after_return have a notion of order by name , Yet, it is unclear if that notion is intended. I.e. does before_start block the task from executing until it finished or is it part of the execution?
• on_success, on_retry and on_failure do not indicate whether they are run as part of task or outside the task

The documentation also is not clear on the order of execution. Especially the temporal relation between the task being recorded as completed in the celery platform/back-end and the execution of the handler callbacks. We assumed tasks would only enter the ready-state when these callbacks also completed. But that is not the case, they are run asynchronously and independently.

Suggestions

We can only make suggestions for on_failure and on_success as these are the ones we used:

...

.. method:: on_failure(self, exc, task_id, args, kwargs, einfo)

    This is an independent process that is triggered after the task failed and returned. It is run by the worker the task was executed on.

...

.. method:: on_success(self, retval, task_id, args, kwargs)

    This is an independent process that is triggered after the task succeeded and returned. It is run by the worker the task was executed on.

...

cc @dgatland