Skip to content

Invoke workflow complete handler on all workflow errors#4299

Merged
bentsherman merged 2 commits intomasterfrom
3129-fix-workflow-complete-handler
Jul 30, 2025
Merged

Invoke workflow complete handler on all workflow errors#4299
bentsherman merged 2 commits intomasterfrom
3129-fix-workflow-complete-handler

Conversation

@bentsherman
Copy link
Member

@bentsherman bentsherman commented Sep 12, 2023

Close #3129

Call workflow complete handler on all workflow errors, including session abort() (as well as other shutdown callbacks and observers).

@netlify
Copy link

netlify bot commented Sep 12, 2023

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit 282230c
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/6889b5ed53e3990008d7bae8

@bentsherman
Copy link
Member Author

But if we don't make these changes, we should document them as limitations instead.

@pditommaso
Copy link
Member

No chance to add some unit tests?

@bentsherman
Copy link
Member Author

When I'm not sure about a PR, I like to get your overall opinion before I write unit tests

@bentsherman
Copy link
Member Author

Also, I'm not sure that unit tests would help here. Probably better to add an integration test based on the reproducible examples.

u6756 added a commit to u6756/nextflow that referenced this pull request Nov 14, 2024
@pditommaso pditommaso force-pushed the master branch 2 times, most recently from 5a93547 to 27345a6 Compare February 10, 2025 21:46
@pditommaso pditommaso force-pushed the master branch 3 times, most recently from b4b321e to 069653d Compare June 4, 2025 18:54
@bentsherman bentsherman changed the title Fix issues with workflow complete handler Invoke workflow complete handler on all workflow errors Jun 27, 2025
@bentsherman bentsherman force-pushed the 3129-fix-workflow-complete-handler branch from f5f413a to bfd5174 Compare June 27, 2025 17:32
@bentsherman
Copy link
Member Author

@pditommaso I have trimmed this PR to focus on calling the onComplete on all error conditions, including the error function. Also added an e2e test

Notes:

  • This PR calls all shutdown callbacks on abort, which includes the onComplete and the task monitor cleanup. This seems like a good thing? In fact I thought it was already doing this but I don't see any logic for it

  • The onComplete is also allowed to call the error function, which could lead to the onComplete being called multiple times. It shouldn't cause an infinite recursion thanks to the existing error-handling logic in Session::abort() and other places. I think it's still fine to call error() in the onComplete, and as long as you do it in a sensible way, you will get sensible error messages

@bentsherman bentsherman added this to the 25.10 milestone Jul 14, 2025
@bentsherman bentsherman force-pushed the 3129-fix-workflow-complete-handler branch from bfd5174 to 069d134 Compare July 28, 2025 15:16
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman bentsherman force-pushed the 3129-fix-workflow-complete-handler branch from 069d134 to da5acd8 Compare July 28, 2025 15:17
@bentsherman bentsherman requested review from jorgee and removed request for pditommaso July 28, 2025 15:18
@bentsherman bentsherman merged commit 9d24615 into master Jul 30, 2025
22 checks passed
@bentsherman bentsherman deleted the 3129-fix-workflow-complete-handler branch July 30, 2025 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unpredictable behaviour for workflow.onComplete handler

3 participants