It mostly works. There are just 3 jobs in my workflow that run significantly longer than the others (2181 jobs) and Planemo waits patiently until all but these 3 jobs are finished, and creates the correct XUnit file. However, these 3 jobs are still reported as running in the XUnit file.

https://github.com/galaxyproject/planemo/actions/runs/14729205708/job/41346995815?pr=1518 that looks relevant

https://github.com/galaxyproject/planemo/actions/runs/14729205708/job/41346995815?pr=1518 that looks relevant

Thanks @mvdbeek I think I fixed it in 06f7138, can you please approve the workflow to re-run the tests?

@mvdbeek Can you please approve the workflow run again?

Thank you @kostrykin!

@kostrykin do you think you could provide a small test workflow that exercises the behavior ? I think #1510 is also doing this in a conflicting way, and does in fact effectively make --no_early_termination the default (and only) behavior.

@mvdbeek You mean the behavior that I described here?

I think this workflow should reproduce this: https://usegalaxy.eu/u/kostrykin/w/test-input-images-small-1

Steps to reproduce:

1. Clone the repo https://github.com/FAIR-imaging/galaxy-tool-test-workflow
2. Save https://usegalaxy.eu/u/kostrykin/w/test-input-images-small-1 as test_tools.ga (overwrite) or use the workflow that comes with the repo (takes longer to run).
3. Run python create_test_data.py
4. Set GALAXY_API_KEY and run

planemo run test_tools.ga test_tools-job.yml \
  --engine external_galaxy \
  --galaxy_url https://usegalaxy.eu \
  --galaxy_user_key $GALAXY_API_KEY \
  --test_output_xunit tests.xunit \
  --simultaneous_uploads \
  --no_early_termination

Yes, thank you!