Skip to content

Make simultaneous file upload configurable for planemo run / test#1156

Merged
jmchilton merged 1 commit intogalaxyproject:masterfrom
simonbray:no-upload-wait
Mar 29, 2021
Merged

Make simultaneous file upload configurable for planemo run / test#1156
jmchilton merged 1 commit intogalaxyproject:masterfrom
simonbray:no-upload-wait

Conversation

@simonbray
Copy link
Member

So that if we are running a tool or workflow with multiple input datasets that need to be uploaded, we can choose to have all uploads started at once, rather than waiting for each to complete before the next one starts.

In my opinion this simultaneous uploading could even be the default (it makes more sense for running against an external galaxy at least) but it seems it was considered a bad idea, as planemo explicitly overrides the default behaviour of the Galaxy StagingInterface here: https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/tool_util/client/staging.py#L57. Do you remember why you thought this was necessary @jmchilton?

@simonbray simonbray requested a review from jmchilton March 29, 2021 16:57
@jmchilton
Copy link
Member

It was off by default because Planemo runs against an sqlite database by default that used to fail frequently. I think things have gotten more stable and this definitely doesn't need to be one at a time against a production, external Galaxy. Seems like a good enhancement.

@jmchilton jmchilton merged commit e4ac4b4 into galaxyproject:master Mar 29, 2021
@bernt-matthias
Copy link
Collaborator

Hey @simonbray .. I just discovered this option and wonder how to make use of it. It appears to me that it does not make a difference in a standard galaxy test tool.xml call?

@simonbray simonbray deleted the no-upload-wait branch September 24, 2021 09:02
@simonbray
Copy link
Member Author

Hey @simonbray .. I just discovered this option and wonder how to make use of it. It appears to me that it does not make a difference in a standard galaxy test tool.xml call?

planemo test runs tool tests by default through Galaxy's ./run_tests.sh script afaik, so it would not touch this code at all. If you have a local Galaxy server running separately and run planemo test against it, hopefully you should see the difference.

I'm not sure if this option is useful for standard tool testing though? It was definitely meant more for external galaxies, see the discussion above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants