Skip to content

Integrate some features from gxwf #1076

Merged
jmchilton merged 24 commits intogalaxyproject:masterfrom
simonbray:gxwf-integration
Nov 23, 2020
Merged

Integrate some features from gxwf #1076
jmchilton merged 24 commits intogalaxyproject:masterfrom
simonbray:gxwf-integration

Conversation

@simonbray
Copy link
Member

@simonbray simonbray commented Sep 4, 2020

Start working on integrating some features from gxwf (see https://github.com/simonbray/gxwf):

  • some work developing the profile commands, in particular to allow creation of profiles for external galaxies
  • allow execution of workflows on an external galaxy instance with only a workflow ID
  • allow creation of aliases which can be used instead of workflow (eventually also dataset) IDs
  • a list_invocations command
  • various other small fixes for things I found along the way

with engine_context(ctx, **kwds) as galaxy_engine:
with galaxy_engine.ensure_runnables_served([runnable]) as config:
workflow_id = config.workflow_id(workflow_path)
workflow_id = runnable.workflow_id or config.workflow_id(runnable.path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool!

@jmchilton
Copy link
Member

jmchilton commented Oct 20, 2020

This is awesome - I want to get this merged. I need it for a project 😆. What can I do to help?

There is a lot of really solid little fixes and refactorings in here - if I can try to preserve your commit authorship do you mind if I rebase some smaller pieces out an merge them ahead of the big PR?

}


def _create_profile_external(ctx, profile_directory, profile_name, kwds):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so cool!

@simonbray
Copy link
Member Author

This is awesome - I want to get this merged. I need it for a project laughing. What can I do to help?

There is a lot of really solid little fixes and refactorings in here - if I can try to preserve your commit authorship do you mind if I rebase some smaller pieces out an merge them ahead of the big PR?

Hi @jmchilton, yes, don't feel you need to wait for me.

That said, I think I'm mostly done here. I still need to write some tests and documentation, but at least the latter can be done in another PR.

@simonbray
Copy link
Member Author

Okay, the tests I wrote are passing; failures are the same as #1093 (comment).

@simonbray simonbray changed the title [WIP] Begin gxwf integration Integrate some features from gxwf Oct 21, 2020
@simonbray simonbray requested a review from jmchilton October 21, 2020 16:25
@simonbray
Copy link
Member Author

@jmchilton, I've taken this out of WIP if that's ok with you?

We can also do it the way you suggested and keep this one as WIP if you prefer.

@simonbray
Copy link
Member Author

Hey @jmchilton - sorry to bother you, but you said you were keen to get this merged. Could I get a review here - or is there anything else you'd like me to work on first?

I prefer these abstractions.
@jmchilton
Copy link
Member

What do you think about these abstractions 693da35.

I haven't tested it - but I really prefer this abstraction around the runnable. I think it prevents adding specialized attributes to runnables that don't need it (e.g. workflow_dict) and cleans up operations like fetching the workflow_id from a runnable. If you can make something like that commit work I'll definitely merge this. At all the other levels - I really love everything about this PR.

@simonbray
Copy link
Member Author

Thanks for that commit @jmchilton, I agree that that looks a lot nicer than my runnable.workflow_id solution. It seems to be working fine, after a bit of tinkering.

@jmchilton jmchilton merged commit 1f9fdff into galaxyproject:master Nov 23, 2020
@jmchilton
Copy link
Member

Thanks so much, this is awesome. And thanks for cleaning up that commit.

@simonbray simonbray deleted the gxwf-integration branch November 23, 2020 19:36
@simonbray
Copy link
Member Author

Thanks for the merge!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants