Implement Engine and Runnable abstractions.#454
Merged
jmchilton merged 6 commits intogalaxyproject:masterfrom May 15, 2016
Merged
Implement Engine and Runnable abstractions.#454jmchilton merged 6 commits intogalaxyproject:masterfrom
Engine and Runnable abstractions.#454jmchilton merged 6 commits intogalaxyproject:masterfrom
Conversation
5606bbb to
461d2ce
Compare
5a4baee to
2433f6d
Compare
557ccdc to
6acb923
Compare
4a7e9c9 to
c24e088
Compare
83fc78f to
e88db78
Compare
19fc6a4 to
23d5208
Compare
fe92e34 to
0c7ffd0
Compare
This switches planemo in a subtle way from to depending more on the sturctured data json than the XUnit - this will be benefical for non-Galaxy tool testing when XUnit will be a product of the reporting stuff and not an input to it. This refactoring/reworking should allow most of planemo/galaxy/test/actions to be used with the newer planemo-driven testing stuff. Adds tests and doc strings.
043d14e to
526a484
Compare
``Runnable``s are things such as Galaxy tools, Galaxy workflows, CWL tools, CWL workflows and ``Engine``s provide operations over ``Runnable``s. There are two Engines right now - cwltool and Galaxy and one command that uses the Engine abstraction (``run`` - formerly ``cwl_run``). Exit code handling is improved to so fixed exit codes result when unknown runnables are encountered or when a runnable is matched with an incompatible engine. A dependency on ``aenum`` is added to get enable nice enum functionality in planemo to describe runnable types. The ``Engine`` abstraction implements running Galaxy and CWL tools so far (with either Galaxy or cwltool) but this commit contain a lot of infrastructure for running Galaxy tools and workflows of either type. In particular, workflows will now be uploaded as needed and IDs tracked, they can be executed, and what remains is to collect outputs. The generalized run command implements galaxyproject#449 and galaxyproject#364 - at least in some initial fashion. Initial infrastructure for a generic test command across engine and runnable types is also present (xref galaxyproject#448). If ``planemo test`` encounters a runnable type other than galaxy_tool and directory - such as a cwl tool, a workflow, etc... it will use a new planemo engine-driven testing procedure. To do this an engine "job" is executed, the results collected, and the results "checked". Currently, Galaxy workflows do not have their results collected and the only "check" available is exact non-File matches and sha1 checksums as used by ``cwltest.py``. A complete suite of checks matching what is available in Galaxy will hopefully be implemented shortly. This new engine-driven testing produces a structured test data structure that should match what is generated by Galaxy regardless of which engine is used. The upshot of this is that existing test report infrastructure in planemo should be reusable. ``planemo test_report`` and ``planemo share_test`` should work for any ``Engine`` and any ``RunnableType``. Despite not yet using ``Engine``s infrastructure, the ``serve`` command and related infrastructure for Galaxy has been updated to consume runnables instead of paths. An upshot of this is that serve has been updated to allow serving workflows (either ``.ga`` files directly or Format 2 Workflows through the new gxformat2 library). In order to serve, run, and test gxformat2 workflows - planemo now depends on the gxformat2 library (https://github.com/jmchilton/gxformat2). The depenedency on bioblend has been constrained to be newer so that workflow invocation client stuff can be used. The commit also contains general improvements for CWL support - in particular the cwl-runner implementation driven by planemo is much enhanced and now handles (in at least a minimal fashion) all arguments required to be used by the conformance test code. This commit also introduces a test script ``scripts/test_confromance_cwl.bash`` that runs the common workflow language conformance tests against Planemo/Galaxy. Fixes galaxyproject#453. This commit also includes a variety of improvements to ``run`` and ``serve`` tests. - Refactor serve test cases to allow more reuse. - Fix profile test to allow rerunning on the same machine more than once. - Docstrings.
- Start work on workflow SDK slides for GCC/BOSC. - Rebuild slides as part of ready-docs Make target.
430ff44 to
8901104
Compare
Aimed at improving and fixing engine related test cases.
Engine and Runnable abstractions.
Contributor
|
Congratulations! I look forward to trying this out tomorrow :-) |
jmchilton
added a commit
that referenced
this pull request
May 15, 2016
This was referenced May 15, 2016
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runnables are things such as Galaxy tools, Galaxy workflows, CWL tools, CWL workflows andEngines provide operations overRunnables. There are two Engines right now - cwltool and Galaxy and two command that uses the Engine abstraction (testandrun- formerlycwl_run).Exit code handling is improved to so fixed exit codes result when unknown runnables are encountered or when a runnable is matched with an incompatible engine. A dependency on
aenumis added to get enable nice enum functionality in planemo to describe runnable types.The
Engineabstraction implements running Galaxy and CWL tools so far (with either Galaxy or cwltool) but this commit contain a lot of infrastructure for running Galaxy tools and workflows of either type. In particular, workflows will now be uploaded as needed and IDs tracked, they can be executed, and what remains is to collect outputs. The generalized run command implements #449 and #364 - at least in some initial fashion.Initial infrastructure for a generic test command across engine and runnable types is also present (xref #448). If
planemo testencounters a runnable type other than galaxy_tool and directory - such as a cwl tool, a workflow, etc... it will use a new planemo engine-driven testing procedure. To do this an engine "job" is executed, the results collected, and the results "checked". Currently, Galaxy workflows do not have their results collected and the only "check" available is exact non-File matches and sha1 checksums as used bycwltest.py. A complete suite of checks matching what is available in Galaxy will hopefully be implemented shortly.This new engine-driven testing produces a structured test data structure that should match what is generated by Galaxy regardless of which engine is used. The upshot of this is that existing test report infrastructure in planemo should be reusable.
planemo test_reportandplanemo share_testshould work for anyEngineand anyRunnableType.Despite not yet using
Engines infrastructure, theservecommand and related infrastructure for Galaxy has been updated to consume runnables instead of paths. An upshot of this is that serve has been updated to allow serving workflows (either.gafiles directly or Format 2 Workflows through the new gxformat2 library).In order to serve, run, and test gxformat2 workflows - planemo now depends on the gxformat2 library (https://github.com/jmchilton/gxformat2). The depenedency on bioblend has been constrained to be newer so that workflow invocation client stuff can be used.
The commit also contains general improvements for CWL support - in particular the cwl-runner implementation driven by planemo is much enhanced and now handles (in at least a minimal fashion) all arguments required to be used by the conformance test code. This commit also introduces a test script
scripts/test_confromance_cwl.bashthat runs the common workflow language conformance tests against Planemo/Galaxy. Fixes #453.This commit also includes a variety of improvements to
runandservetests.