Skip to content

Implement Engine and Runnable abstractions.#454

Merged
jmchilton merged 6 commits intogalaxyproject:masterfrom
jmchilton:engine
May 15, 2016
Merged

Implement Engine and Runnable abstractions.#454
jmchilton merged 6 commits intogalaxyproject:masterfrom
jmchilton:engine

Conversation

@jmchilton
Copy link
Member

@jmchilton jmchilton commented Apr 15, 2016

Runnables are things such as Galaxy tools, Galaxy workflows, CWL tools, CWL workflows and Engines provide operations over Runnables. There are two Engines right now - cwltool and Galaxy and two command that uses the Engine abstraction (test and run - formerly cwl_run).

Exit code handling is improved to so fixed exit codes result when unknown runnables are encountered or when a runnable is matched with an incompatible engine. A dependency on aenum is added to get enable nice enum functionality in planemo to describe runnable types.

The Engine abstraction implements running Galaxy and CWL tools so far (with either Galaxy or cwltool) but this commit contain a lot of infrastructure for running Galaxy tools and workflows of either type. In particular, workflows will now be uploaded as needed and IDs tracked, they can be executed, and what remains is to collect outputs. The generalized run command implements #449 and #364 - at least in some initial fashion.

Initial infrastructure for a generic test command across engine and runnable types is also present (xref #448). If planemo test encounters a runnable type other than galaxy_tool and directory - such as a cwl tool, a workflow, etc... it will use a new planemo engine-driven testing procedure. To do this an engine "job" is executed, the results collected, and the results "checked". Currently, Galaxy workflows do not have their results collected and the only "check" available is exact non-File matches and sha1 checksums as used by cwltest.py. A complete suite of checks matching what is available in Galaxy will hopefully be implemented shortly.

This new engine-driven testing produces a structured test data structure that should match what is generated by Galaxy regardless of which engine is used. The upshot of this is that existing test report infrastructure in planemo should be reusable. planemo test_report and planemo share_test should work for any Engine and any RunnableType.

Despite not yet using Engines infrastructure, the serve command and related infrastructure for Galaxy has been updated to consume runnables instead of paths. An upshot of this is that serve has been updated to allow serving workflows (either .ga files directly or Format 2 Workflows through the new gxformat2 library).

In order to serve, run, and test gxformat2 workflows - planemo now depends on the gxformat2 library (https://github.com/jmchilton/gxformat2). The depenedency on bioblend has been constrained to be newer so that workflow invocation client stuff can be used.

The commit also contains general improvements for CWL support - in particular the cwl-runner implementation driven by planemo is much enhanced and now handles (in at least a minimal fashion) all arguments required to be used by the conformance test code. This commit also introduces a test script scripts/test_confromance_cwl.bash that runs the common workflow language conformance tests against Planemo/Galaxy. Fixes #453.

This commit also includes a variety of improvements to run and serve tests.

  • Refactor serve test cases to allow more reuse.
  • Fix profile test to allow rerunning on the same machine more than once.
  • Docstrings.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.1%) to 79.37% when pulling 9b87cc0 on jmchilton:engine into 47bb477 on galaxyproject:master.

@jmchilton jmchilton force-pushed the engine branch 5 times, most recently from 5606bbb to 461d2ce Compare April 22, 2016 19:19
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.09%) to 79.379% when pulling 461d2ce on jmchilton:engine into 069e7ba on galaxyproject:master.

@jmchilton jmchilton force-pushed the engine branch 2 times, most recently from 5a4baee to 2433f6d Compare April 25, 2016 18:32
@coveralls
Copy link

Coverage Status

Coverage increased (+1.6%) to 81.092% when pulling 2433f6d on jmchilton:engine into 473e12d on galaxyproject:master.

@jmchilton jmchilton force-pushed the engine branch 2 times, most recently from 557ccdc to 6acb923 Compare April 25, 2016 23:36
@coveralls
Copy link

Coverage Status

Coverage increased (+1.9%) to 81.263% when pulling 6acb923 on jmchilton:engine into 4035d37 on galaxyproject:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+2.0%) to 81.343% when pulling 672893f on jmchilton:engine into 4035d37 on galaxyproject:master.

@jmchilton jmchilton force-pushed the engine branch 3 times, most recently from 4a7e9c9 to c24e088 Compare May 1, 2016 20:03
This was referenced May 1, 2016
@jmchilton jmchilton force-pushed the engine branch 7 times, most recently from 83fc78f to e88db78 Compare May 8, 2016 14:13
@jmchilton jmchilton force-pushed the engine branch 3 times, most recently from 19fc6a4 to 23d5208 Compare May 10, 2016 16:11
@jmchilton jmchilton force-pushed the engine branch 3 times, most recently from fe92e34 to 0c7ffd0 Compare May 12, 2016 02:29
This switches planemo in a subtle way from to depending more on the sturctured data json than the XUnit - this will be benefical for non-Galaxy tool testing when XUnit will be a product of the reporting stuff and not an input to it. This refactoring/reworking should allow most of planemo/galaxy/test/actions to be used with the newer planemo-driven testing stuff.

Adds tests and doc strings.
@jmchilton jmchilton force-pushed the engine branch 4 times, most recently from 043d14e to 526a484 Compare May 12, 2016 20:45
jmchilton added 2 commits May 12, 2016 21:48
``Runnable``s are things such as  Galaxy tools, Galaxy workflows, CWL tools, CWL workflows and ``Engine``s provide operations over ``Runnable``s. There are two Engines right now - cwltool and Galaxy and one command that uses the Engine abstraction (``run`` - formerly ``cwl_run``).

Exit code handling is improved to so fixed exit codes result when unknown runnables are encountered or when a runnable is matched with an incompatible engine. A dependency on ``aenum`` is added to get enable nice enum functionality in planemo to describe runnable types.

The ``Engine`` abstraction implements running Galaxy and CWL tools so far (with either Galaxy or cwltool) but this commit contain a lot of infrastructure for running Galaxy tools and workflows of either type. In particular, workflows will now be uploaded as needed and IDs tracked, they can be executed, and what remains is to collect outputs. The generalized run command implements galaxyproject#449 and galaxyproject#364 - at least in some initial fashion.

Initial infrastructure for a generic test command across engine and runnable types is also present (xref galaxyproject#448). If ``planemo test`` encounters a runnable type other than galaxy_tool and directory - such as a cwl tool, a workflow, etc... it will use a new planemo engine-driven testing procedure. To do this an engine "job" is executed, the results collected, and the results "checked". Currently, Galaxy workflows do not have their results collected and the only "check" available is exact non-File matches and sha1 checksums as used by ``cwltest.py``. A complete suite of checks matching what is available in Galaxy will hopefully be implemented shortly.

This new engine-driven testing produces a structured test data structure that should match what is generated by Galaxy regardless of which engine is used. The upshot of this is that existing test report infrastructure in planemo should be reusable. ``planemo test_report`` and ``planemo share_test`` should work for any ``Engine`` and any ``RunnableType``.

Despite not yet using ``Engine``s infrastructure, the ``serve`` command and related infrastructure for Galaxy has been updated to consume runnables instead of paths. An upshot of this is that serve has been updated to allow serving workflows (either ``.ga`` files directly or Format 2 Workflows through the new gxformat2 library).

In order to serve, run, and test gxformat2 workflows - planemo now depends on the gxformat2 library (https://github.com/jmchilton/gxformat2). The depenedency on bioblend has been constrained to be newer so that workflow invocation client stuff can be used.

The commit also contains general improvements for CWL support - in particular the cwl-runner implementation driven by planemo is much enhanced and now handles (in at least a minimal fashion) all arguments required to be used by the conformance test code. This commit also introduces a test script ``scripts/test_confromance_cwl.bash`` that runs the common workflow language conformance tests against Planemo/Galaxy. Fixes galaxyproject#453.

This commit also includes a variety of improvements to ``run`` and ``serve`` tests.

 - Refactor serve test cases to allow more reuse.
 - Fix profile test to allow rerunning on the same machine more than once.
 - Docstrings.
 - Start work on workflow SDK slides for GCC/BOSC.
 - Rebuild slides as part of ready-docs Make target.
@jmchilton jmchilton force-pushed the engine branch 2 times, most recently from 430ff44 to 8901104 Compare May 15, 2016 15:44
@jmchilton jmchilton changed the title [WIP] Refactor toward abstract description of an "Engine" Implement Engine and Runnable abstractions. May 15, 2016
@jmchilton jmchilton merged commit 667e404 into galaxyproject:master May 15, 2016
@mr-c
Copy link
Contributor

mr-c commented May 15, 2016

Congratulations! I look forward to trying this out tomorrow :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CWL CI for Galaxy

3 participants