diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index d97fedfb49be..4ef53e18e99f 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -48,6 +48,7 @@ The following individuals have contributed code to Galaxy: * Eric Enns * fescudie * Dorine Francheteau +* Jean-Frédéric (@JeanFred on Github) * Jaime Frey * Carrie Ganote * Ryan Golhar @@ -77,6 +78,7 @@ The following individuals have contributed code to Galaxy: * Simone Leo * Kanwei Li * Michael Li +* Pierre Lindenbaum * Mikael Loaec * Philip Mabon * Remi Marenco diff --git a/lib/galaxy/tools/xsd/LICENSE b/lib/galaxy/tools/xsd/LICENSE new file mode 100644 index 000000000000..09eb8b9bca28 --- /dev/null +++ b/lib/galaxy/tools/xsd/LICENSE @@ -0,0 +1,22 @@ +The MIT License (MIT) + +Copyright (c) 2011-2016 John Chilton, Jean-Frédéric Berthelot, Pierre Lindenbaum + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + diff --git a/lib/galaxy/tools/xsd/README.md b/lib/galaxy/tools/xsd/README.md new file mode 100644 index 000000000000..60274d37b3b2 --- /dev/null +++ b/lib/galaxy/tools/xsd/README.md @@ -0,0 +1,45 @@ +Galaxy-XSD +========== +[![Build Status](https://travis-ci.org/JeanFred/Galaxy-XSD.svg)](http://travis-ci.org/JeanFred/Galaxy-XSD) +[![License](http://img.shields.io/badge/license-MIT-orange.svg?style=flat)](http://opensource.org/licenses/MIT) + +A Galaxy XML tool wrapper __XML schema definition__ (__XSD__) + + + +# History + +* Feb-2015 : Pierre Lindenbaum added doc, tests, Java-XML binding file (jxb) for java xml compiler (xjc) ( https://docs.oracle.com/cd/E19575-01/819-3669/bnbal/index.html ) +* 2013 : extraction to standalone and improvements by Jean-Fred +* 2011 : Initial work by John Chilton + +# Validating a `tool.xml` + +```bash +$ xmllint --noout --schema galaxy.xsd tool.xml +``` + +# Creating java code + +```bash +$ ${JAVA_HOME}/bin/xjc -b galaxy.jxb galaxy.xsd +``` + + +# Authors + +* Jean-Frédéric @JeanFred +* Pierre Lindenbaum @yokofakun +* John Chilton @jmchilton + + +# Licence + +This code is free software released under the terms of the MIT license. + + +# See also: + +* Galaxy https://usegalaxy.org/ +* Galaxy Tool XML File https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax + diff --git a/lib/galaxy/tools/xsd/galaxy.jxb b/lib/galaxy/tools/xsd/galaxy.jxb new file mode 100644 index 000000000000..29eac0bfabc6 --- /dev/null +++ b/lib/galaxy/tools/xsd/galaxy.jxb @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + + + diff --git a/lib/galaxy/tools/xsd/galaxy.xsd b/lib/galaxy/tools/xsd/galaxy.xsd new file mode 100644 index 000000000000..a6bdcdba6ab8 --- /dev/null +++ b/lib/galaxy/tools/xsd/galaxy.xsd @@ -0,0 +1,5006 @@ + + + + Galaxy Schema + A Galaxy XML tool wrapper + + + + + +``` + +A ``data_source`` tool contains a few more relevant attributes. + +```xml + + + + + + + + + + `` tag set described above). + +### Example + +```xml +table browser +``` +]]> + + + + + + tophat -version +``` +]]> + + + + + + + + + + + + + + + +.. class:: warningmark + +'''TIP''' This tool requires *fasta* format. + +---- + +'''Example''' + +Query sequence:: + >seq1 + ATCG... + +.. image:: my_figure.png + :height: 500 + :width: 600 + + +``` + +]]> + + + + + + + + + + + Must be unique across all tools; +should be lowercase and contain only letters, numbers, and underscores. +It allows for tool versioning and metrics of the number of times a tool is used, +among other things. + + + + + This string is what is displayed as a +hyperlink in the tool menu. + + + + + This string defaults to ``1.0.0`` if it is not +included in the tag. It allows for tool versioning and should be increased with each new version +of the tool. + + + + + Allows for tools to be loaded upon +server startup, but not displayed in the tool menu. This attribute should be +applied in the toolbox configuration instead and so should be considered +deprecated. + + + + + + Disable the display the tool's +graphical tool form by setting this to ``false``. + + + + + Allows for certain framework +functionality to be performed on certain types of tools. Normal tools that execute +typical command-line jobs do not need to specify this, special kinds of tools such +as [Data Source](https://wiki.galaxyproject.org/Admin/Internals/DataSources) and +[Data Manager](https://wiki.galaxyproject.org/Admin/Tools/DataManagers) tools should +set this to have values such as ``data_source`` or ``manage_data``. + + + + + This string specified the minimum Galaxy +version that should be required to run this tool. Certain legacy behaviors such +as using standard error content to detect errors instead of exit code are disabled +automatically if profile is set to any version newer than ``16.01``, such as +``16.04``. + + + + + This attribute indicates if +this tool is usable within a workflow (defaults to ``true`` for normal tools and +``false`` for data sources). + + + + + Only used if ``tool_type`` attribute value +is ``data_source`` - this attribute defines the HTTP request method to use when +communicating with an external data source application (the default is ``get``). + + + + + + + + Describe the backend Python action to execute for this Galaxy tool. + + + + + + + + + + + + + + + + + + + + + + + samtools + +``` + +This older example shows a tool that requires R version 2.15.1. The +``tool_depensencies.xml`` should contain matching declarations for Galaxy to +actually install the R runtime. The ``set_envirornment`` type is only respected +by the tool shed and is ignored by the newer and preferred conda dependency +resolver. + +```xml + + R_SCRIPT_PATH + R + +``` + +]]> + + + + + + This value defines the which type of the 3rd party module required by this tool. + + + + + For package type requirements this value defines a specific version of the tool dependency. + + + + + + + + + + + + + + This value describes the type of container that the tool may be executed in and currently must be 'docker'. + + + + + + + + + Documentation for Parallelism + + + + Documentation for method + + + + + Documentation for merge_outputs + + + + + Documentation for split_inputs + + + + + Documentation for split_size + + + + + Documentation for split_mode + + + + + + +``` + +Here are the relevant input parameters in our tool config. The first parameter +is the input dataset that includes the above metadata elements. + +```xml + + value is not None and len(value.metadata.field_names) > 0 + +``` + +The following parameter dynamically renders a select list consisting of the +elements in the ``field_names`` metadata element associated with the selected +input dataset. + +```xml + + + + + + +``` + +The following parameter calls the ``get_field_components_options()`` function in +the ``tool_form_utils.py`` code file discussed above. This function returns the +value of the input dataset's ``field_components`` metadata element dictionary +whose key is the currently selected ``field_name`` from the select list parameter +above. + +```xml + +``` + +Changing the selected option in the ``field_name`` select list will dynamically +re-render the options available in the associated ``field_component_index`` select +list, which is the behavior we want. + +The ``get_field_components_options()`` method looks like this. + +```python +def get_field_components_options( dataset, field_name ): + options = [] + if dataset.metadata is None: + return options + if not hasattr( dataset.metadata, 'field_names' ): + return options + if dataset.metadata.field_names is None: + return options + if field_name is None: + # The expression validator that helps populate the select list of input + # datsets in the icqsol_color_surface_field tool does not filter out + # datasets with no field field_names, so we need this check. + if len( dataset.metadata.field_names ) == 0: + return options + field_name = dataset.metadata.field_names[0] + field_components = dataset.metadata.field_components.get( field_name, [] ) + for i, field_component in enumerate( field_components ): + options.append( ( field_component, field_component, i == 0 ) ) + return options +``` + +]]> + + + + + This value is the name of the executable code file, and is called in the exec_before_process(), exec_before_job(), exec_after_process() and exec_after_job()( methods. + + + + + + Used only for data source tools, this directive contains UI options (currently only ``minwidth`` is valid). + + + + Documentation for minwidth + + + + + + + This directive is used to specify some rarely modified options. + + + + Deprecated, likely unused attribute. + + + + + This attribute can be used to turn off all input sanitization for a tool. + + + + + + + This directive is used to specify some rarely modified trackster options. + + + + + + + + + + + + + + + + + + + + + + + + + tag sets. Any number of tests can be included, +and each test is wrapped within separate tag sets. Functional tests are +executed via [Planemo](http://planemo.readthedocs.io/) or the +[run_tests.sh](https://github.com/galaxyproject/galaxy/blob/dev/run_tests.sh) +shell script distributed with Galaxy. + +The documentation contained here is mostly reference documentation, for +tutorials on writing tool tests please check out Planemo's +[Test-Driven Development](http://planemo.readthedocs.io/en/latest/writing_advanced.html#test-driven-development) +documentation or the much older wiki content for +[WritingTests](https://wiki.galaxyproject.org/Admin/Tools/WritingTests). + +]]> + + + + + + + + + + + + + + + + + + + + + + + + +``` + +The following example, tests the execution of the MAF-to-FASTA converter +([/tools/maf/maf_to_fasta.xml](https://github.com/galaxyproject/galaxy/blob/master/tools/maf/maf_to_fasta.xml)). + +```xml + + + + + + + + +``` + +This test demonstrates verifying specific properties about a test output instead +of directly comparing it to another file. Here the file attribute is not +specified and instead a series of assertions is made about the output. + +```xml + + + + + + + + + + + + + + + +``` + +]]> + + + + + + + Describe the job's expected exit code. + + + + + Assert the number of outputs this test +should produce, this is useful to ensure ``filter`` directives are implemented correctly. + + + + + Setting this to ``true`` indicates +the expectation is for the job fail. If set to ``true`` no job output checks may +be present in ``test`` definition. + + + + + Maximum amount of time to let test run. + + + + + + + + + + + + + + Describe assertions about the job's +generated command-line. + +$assertions + + + + + + Describe assertions about the job's +standard output. + +$assertions + + + + + + Describe assertions about the job's +standard error. + +$assertions + + + + + + + + +
+ +
+
+ +
+ + + + + + + +``` + +]]> + + + + + + + + This value must match the name of the +associated input ``section``. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +The [tophat2](https://github.com/galaxyproject/tools-devteam/blob/master/tools/tophat2/tophat2_wrapper.xml) +tool demonstrates a real tool that benefits from more structured test cases +using the ``conditional`` test directive. One such test case from that tool is +shown below. + +```xml + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +]]> + + + + + + + This value must match the name of the +associated input ``conditional``. + + + + + + `` elements in the ```` definition. + +```xml + + + + + + + + + + + + + + + +``` + +The second definition in that file demonstrates repeated ```` blocks +allowing multiple instances of a single repeat to be specified. + +```xml + + + + + + + + + + + + + + + + +``` + +]]> + + + + + + + This value must match the name of the +associated input ``repeat``. + + + + + + + + + + + + + + + This value must match the name of the +associated input parameter (``param``). + + + + + This value must be one of the legal +values that can be assigned to an input parameter. + + + + + This attribute name should be included +only with parameters of ``type`` ``data`` for the tool. If this +attribute name is not included, the functional test framework will attempt to +determine the data type for the input dataset using the data type sniffers. + + + + + Specifies a ``dbkey`` value for the +referenced input dataset. This is only valid if the corresponding parameter is +of ``type`` ``data``. + + + + + + Define extra composite input files for test input. + + + + Path relative to test-data of composite file. + + + + + Optional datatype of composite file for test input. + + + + + + Definition of a collection for test input. + + + + + + + Type of collection to create. + + + + + + `` tag sets and generate a +temporary file, which will either be compared with the file named in the +``file`` attribute value or checked against assertions made by a child +``assert_contents`` tag to verify that the tool is functionally correct. + + ]]> + + + + + + + + `` +tag set contained within the tool's ```` tag set. + +]]> + + + + + + + + + + + + + + + This flag causes the lines of the output +to be sorted before they are compared to the expected output. This could be +useful for non-deterministic output. + + + + + An alias for ``file``. + + + + + + + + + + + + + + + + + If ``compare`` is set to ``diff``, the number of lines of difference to allow (each line with a modification is a line added and a line removed so this counts as two lines). + + + + + If ``compare`` is set to ``sim_size``, this is the number of bytes different allowed. + + + + + + + + + + + + + + + + + + + + + + +``` + +The following demonstrtes a wide variety of XML assertion statements. + +```xml + + + + + + + + + + + + + +``` + +The following demonstrtes verifying XML content with XPath-like expressions. + +```xml + + + + + + +``` + +]]> + + + + + + + + + + + Documentation for name + + + + + Documentation for value + + + + + + + + + + + + + +``` + +]]> + + + + Name of the metdata element to check. + + + + + Expected value (as a string) of metadata value. + + + + + + + + + + + + + + + + + + + + + +``` + +]]> + + + + + + The designation of the discovered dataset. + + + + + + + + Define test for extra files on corresponding output. + + + + + + Extra file type (either ``file`` or ``directory``). + + + + + + + + + + + + + + + + + + + +``` + +The [CWPair2](https://github.com/galaxyproject/tools-iuc/blob/master/tools/cwpair2/cwpair2.xml) +tool demonstrates that ``element``s can specify a ``compare`` attribute just +like [``output``](#tool|tests|test|output). + +```xml + + + + + + + + + + + + + + + + +``` + +The +[collection_creates_dynamic_nested](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/collection_creates_dynamic_nested.xml) +test tool demonstrates the use of nested ``element`` directives as described +above. Notice also that it tests the output with ``assert_contents`` instead of +supplying a ``file`` attribute. Like hinted at with with ``compare`` attribute +above, the ``element`` tag can specify any of the test attributes that apply to +the [``output``](#tool|tests|test|output) (e.g. ``md5``, ``compare``, ``diff``, +etc...). + +```xml + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +]]> + + + + + + + `` tag set contained within the tool's ```` tag set. + +]]> + + + + + Expected collection type (e.g. ``list``, ``paired``, +or ``list:paired``). + + + + + Number of elements in output collection. + + + + + + + + + + + + + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + `` ).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + ``).]]> + + + + + + `` ).]]> + + + + + + ``).]]> + + + + + ``).]]> + + + + + + + `` tag within the ```` tag set +maps to a command line parameter within the [``command``](#tool|command) tag. Most +tools will not need to specify any attributes on this tag itself.]]> + + + + + + + + URL used by data source tools. + + + + + Set to ``false`` to disable parameter checking in data source tools. + + + + + Data source HTTP action (e.g. ``get`` or ``put``) to use. + + + + + UI link target to use for data source tools (e.g. ``_top``). + + + + + This boolean indicates if this is an upload tool or not. + + + + + + + + + + + + + + Internal, intentionally undocumented feature. + + + + + Documentation for display + + + + + + + Documentation for InputType + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +The first directive following the conditional is a [``param``](#tool|inputs|param), +this param must be of type ``select`` or ``boolean``. Depending on the value a +user selects for this "test" parameter - different UI elements will be shown. +These different paths are described by the following the ``when`` blocks shown +above. + +The following Cheetah block demonstrates the use of the ``conditional`` +shown above: + +``` +biom convert -i "${input_type.input_table}" -o "${output_table}" +#if str( $input_type.input_type_selector ) == "tsv": + #if $input_type.process_obs_metadata: + --process-obs-metadata "${input_type.process_obs_metadata}" + #end if +#end if +``` + +Notice that the parameter ``input_table`` appears down both ``when`` clauses +so ``${input_type.input_table}`` appears unconditionally but we need to +conditionally reference ``${input_type.process_obs_metadata}`` with a Cheetah +``if`` statement. + +A common use of the conditional wrapper is to select between reference data +managed by the Galaxy admins (for instance via +[data managers](https://wiki.galaxyproject.org/Admin/Tools/DataManagers) +) and +history files. A good example tool that demonstrates this is +the [Bowtie 2](https://github.com/galaxyproject/tools-devteam/blob/master/tools/bowtie2/bowtie2_wrapper.xml) wrapper. + +```xml + + + + + + + + + + + + + + + + + +``` + +The Bowtie 2 wrapper also demonstrates other conditional paths - such as choosing +between paired inputs of single stranded inputs. +]]> + + + + + + + + + + Name for this element + + + + + + + + + + + + + + + + + + + + + Human readable description for the conditional, unused in the Galaxy UI currently. + + + + + + + + + + + + This directive describes one potential +set of input for the tool at this depth. See documentation for the +[``conditional``](#tool|inputs|conditional) block for more details and examples (XML +and corresponding Cheetah conditionals). + + + + + + + Value for the tool form test parameter +corresponding to this ``when`` block. + + + + + + + + + `` tag set. When this is used, the +tool will allow the user to add any number of additional sets of the contained +parameters (an option to add new iterations will be displayed on the tool form). +All input parameters contained within the ```` tag can be retrieved by +enumerating over ``$`` in the relevant Cheetah code. +This returns the rank and the parameter objects of the repeat container. See the +Cheetah code below. + +### Example + +This part is contained in the ```` tag set. + +```xml + + + + + +``` + +This Cheetah code can be used in the ```` tag set or the +```` tag set. + +```xml +#for $i, $s in enumerate( $series ) + rank_of_series=$i + input_path='${s.input}' + x_colom=${s.xcol} + y_colom=${s.ycol} +#end for +``` + +### Testing + +This is an example test case with multiple repeat elements for the example above. + +```xml + + + + + + + + + + + + + +``` + +See the documentation on the [``repeat`` test directive](#tool|tests|test|repeat). + +An older way to specify repeats in a test is by instances that are created by referring to names with a special format: "_|" + +```xml + + + + + + + + + +``` + +The test tool [disambiguate_repeats.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/disambiguate_repeats.xml) +demonstrates both testing strategies. + +]]> + + + + + + + + + Name for this element + + + + + The title of the repeat section, which will be displayed on the tool form. + + + + + The minimum number of repeat units. + + + + + The maximum number of repeat units. + + + + + The default number of repeat units. + + + + + Short help description for repeat element. + + + + + + + + + +
+ +
+ +``` + +In your command template, you'll need to include the section name to access the +variable: + +``` +--color $adv.plot_color +``` + +Further examples can be found in the [test case](https://github.com/galaxyproject/galaxy/blob/master/test/functional/tools/section.xml) from [PR #35](https://github.com/galaxyproject/galaxy/pull/35) +]]> + + + + + + + The internal key used for the section. + + + + + Human readable label for the section. + + + + + Whether the section should be expanded by default or not. If not, the default set values are used. + + + + + + + `` tag set - each of these specifies a field that +will be displayed on the tool form. Ultimately, the values of these form fields +will be passed as the command line parameters to the tool's executable. + +### Common Attributes + +The attributes valid for this tag vary wildly based on the ``type`` of the +parameter being described. All the attributes for the ``param`` element are +documented below for completeness, but here are the common ones for each +type are as follows: + +$attribute_list:name,type,optional,label,help,argument + +### Parameter Types + +#### ``text`` + +When ``type="text"``, the parameter is free form text and appears as a text box +in the tool form. + +$attribute_list:value,size,area + +##### Examples + +Sometimes you need labels for data or graph axes, chart titles, etc. This can be +done using a text field. The following will create a text box 30 characters wide +with the default value of "V1". + +```xml + +``` + +The ``size`` parameter can be two dimensional, if it is the textbox will be +rendered on the tool form as a text area instead of a single line text box. + +```xml + +``` + +#### ``integer`` and ``float`` + +These parameters represent whole number and real numbers, respectively. + +$attribute_list:value,min,max + +##### Example + +``` + +``` + +#### ``boolean`` + +This represents a binary true or false value. + +$attribute_list:checked,truevalue,falsevalue + +#### ``data`` + +A dataset from the current history. Multiple types might be used for the param form. + +##### Examples + +The following will find all "coordinate interval files" contained within the +current history and dynamically populate a select list with them. If they are +selected, their destination and internal file name will be passed to the +appropriate command line variable. + +```xml + +``` + +The following demonstrates a ``param`` which may accept multiple files and +multiple formats. + +```xml + +``` + +$attribute_list:format,multiple + +#### ``select`` + +The following will create a select list containing the options "Downstream" and +"Upstream". Depending on the selection, a ``d`` or ``u`` value will be passed to +the ``$upstream_or_down`` variable on the command line. + +```xml + + + + +``` + +The following will create a checkbox list allowing the user to select +"Downstream", "Upstream", both, or neither. Depending on the selection, the +value of ``$upstream_or_down`` will be ``d``, ``u``, ``u,d``, or "". + +```xml + + + + +``` + +$attribute_list:data_ref,dynamic_options,display,multiple + +#### ``data_column`` + +This parameter type is used to select columns from a parameter. + +$attribute_list:force_select,numerical,use_header_name + +#### ``drill_down`` + +$attribute_list:hierarchy + +#### ``data_collection`` + +$attribute_list:format,collection_type + +The following will create a parameter that only accepts paired FASTQ files grouped into a collection. + +##### Examples + +```xml + + +``` + +More detailed information on writing tools that consume collections can be found +in the [planemo documentation](http://planemo.readthedocs.io/en/latest/writing_advanced.html#collections). + +#### ``color`` + +$attribute_list:value + +##### Examples + +The following example will create a color selector parameter. + +```xml + + +``` + +Given that the output includes a pound sign, it is often convenient to use a +sanitizer to prevent Galaxy from escaping the result. + +```xml + + + + + + + +``` + +This covers examples of the most common parameter types, the remaining parameter +types are more obsecure and less likely to be useful for most tool authors. + +]]> + + + + + + + + + + + + + + + + + + + + Boolean indicating if this should be +rendered as a one line text box (if ``false``) or a multi-line text area (if +``true``). + + + + + + + + + + The attribute value will be +displayed on the tool page as the label of the form field +(``label="Sort Query"``). + + + + + Short bit of text, rendered on the +tool form just below the associated field to provide information about the +field. + + + + + The default value for this +parameter. + + + + + If ``false``, parameter must have a +value. Defaults to "false". + + + + + Minimum valid parameter value - only +valid when ``type`` is ``integer`` or ``float``. + + + + + Maximum valid parameter value - only +valid when ``type`` is ``integer`` or ``float``. + + + + + Only if ``type`` attribute value is +``data`` or ``data_collection`` - the list of supported data formats is +contained in the +[/config/datatypes_conf.xml.sample](https://github.com/galaxyproject/galaxy/blob/dev/config/datatypes_conf.xml.sample) +file. Use the file extension. + + + + + + + + + + + + + + + + + + + + Used only if the ``type`` attribute +value is ``data_column``, this is deprecated and the inverse of ``optional``. +Set to ``false`` to not force user to select an option in the list. + + + + + Used only if the ``type`` attribute +value is ``data_column``, if ``true`` Galaxy assumes first row of ``data_ref`` +is a header and builds the select list with these values rather than the more +generic ``c1`` ... ``cN``. + + + + + + This attribute is used only if +``type`` attribute value is ``select`` - render a select list as a set of check +boxes or radio buttons. Defaults to a drop-down menu select list. + + + + + Allow multiple valus to be selected. +Valid with ``data`` and ``select`` parameters. + + + + + Used only if the ``type`` attribute +value is ``data_column``, if ``true`` the column will be treated as numerical +when filtering columns based on metadata. + + + + + Used only if the ``type`` attribute +value is ``drill_down``, this attribute determines the drill down is +``recursive`` or ``exact``. + + + + + Set to ``true`` if the ``boolean`` +parameter should be checked (or ``true``) by default. + + + + + The parameter value in the Cheetah +template if the parameter is ``true`` or checked by the user. Only valid if +``type`` is ``boolean``. + + + + + The parameter value in the Cheetah +template if the parameter is ``false`` or not checked by the user. Only valid if +``type`` is ``boolean``. + + + + + + Used only if ``type`` attribute +value is ``text``. To create a multi-line text box add an ``area="True"`` +attribute to the param tag. This can be one dimensional (e.g. ``size="40"``) +or two dimensional (e.g. ``size="5x25"``). + + + + + + + Deprecated/discouraged method to +allow access to Python code to generate options for a select list. See +``code``'s documentation for an example. + + + + + + + + + + + + Documentation for label + + + + + + + + + Documentation for help + + + + + + + + Documentation for ParamType + + + + + + + + + + + + + + + + + + + + + + + + bed12ToBed6 -i '$input' > '$output' +``` + +A few things to note about even this simple example: + +* Input and output variables (boringly named ``input`` and ``output``) + are expanded into paths using the ``$`` Cheetah directive. +* Paths should be quoted so that the Galaxy database files may contain spaces. +* We are building up a shell script - so special characters like ``>`` can be used + (in this case the standard output of the bedtools call is written to the path + specified by ``'$output'``). + +The bed12ToBed6 tool can be found [here](https://github.com/galaxyproject/tools-iuc/blob/master/tools/bedtools/bed12ToBed6.xml). + +A more sophisticated bedtools example demonstrates the use of loops, conditionals, +and uses whitespace to make a complex command very readable can be found in +[annotateBed](https://github.com/galaxyproject/tools-iuc/blob/master/tools/bedtools/annotateBed.xml) +tool. + +```xml + "${output}" +]]]]> +``` + +The following example (taken from [xpath](https://github.com/galaxyproject/tools-iuc/blob/master/tools/xpath/xpath.xml) tool) +uses an interpreted executable. In this case a Perl script is shipped with the +tool and the directory of the tool itself is referenced with ``$__tool_directory__``. + +```xml + + perl $__tool_directory__/xpath -q -e '$expression' '$input' > '$output' + +``` + +The following example demonstrates accessing metadata from datasets. Metadata values +(e.g., ``${input.metadata.chromCol}``) are acquired from the ``Metadata`` model associated +with the objects selected as the values of each of the relative form field +parameters in the tool form. Accessing this information is generally enabled using +the following feature components. + +A set of "metadata information" is defined for each supported data type (see the +``MetadataElement`` objects in the various data types classes in +[``/lib/galaxy/datatypes``](https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/datatypes). +The ``DatasetFilenameWrapper`` class in the +[/lib/galaxy/tools/wrappers.py](https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/tools/wrappers.py) +code file wraps a metadata collection to return metadata parameters wrapped +according to the Metadata spec. + +```xml + +``` + +In additon to demonstrating accessing metadata, this example demonstrates: + +* ``$input.is_of_type("gff")`` which can be used to check if an input is of a + given datatype. +* ``#set datatype = $input.datatype`` which is the syntax for defining variables + in Cheetah. + + + +### Reserved Variables + +Galaxy provides a few pre-defined variables which can be used in your command line, +even though they don't appear in your tool's parameters. + +Name | Description +---- | ----------- +``$__tool_directory__`` | The directory the tool description (XML file) currently resides in (new in 15.03) +``$__new_file_path__`` | ``config/galaxy.ini``'s ``new_file_path`` value +``$__tool_data_path__`` | ``config/galaxy.ini``'s tool_data_path value +``$__root_dir__`` | Top-level Galaxy source directory made absolute via ``os.path.abspath()`` +``$__datatypes_config__`` | ``config/galaxy.ini``'s datatypes_config value +``$__user_id__`` | Email's numeric ID (id column of ``galaxy_user`` table in the database) +``$__user_email__`` | User's email address +``$__app__`` | The ``galaxy.app.UniverseApplication`` instance, gives access to all other configuration file variables (e.g. $__app__.config.output_size_limit). Should be used as a last resort, may go away in future releases. + +Additional runtime properties are available as environment variables. Since these +are not Cheetah variables (the values aren't available until runtime) these should likely +be escaped with a backslash (``\``) when appearing in ``command`` or ``configfile`` elements. + +Name | Description +---- | ----------- +``\${GALAXY_SLOTS:-4}`` | Number of cores/threads allocated by the job runner or resource manager to the tool for the given job (here 4 is the default number of threads to use if running via custom runner that does not configure GALAXY_SLOTS or in an older Galaxy runtime). + +See the [Planemo docs](http://planemo.readthedocs.io/en/latest/writing_advanced.html#cluster-usage) +on the topic of ``GALAXY_SLOTS`` for more information and examples. + +### Attributes + +#### ``detect_errors`` + +If present on the ``command`` tag, this attribute can be one of: + +* ``default`` no-op fallback to ``stdio`` tags and erroring on standard error output (for legacy tools). +* ``exit_code`` error if tool exit code is not 0. (The @jmchilton recommendation). +* ``aggressive`` error if tool exit code is not 0 or either ``Exception:`` or ``Error:`` + appears in standard error/output. (The @bgruening recommendation). + +For newer tools with ``profile>=16.04``, the default behavior is ``exit_code``. +Legacy tools default to ``default`` behavior described above (erroring if the tool +produces any standard error output). + +See [PR 117](https://github.com/galaxyproject/galaxy/pull/117) for more implementation +information and discussion on the ``detect_errors`` attribute. + +#### ``strict`` + +This boolean forces the ``#set -e`` directive on in shell scripts - so that in a +multi-part command if any part fails the job exits with a non-zero exit code. +This is enabled by default for tools with ``profile>=16.04`` and disabled on +legacy tools. + +#### ``interpreter`` + +Older tools may define an ``intepreter`` attribute on the command, but this is +deprecated and using the ``$__tool_directory__`` variable is superior. + +]]> + + + + + + + + + + + This attribute defines the programming language in which the tool's executable file is written. Any language can be used (tools can be written in Python, C, Perl, Java, etc.). The executable file must be in the same directory of the XML file. If instead this attribute is not specified, the tag content should be a Bash command calling executable(s) available in the $PATH. + + + + + + + + + + + + + + + ``). + +### Example + +```xml + + + + + +``` + +An option can also be annotated with ``selected="true"`` to specify a +default option. + +```xml + + + + + + + +``` +]]> + + + + + + + + + + + + A boolean parameter indicating +if the corresponding option is selected by default (the default is ``false``). + + + + + + + + + + + `` tag when the ``type`` attribute value is ``select`` or +``data`` and used to dynamically generated lists of options. This tag set +dynamically creates a list of options whose values can be +obtained from a predefined file stored locally or a dataset selected from the +current history. + +There are at least five basic ways to use this tag - four of these correspond to +a ``from_XXX`` attribute on the ``options`` directive and the other is to +exclusively use ``filter``s to populate options. + +* ``from_data_table`` - The options for the select list are dynamically obtained + from a file specified in the Galaxy configuration file + ``tool_data_table_conf.xml`` or from a Tool Shed installed data manager. +* ``from_dataset`` - The options for the select list are dynamically obtained + from input dataset selected for the tool from the current history. +* ``from_file`` - The options for the select list are dynamically obtained from + a file. This mechanis is discourage in favor of the more generic + ``from_data_table``. +* ``from_parameter`` - The options for the select list are dynamically obtained + from a parameter. +* Using ``filter``s - various filters can be used to populate options, see + examples in the [``filter``](#tool|inputs|param|options|filter) documentation. + +### ``from_data_table`` + +See Galaxy's +[data tables documentation](https://wiki.galaxyproject.org/Admin/Tools/Data%20Tables) +for information on setting up data tables. + +Once a data table has been configured and populated, these can be easily +leveraged via tools. + +This ``conditional`` block in the +[bowtie2](https://github.com/galaxyproject/tools-devteam/blob/master/tools/bowtie2/bowtie2_wrapper.xml) +wrapper demonstrates using ``from_data_table`` options as an +alternative to local reference data. + +```xml + + + + + + + + + + + + + + + + + +``` + +A minimal example wouldn't even need the ``filter`` or ``validator`` above, but +they are frequently nice features to add to your wrapper and can improve the user +experience of a tool. + +### ``from_dataset`` + +The following example is taken from the Mothur tool +[remove.lineage.xml](https://github.com/galaxyproject/tools-iuc/blob/master/tools/mothur/remove.lineage.xml) +and demonstrates generating options from a dataset directly. + +```xml + + + + + + + + + + + + + + + + + +``` + +Filters can be used to generate options from dataset directly also as the +example below demonstrates (many more examples are present in the +[``filter``](#tool|inputs|param|options|filter) documentation). + +```xml + + + + + +``` + +### ``from_file`` + +The following example is for Blast databases. In this example users maybe select +a database that is pre-formatted and cached in Galaxy clusters. When a new +dataset is available, admins must add the database to the local file named +"blastdb.loc". All such databases in that file are included in the options of +the select list. For a local instance, the file (e.g. ``blastdb.loc`` or +``alignseq.loc``) must be stored in the configured +[``tool_data_path``](https://github.com/galaxyproject/galaxy/tree/master/tool-data) +directory. In this example, the option names and values are taken from column 0 +of the file. + +```xml + + + + + + +``` + +In general, ``from_file`` should be considered deprecated and ``from_data_table`` +should be prefered. + +### ``from_parameter`` + +This variant of the ``options`` directive is discouraged because it exposes +internal Galaxy structures. See the older +[bowtie](https://github.com/galaxyproject/tools-devteam/blob/master/tools/bowtie_wrappers/bowtie_wrapper.xml) +wrappers for an example of these. + +### Other Ways to Dynamically Generate Options + +Though deprecated and discouraged, [``code``](#tool|code) blocks can also be +used to generate dynamic options. + +]]> + + + + + + + + Documentation for from_dataset + + + + + Documentation for from_file + + + + + Documentation for from_data_table + + + + + Documentation for from_parameter + + + + + Documentation for options_filter_attribute + + + + + Documentation for transform_lines + + + + + Documentation for startswith + + + + + + + + + + + Documentation for file + + + + + + + `` tag set - specifies columns used in building select options from a +file stored locally (i.e. index or tool data) or a dataset in the +current history. + +Any number of columns may be described, but at least one must be given the name +``value`` and it will serve as the value of this parameter in the Cheetah +template and elsewhwere (e.g. in API for instance). + +If a column named ``name`` is defined, this too has special meaning and it will +be the value the tool form user sees for each option. If no ``name`` column +appears, ``value`` will serve as the name. + +### Examples + +The following fragment shows options from the dataset in the current history +that has been selected as the value of the parameter named ``input1``. + +```xml + + + + +``` + +The [interval2maf](https://github.com/galaxyproject/galaxy/blob/dev/tools/maf/interval2maf.xml) +tool makes use of this tag with files from a history, and the +[star_fusion](https://github.com/galaxyproject/tools-iuc/blob/master/tools/star_fusion/star_fusion.xml) +tool makes use of this to reference a data table. +]]> + + + + Name given to the column with index +``index``, the names ``name`` and ``value`` have special meaning as described +above. + + + + + 0-based index of the column in the +target file. + + + + + + + tag set - it applies a validator to the containing parameter. + +### Examples + +The following demonstrates a simple validator ``unspecified_build`` ensuring +that a dbkey is present on the selected dataset. This example is taken from the +[extract_genomic_dna](https://github.com/galaxyproject/tools-iuc/blob/master/tools/extract_genomic_dna/extract_genomic_dna.xml#L42) +tool. + +```xml + + + +``` + +Along the same line, the following example taken from +[samtools_mpileup](https://github.com/galaxyproject/tools-devteam/blob/master/tool_collections/samtools/samtools_mpileup/samtools_mpileup.xml) +ensures that a dbkey is present and that FASTA indices in the ``fasta_indexes`` +tool data table are present. + +```xml + + + + +``` + +In this older, somewhat deprecated example - a genome build of the dataset must +be stored in Galaxy clusters and the name of the genome (``dbkey``) must be one +of the values in the first column of file ``alignseq.loc`` - that could be +expressed with the validator. In general, ``dataset_metadata_in_file`` should be +considered deprecated in favor of + +```xml + +``` + +A very common validator is simply ensure a Python expression is valid for a +specified value. In the following example - paths/names that downstream tools +use in filenames may not contain ``..``. + +```xml +'..' not in value +``` +]]> + + + + + + + + + + + +The error message displayed on the tool form if validation fails. + + + + + Comma-seperated list of metadata +fields to check for if type is ``metadata``. If not specified, all non-optional +metadata fields will be checked unless they appear in the list of fields +specified by the ``skip`` attribute. + + + + + Tool data table name to check against +if ``type`` is ``dataset_metadata_in_tool_data``. See the documentation for +[tool data tables](https://wiki.galaxyproject.org/Admin/Tools/Data%20Tables) +and [data managers](https://wiki.galaxyproject.org/Admin/Tools/DataManagers) for +more information. + + + + + Tool data filename to check against +if ``type`` is ``dataset_metadata_in_file``. File should be present Galaxy's +``tool-data`` directory. + + + + + Target metadata attribute name for +``dataset_metadata_in_data_table`` and ``dataset_metadata_in_file`` options. + + + + + Target column for metadata attribute +in ``dataset_metadata_in_data_table`` and ``dataset_metadata_in_file`` options. +This can be an integer index to the column or a column name. + + + + + Used to indicate lines in the file +being used for validation start with a this attribute value. + + + + + When the ``type`` attribute value is +``in_range`` - this is the minimum number allowed. + + + + + When the ``type`` attribute value is +``in_range`` - this is the maximum number allowed. + + + + + When the ``type`` attribute value is +``in_range`` - this boolean indicates if the ``min`` value is allowed. + + + + + When the ``type`` attribute value is +``in_range`` - this boolean indicates if the ``max`` value is allowed. + + + + + If ``type`` is `dataset_metadata_in_file``, +this attribute is the column separator to use for values in the specified file. +This default is ``\t`` and due to a bug in older versions of Galaxy, should +not be modified. + + + + + Comma-seperated list of metadata +fields to skip if type is ``metadata``. If not specified, all non-optional +metadata fields will be checked unless ``check`` attribute is specified. + + + + + + + + `` tag set - it contains a set of ```` and +```` tags. + +### Examples + +This example replaces the invalid character default of ``X`` with the empty +string (so invalid characters are effectively dropped instead of replaced with +``X``) and indicates the only valid characters for this input are ASCII letters, +ASCII digits, and ``_``. + +``` + + + + + +``` + +This example allows many more valid characters and specifies that ``&`` will just +be dropped from the input. + +``` + + + + + + + + +``` +]]> + + + + + + + This boolean parameter determines if the +input is sanitized at all (the default is ``true``). + + + + + The attribute specifies the character +used as a replacement for invalid characters. + + + + + + + + + + + + `` tag set, these are used to specify a list of allowed characters. +Contains ```` and ```` tags.]]> + + + + + + + This describes the initial characters to +allow as valid, the default is ``string.letters + string.digits + " -=_.()/+*^,:?!"`` + + + + + + This directive is used to add individual +characters or preset lists of characters. Character must not be allowed as a +valid input for the mapping to occur. Preset lists include default and none as well as those available from string.* (e.g. ``string.printable``). + + + + Add target characters from the list of valid characters (e.g. ``string.printable``). + + + + + Add a character to the list of valid characters. + + + + + + This directive is used to remove +individual characters or preset lists of characters. +Character must not be allowed as a valid input for the mapping to occur. +Preset lists include default and none as well as those available from string.* (e.g. ``string.printable``). + + + + Remove characters from the list of valid characters (e.g. ``string.printable``). + + + + + A character to remove from the list of valid characters. + + + + + + + + + + + + tag set. Used to specify a mapping of disallowed character to replacement string. Contains and tags.]]> + + + + + + + initial character mapping (default is ``galaxy.util.mapped_chars``) + + + + + + + + + + Replace all occurrences of this character with the string of ``target``. + + + + + Replace all occurrences of ``source`` with this string + + + + + + + + + + Character to remove from mapping. + + + + + + + + + + + + `` tag set - filter out values obtained from a locally stored file (e.g. +a tool data table) or a dataset in the current history. + +### Examples + +The following example from Mothur's +[remove.groups.xml](https://github.com/galaxyproject/tools-iuc/blob/master/tools/mothur/remove.groups.xml) +tool demonstrates filtering a select list based on the metadata of an input to +to the tool. + +```xml + + + + + + +``` + +This more advanced example, taken from Mothur's +[remove.linage.xml](https://github.com/galaxyproject/tools-iuc/blob/master/tools/mothur/remove.lineage.xml) +tool demonstrates using filters to sort a list and remove duplicate entries. + +```xml + + + + + + + + + + + + + + + + + +``` + +This example taken from the +[hisat2](https://github.com/galaxyproject/tools-iuc/blob/master/tools/hisat2/hisat2.xml) +tool demonstrates filtering values from a tool data table. + +```xml + + + + + + +``` + +The +[gemini_load.xml](https://github.com/galaxyproject/tools-iuc/blob/master/tools/gemini/gemini_load.xml) +tool demonstrates adding values to an option list using ``filter``s. + +```xml + + + + + + + +``` + +While this fragment from maf_to_interval.xml demonstrates removing items. + +```xml + + + + + + +``` + +This example taken from +[snpSift_dbnsfp.xml](https://github.com/galaxyproject/tools-iuc/blob/master/tool_collections/snpsift/snpsift_dbnsfp/snpSift_dbnsfp.xml) +demonstrates splitting up strings into multiple values. + +```xml + + + + + + + + +``` +]]> + + + + + + + + + Column targeted by this filter - this +attribute is unused and invalid if ``type`` is ``add_value`` or ``remove_value``. +This can be a column index or a column name. + + + + + Name displayed for value to add (only +used with ``type`` of ``add_value``). + + + + + The attribute name of the reference file +(tool data) or input dataset. Only used when ``type`` is +``data_meta`` (required), ``param_value`` (required), or ``remove_value`` +(optional). + + + + + When ``type`` is ``data_meta``, ``param_value``, +or ``remove_value`` - this is the name of the metadata key of ref to filter by. + + + + + + + + + + + + + + + If ``true``, keep columns matching the +value, if ``false`` discard columns matching the value. Used when ``type`` is +either ``static_value`` or ``param_value``. + + + + + Target value of the operations - has +slightly different meanings depending on ``type``. For instance when ``type`` is +``add_value`` it is the value to add to the list and when ``type`` is +``static_value`` it is the value compared against. + + + + + Only used when ``type`` is +``param_value``. Period (``.``) separated attribute chain of input (``ref``) +attributes to use as value for filter. + + + + + Used when ``type`` is ``add_value``, it +is the index into the list to add the option to. If not set, the option will be +added to the end of the list. + + + + + + `` and ```` tag sets. +The files and collections created by tools as a result of their execution are +named by Galaxy. You specify the number and type of your output files using the +contained ```` and ```` tags. These may be passed to your tool +executable through using line variables just like the parameters described in +the ```` documentation. +]]> + + + + + + + + + + + + + + + + + + + + + + `` tag set, and it defines the +output data description for the files resulting from the tool's execution. The +value of the attribute ``label`` can be acquired from input parameters or metadata +in the same way that the command line parameters are (discussed in the + tag set section above). + +### Examples + +The following will create a variable called ``$out_file1`` with data type +``pdf``. + +```xml + + + +``` + +The valid values for format can be found in +[/config/datatypes_conf.xml.sample](https://github.com/galaxyproject/galaxy/blob/dev/config/datatypes_conf.xml.sample). + +The following will create a dataset in the history panel whose data type is the +same as that of the input dataset selected (and named ``input1``) for the tool. + +```xml + + + +``` + +The following will create datasets in the history panel, setting the output data +type to be the same as that of an input dataset named by the ``format_source`` +attribute. Note that a conditional name is not included, so 2 separate +conditional blocks should not contain parameters with the same name. + +```xml + + + + + + + + + + + + + + + + + + + qual['add'] == 'yes' + + +``` + +Assume that the tool includes an input parameter named ``database`` which is a +select list (as shown below). Also assume that the user selects the first option +in the ``$database`` select list. Then the following will ensure that the tool +produces a tabular data set whose associated history item has the label ``Blat +on Human (hg18)``. + +```xml + + + + + + + + + + +``` + +]]> + + + + + + + + + + + + + + + + + + The short name for the output datatype. +The valid values for format can be found in +[/config/datatypes_conf.xml.sample](https://github.com/galaxyproject/galaxy/blob/dev/config/datatypes_conf.xml.sample) +(e.g. ``format="pdf"`` or ``format="fastqsanger"``). + + + + + This sets the data type of the output file to be the same format as that of a tool input dataset. + + + + + This copies the metadata information +from the tool's input dataset. This is particularly useful for interval data +types where the order of the columns is not set. + + + + + .}``, as +discussed for command line parameters in the ```` tag set section +above. The default label is ``${tool.name} on ${on_string}``. + +]]> + + + + + Relative path to a file produced by the +tool in its working directory. Output's contents are set to this file's +contents. + + + + + Boolean indicating whether to hide +dataset in the history view. (Default is ``false``.) + + + + + + + + + + + + + + + `` tag set, and it defines the +output dataset collection description resulting from the tool's execution. The +value of the attribute ``label`` can be acquired from input parameters or +metadata in the same way that the command line parameters are (discussed in the +[``command``](#tool|command) directive). + +Creating collections in tools is covered in-depth in +[planemo's documentation](http://planemo.readthedocs.io/en/latest/writing_advanced.html#creating-collections). + +]]> + + + + + + + + + + + + Collection type for output (e.g. ``paired``, ``list``, or ``list:list``). + + + + + .}``, as +discussed for command line parameters in the ```` tag set section +above. The default label is ``${tool.name} on ${on_string}``. + +]]> + + + + + This is the name of input collection or +dataset to derive output dataset collection's element's format/datatype from. + + + + + This is the name of input collection to +derive collection's type (e.g. ``collection_type``) from. + + + + + This is the name of input collection or +dataset to derive "structure" of the output from (output element count and +identifiers). For instance, if the referenced input has three ordered items with +identifiers ``sample1``, ``sample2``, and ``sample3`` + + + + + If ``structured_like`` is set, inherit +format of outputs from format of corresponding input. + + + + + + + `` tag can contain a ```` tag which includes a Python code +block to be executed to test whether to include this output in the outputs the +tool ultimately creates. If the code, when executed, returns ``True``, +the output dataset is retained. In these code blocks the tool parameters appear +as Python variables and are thus referred to without the $ used for the Cheetah +template (used in the ```` tag). Variables that are part of +conditionals are accessed using a hash named after the conditional. + +### Example + +```xml + + + + + + + + + + + + + + + + + + + + + + + + + + + + Regular expression used to find filenames and parse dynamic properties. + + + + + Directory (relative to working directory) to search for files. + + + + + Format (or datatype) of discovered datasets (an alias with ``ext``). + + + + + Format (or datatype) of discovered datasets (an alias with ``format``). + + + + + Indication if this dataset is visible in output history. This defaults to ``false``, but probably shouldn't - be sure to set to ``true`` if that is your intention. + + + + + Replace the primary dataset described by the parameter ``data`` parameter with the first output discovered. + + + + + + + + + + Regular expression used to find filenames and parse dynamic properties. + + + + + Directory (relative to working directory) to search for files. + + + + + Format (or datatype) of discovered datasets (an alias with ``ext``). + + + + + Format (or datatype) of discovered datasets (an alias with ``format``). + + + + + Indication if this dataset is visible in output history. This defaults to ``false``, but probably shouldn't - be sure to set to ``true`` if that is your intention. + + + + + + `` in the Bowtie 2 wrapper is used in lieu of the deprecated +```` tag to set the ``dbkey`` of the output dataset. In +[bowtie2_wrapper.xml](https://github.com/galaxyproject/tools-devteam/blob/master/tools/bowtie2/bowtie2_wrapper.xml) +(see below), according to the first action block, if the +```reference_genome.source`` is ``indexed`` (not ``history``), then it will assign +the ``dbkey`` of the output file to be the same as that of the reference file. It +does this by looking at through the data table and finding the entry that has the +value that's been selected in the index dropdown box as column 1 of the loc file +entry and using the dbkey, in column 0 (ignoring comment lines (starting with #) +along the way). + +If ``reference_genome.source`` is ``history``, it pulls the ``dbkey`` from the +supplied file. + +```xml + + analysis_type['analysis_type_selector'] == "simple" or analysis_type['sam_opt'] is False + + + + + + + + + + + + + + +``` + +### Format + +The Bowtie 2 example also demonstrates conditionally setting an output format +based on inputs, as shown below: + +```xml + + ( library['type'] == "paired" or library['type'] == "paired_collection" ) and library['unaligned_file'] is True + + + + + + + + + + + + + +``` + +### Unconditional Actions - An Older Example + +The first approach above to setting ``dbkey`` based on tool data tables is +prefered, but an older example using so called "loc files" directly is found +below. + +In addition to demonstrating this lower-level direct access of .loc files, it +demonstrates an unconditional action. The second block would not be needed for +most cases - it was required in this tool to handle the specific case of a small +reference file used for functional testing. It says that if the dbkey has been +set to ``equCab2chrM`` (which is what the ````` tag does), then it should be changed to ``equCab2`` (which is the +`` + + + + + + + + + + + + + + + + + + + + + Type of action (either ``format`` or +``metadata`` currently). + + + + + If ``type="metadata"``, the name of the +metadata element. + + + + + If ``type="format"``, the default format +if none of the nested options apply. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Name of the input parameter to base +conditional logic on. The value of this parameter will be matched against nested +``when`` directives. + + + + + + + + + + + + + Value to match conditional input value +against. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +This directive should contain one or more ``environment_variable`` definition. + + + + + + + + + + + $inttest + #if int($inttest) == 3 +ISTHREE +#else# +NOTTHREE +#end if# + + +``` + +If these environment variables are used in another Cheetah context, such as in +the ``command`` block, the ``$`` used indicate shell expansion of a variable +should be escaped with a ``\`` so prevent it from being evaluated as a Cheetah +variable instead of shell variable. + +```xml + + echo "\$INTVAR" > $out_file1; + echo "\$IFTEST" >> $out_file1; + +``` + +]]> + + + + + + Name of the environment variable to +define. + + + + + + + + + + `` and ```` tag sets - which can be used +to setup configuration files for use by tools.]]> + + + + + + + + + + + + + + + + `` tag set. It allows for +the creation of a temporary file for file-based parameter transfer. + +*Example* + +The following is taken from the [xy_plot.xml](https://github.com/galaxyproject/tools-devteam/blob/master/tools/xy_plot/xy_plot.xml) +tool config. + +```xml + + + ## Setup R error handling to go to stderr + options( show.error.messages=F, error = function () { cat( geterrmessage(), file=stderr() ); q( "no", 1, F ) } ) + ## Determine range of all series in the plot + xrange = c( NULL, NULL ) + yrange = c( NULL, NULL ) + #for $i, $s in enumerate( $series ) + s${i} = read.table( "${s.input.file_name}" ) + x${i} = s${i}[,${s.xcol}] + y${i} = s${i}[,${s.ycol}] + xrange = range( x${i}, xrange ) + yrange = range( y${i}, yrange ) + #end for + ## Open output PDF file + pdf( "${out_file1}" ) + ## Dummy plot for axis / labels + plot( NULL, type="n", xlim=xrange, ylim=yrange, main="${main}", xlab="${xlab}", ylab="${ylab}" ) + ## Plot each series + #for $i, $s in enumerate( $series ) + #if $s.series_type['type'] == "line" + lines( x${i}, y${i}, lty=${s.series_type.lty}, lwd=${s.series_type.lwd}, col=${s.series_type.col} ) + #elif $s.series_type.type == "points" + points( x${i}, y${i}, pch=${s.series_type.pch}, cex=${s.series_type.cex}, col=${s.series_type.col} ) + #end if + #end for + ## Close the PDF file + devname = dev.off() + + +``` + +This file is then used in the ``command`` block of the tool as follows: + +```xml +bash "$__tool_directory__/r_wrapper.sh" "$script_file" +``` + +]]> + + + + + + Cheetah variable used to reference +the path to the file created with this directive. + + + + + + + + + tag set. It tells Galaxy to +write out a JSON representation of the tool parameters . + +*Example* + +The following will create a cheetah variable that can be evaluated as ``$inputs`` that +will contain the tool parameter inputs. + +```xml + + + +``` + +The following will instead write the inputs to the tool's working directory with +the specified name (i.e. ``inputs.json``). + +```xml + + + +``` + +A contrived example of a tool that uses this is the test tool +[inputs_as_json.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/inputs_as_json.xml). +]]> + + + + + + + + + + + Path relative to the working directory of the tool for the inputs JSON file created in response to this directive. + + + + + + + + + tag set - it contains a set of tags. + ]]> + + + + + + + + + tag set ( used only in "data_source" tools ) - the external data source application may send back parameter names like "GENOME" which must be translated to "dbkey" in Galaxy.]]> + + + + + + + + Each of these maps directly to a remote_name value + + + + + + + The string representing the name of the parameter in the remote data source + + + + + + + The default value to use for galaxy_name if the remote_name parameter is not included in the request + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + tag set if galaxy_name="URL" - some remote data sources ( e.g., Gbrowse, Biomart ) send parameters back to Galaxy in the initial response that must be added to the value of "URL" prior to Galaxy sending the secondary request to the remote data source via URL.]]> + + + + + + + + + + + + + + + + + + + + + + + + tag set - allows for appending a param name / value pair to the value of URL. + +Example: + +```xml + + + + + + + +``` +]]> + + + + + + + + + + + + + + + + tag set the parameter value received from a remote data source may be named differently in Galaxy, and this tag set allows for the value to be appropriately translated.]]> + + + + + + + + + tag set - allows for changing the data type value to something supported by Galaxy. + +Example: + +```xml + + + + + + + +``` +]]> + + + + + + + + + + + + + + + + `` +tag has two subtags, ```` and ````, to define regular +expressions and exit code processing, respectively. They are defined below. If a +tool does not have any valid ```` or ```` tags, then Galaxy +will use the previous technique for finding errors. + +A note should be made on the order in which exit codes and regular expressions +are applied and how the processing stops. Exit code rules are applied before +regular expression rules. The rationale is that exit codes are more clearly +defined and are easier to check computationally, so they are applied first. Exit +code rules are applied in the order in which they appear in the tool's +configuration file, and regular expressions are also applied in the order in +which they appear in the tool's configuration file. However, once a rule is +triggered that causes a fatal error, no further rules are +checked.]]> + + + + + + + + + + + + + + + + + tag defines a range of exit codes, and each range can be associated with a description of the error (e.g., "Out of Memory", "Invalid Sequence File") and an error level. The description just describes the condition and can be anything. The error level is either a warning or a fatal error. A warning means that stderr will be updated with the error's description. A fatal error means that the tool's execution will be marked as having an error and the workflow will stop. Note that, if the error level is not supplied, then a fatal error is assumed to have occurred. + +The exit code's range can be any consecutive group of integers. More advanced ranges, such as noncontiguous ranges, are currently not supported. Ranges can be specified in the form "m:n", where m is the start integer and n is the end integer. If ":n" is specified, then the exit code will be compared against all integers less than or equal to n. If "m:" is used, then the exit code will be compared against all integers greater than or equal to m. If the exit code matches, then the error level is applied and the error's description is added to stderr. If a tool's exit code does not match any of the supplied tags' ranges, then no errors are applied to the tool's execution. + +Note that most Unix and Linux variants only support positive integers 0 to 255 for exit codes. If an exit code falls out of the range 0 to 255, the usual convention is to only use the lower 8 bits for the exit code. The only known exception is if a job is broken into subtasks using the tasks runner and one of those tasks is stopped with a POSIX signal. (Note that signals should be used as a last resort for terminating processes.) In those cases, the task will receive -1 times the signal number. For example, suppose that a job uses the tasks runner and 8 tasks are created for the job. If one of the tasks hangs, then a sysadmin may choose to send the "kill" signal, SIGKILL, to the process. In that case, the task (and its job) will exit with an exit code of -9. More on POSIX signals can be found at http://en.wikipedia.org/wiki/Unix_signal as well as man pages on "signal". + +The tag's supported attributes are as follows: + +* ``range``: This indicates the range of exit codes to check. The range can be one of the following: + * ``n``: the exit code will only be compared to n; + * ``[m:n]``: the exit code must be greater than or equal to m and less than or equal to n; + * ``[m:]``: the exit code must be greater than or equal to m; + * ``[:n]``: the exit code must be less than or equal to n. +* ``level``: This indicates the error level of the exit code. The level can have one of two values: + * ``warning``: If an exit code falls in the given range, then a description of the error will be added to the beginning of stderr. A warning-level error will not cause the tool to fail. + * ``fatal``: If an exit code falls in the given range, then a description of the error will be added to the beginning of stderr. A fatal-level error will cause the tool to fail. If no level is specified, then the fatal error level will be assumed to have occurred. +* ``description``: This is an optional description of the error that corresponds to the exit code. + +The following is an example of the tag: + +```xml + + + + + +``` + +If the tool returns 0 or 1, then the tool will not be marked as having an error. +If the exit code is 2, then the tool will fail with the description ``Out of +Memory`` added to stderr. If the tool returns 3, 4, or 5, then the tool will not +be marked as having failed, but ``Low disk space`` will be added to stderr. +Finally, if the tool returns any number greater than or equal to 6, then the +description ``Bad input dataset`` will be added to stderr and the tool will be +marked as having failed. + +]]> + + + + + + + + + + + + + + + + + + + + + + tag does not contain the match attribute, then the tag will be ignored. The regular expression can be any valid Python regular expression. All regular expressions are performed case insensitively. For example, if match contains the regular expression "actg", then the regular expression will match against "actg", "ACTG", "AcTg", and so on. Also note that, if double quotes (") are to be used in the match attribute, then the value " can be used in place of double quotes. Likewise, if single quotes (') are to be used in the match attribute, then the value ' can be used if necessary. +* ``level``: This works very similarly to the tag, except that, when a regular expression matches against its source, the description is added to the beginning of the source. For example, if stdout matches on a regular expression, then the regular expression's description is added to the beginning of stdout (instead of stderr). The level can be log, warning or fatal as described below. + * ``log`` and ``warning``: If the regular expression matches against its source input (i.e., stdout and/or stderr), then a description of the error will be added to the beginning of the source, prepended with either 'Log:' or 'Warning:'. A log-level/warning-level error will not cause the tool to fail. + * ``fatal``: If the regular expression matches against its source input, then a description of the error will be added to the beginning of the source. A fatal-level error will cause the tool to fail. If no level is specified, then the fatal error level will be assumed to have occurred. +* ``description``: Just like its ``exit_code`` counterpart, this is an optional description of the regular expression that has matched. + +The following is an example of regular expressions that may be used: + +```xml + + + + + + +``` + +The regular expression matching proceeds as follows. First, if either stdout or +stderr match on ``low space``, then a warning is registered. If stdout contained +the string ``---LOW SPACE---``, then stdout has the string ``Warning: Low space +on device`` added to its beginning. The same goes for if stderr had contained the +string ``low space``. Since only a warning could have occurred, the processing +continues. + +Next, the regular expression ``error`` is matched only against stdout. If stdout +contains the string ``error`` regardless of its capitalization, then a fatal +error has occurred and the processing stops. In that case, stdout would be +prepended with the string ``Fatal: Unknown error encountered``. Note that, if +stderr contained ``error``, ``ERROR``, or ``ErRor`` then it would not matter - +stderr was not being scanned. + +If the second regular expression did not match, then the third regular +expression is checked. The third regular expression does not contain an error +level, so an error level of ``fatal`` is assumed. The third regular expression +also does not contain a source, so both stdout and stderr are checked. The third +regular expression looks for 12 consecutive "C"s or "G"s in any order and in +uppercase or lowercase. If stdout contained ``cgccGGCCcGGcG`` or stderr +contained ``CCCCCCgggGGG``, then the regular expression would match, the tool +would be marked with a fatal error, and the stream that contained the +12-nucleotide CG island would be prepended with ``Fatal: Fatal error - CG island +12 nts long found``. + +Finally, if the tool did not match any of the fatal errors, then the fourth +regular expression is checked. Since no source is specified, both stdout and +stderr are checked. If ``Branch A`` is at the beginning of stdout or stderr, then +a warning will be registered and the source that contained ``Branch A`` will be +prepended with the warning ``Warning: Branch A was taken in execution``. + +]]> + + + + This tells whether the regular expression should be matched against stdout, stderr, or both. If this attribute is missing or is incorrect, then both stdout and stderr will be checked. The source can be one of the follwing values: + + + + + This is the regular expression that will be used to match against stdout and/or stderr. + + + + + This works very similarly to the 'exit_code' tag, except that, when a regular expression matches against its source, the description is added to the beginning of the source. + + + + + an optional description of the regular expression that has matched. + + + + + + + `` tag set and is the container tag set +for the following ```` tag set.]]> + + + + + + + + + + + +``` + +Then whenever the user selects the ``interval`` option from the select list, the +following structure in your tool config will override the ``format="fasta"`` setting +in the ```` tag set with ``format="interval"``. + +```xml + + + + + + + +``` + +See +[extract_genomic_dna.xml](https://github.com/galaxyproject/tools-iuc/blob/master/tools/extract_genomic_dna/extract_genomic_dna.xml) +or the test tool +[output_action_change_format.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/output_action_change_format.xml) +for more examples. + +]]> + + + + + This value must be the attribute name of +the desired input parameter (e.g. ``input="out_format"`` above). + + + + + This value must also be an attribute +name of an input parameter (e.g. ``value="interval"`` above). + + + + + This value must be a supported data type +(e.g. ``format="interval"``). See +[/config/datatypes_conf.xml.sample](https://github.com/galaxyproject/galaxy/blob/dev/config/datatypes_conf.xml.sample) +for a list of supported formats. + + + + + + + + + 10.1093/bioinformatics/btq281 + + + @ARTICLE{Kim07aninterior-point, + author = {Seung-jean Kim and Kwangmoo Koh and Michael Lustig and Stephen Boyd and Dimitry Gorinevsky}, + title = {An interior-point method for large-scale l1-regularized logistic regression}, + journal = {Journal of Machine Learning Research}, + year = {2007}, + volume = {8}, + pages = {1519-1555} + } + +``` + +For more implementation information see the +[pull request](https://bitbucket.org/galaxy/galaxy-central/pull-requests/440/initial-bibtex-doi-citation-support-in/diff) +adding this feature. For more examples of how to add this to tools checkout the +following commits adding this to the +[NCBI BLAST+ suite](https://github.com/peterjc/galaxy_blast/commit/9d2e3906915895765ecc3f48421b91fabf2ccd8b), +[phenotype association tools](https://bitbucket.org/galaxy/galaxy-central/commits/39c983151fe328ff5d415f6da81ce5b21a7e18a4), +[MAF suite](https://bitbucket.org/galaxy/galaxy-central/commits/60f63d6d4cb7b73286f3c747e8acaa475e4b6fa8), +and [MACS2 suite](https://github.com/jmchilton/galaxytools/commit/184971dea73e236f11e82b77adb5cab615b8391b). + +This feature was added to the August 2014 release of Galaxy, tools annotated +with citations will work in older releases of Galaxy but no citation information +will be available to the end user. +]]> + + + + + + + + + Each citations element can contain one or +more ``citation`` tag elements - each of which specifies tool citation +information using either a DOI or a BibTeX entry. + + + + + + Type of citation - currently ``doi`` +and ``bibtex`` are the only supported options. + + + + + + + + + Type of citation represented. + + + + + + + + + + Documentation for RequirementType + + + + + + + + + + + Type of container for tool execution. + + + + + + + + Documentation for ToolTypeType + + + + + + + + + Documentation for URLmethodType + + + + + + + + + Documentation for TargetType + + + + + + + + + Documentation for MethodType + + + + + + + + + Documentation for DisplayType + + + + + + + + + Documentation for HierarchyType + + + + + + + + + Documentation for ValidatorType + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Documentation for ActionType + + + + + + + + + Documentation for ActionsOptionType + + + + + + + + + + Documentation for CompareType + + + + + + + + + Documentation for LevelType + + + + + + + + + + Documentation for RangeType + + + + + + + + Documentation for SourceType + + + + + + + + + + Type of comparison to use when comparing +test generated output files to expected output files. Currently valid value are +``diff`` (the default), ``re_match``, ``sim_size``, ``re_match_multiline``, +and ``contains``. + + + + + + + + + + + + Documentation for PermissiveBoolean + + + + + + + + + + + + +