Skip to content

Merge Galaxy-XSD into project#2923

Merged
martenson merged 110 commits intogalaxyproject:devfrom
jmchilton:merge_xsd_into_core
Sep 16, 2016
Merged

Merge Galaxy-XSD into project#2923
martenson merged 110 commits intogalaxyproject:devfrom
jmchilton:merge_xsd_into_core

Conversation

@jmchilton
Copy link
Member

Transplanted with commit authors maintained.

The most obvious advantages of it living here instead of its own project is that it gives it higher profile and it allows the version of the schema to match that of Galaxy (so we can check at least if the XML structure of a tool is valid for a given release).

This has a few more advantages being in core that will probably come from follow up commits if this makes it in:

  • Allow docs for tool XML generated from this to live in this project.
  • Allow distribution with galaxy-lib.
  • Allow checking testing tools as part of CI, ensures all new tool features are added to XSD and documented.

A Markdown document generated from this XSD is available for preview at https://github.com/jmchilton/planemo/blob/xsd_and_docs/docs/schema.md#tool%7Ctests%7Ctest%7Crepeat. I think in a subsequent commit I'll move those docs here - but for now it is a good way to "visualize" the XSD.

jmchilton and others added 30 commits September 15, 2016 00:08
Element order is not enforced by Galaxy.
This includes:
-<requirements>
-<requirement>
-Enumeration RequirementType
Add attributes min and max to Repeat element
The <param> tag can contain <options> to dynamically create a list of options.
Add seven missing attributes to <param> tag
Add the necessary datatypes for display, hierarchy and checked.
Add three missing attributes to <data> tag
Add <filter> element to sequence
Add the <parallelism> tag (which serves for splitting job)
and the datatype for the 'method' attribute.
Add <stdio> tag set, to handle tool messages on standard output
and standard error.

- Add <stdio> tag
- Add <regex> and <exit_code> sub elements
- Add datatypes for 'range', 'level' and 'source' attributes
Version information can be too complex for xs:decimal
(eg "0.12.7"). Change for xs:string.
According to the documentation, CheckedType can either
be 'yes', 'true' or 'on' ; but many tools use '[Ff]alse'
which seems to be accepted/ignored by Galaxy.
<xs:boolean> can either be 'true', 'false', '1' or 'O' ; but many
Galaxy tools use 'True' and 'False' (capitalized), which appears
to be ignored/accepted by Galaxy.
This PermissiveBoolean type is meant to be used in place of xs:boolean
when needed.
These are undocumented attributes.
Add attributes 'compare', 'keep' and 'value' to <filter> tag
(though these attributes are not documented).
Add CompareType for possible 'compare' values.
@martenson
Copy link
Member

+1 on the principle of shipping xsd with Galaxy

On Thu, Sep 15, 2016, 00:23 John Chilton notifications@github.com wrote:

Transplanted with commit authors maintained.

The most obvious advantages of it living here instead of its own project
is that it gives it higher profile and it allows the version of the schema
to match that of Galaxy (so we can check at least if the XML structure of a
tool is valid for a given release).

This has a few more advantages being in core that will probably come from
follow up commits if this makes it in:

  • Allow docs for tool XML generated from this to live in this project.
  • Allow distribution with galaxy-lib.
  • Allow checking testing tools as part of CI, ensures all new tool
    features are added to XSD and documented.

A Markdown document generated from this XSD is available for preview at
https://github.com/jmchilton/planemo/blob/xsd_and_docs/docs/schema.md#tool%7Ctests%7Ctest%7Crepeat.
I think in a subsequent commit I'll move those docs here - but for now it

is a good way to "visualize" the XSD.

You can view, comment on, or merge this pull request online at:

#2923
Commit Summary

  • Deleting directories from SVN that should have been ignored.
  • Work on galaxy support, implemented repeat blocks in the UI.
  • Add basic ReadMe
  • Remove custom namespaces
  • Change tool element to xs:all
  • Add tag set
  • Complete tag set
  • Add tag to
  • Add tag to
  • Add missing attributes to tag
  • Add missing attributes and element to tag
  • Fix indentation
  • Add tag
  • Add tag set to
  • Change type for requirement version
  • Add column, filter and validator tags
  • Add ftype attribute to param tag in tests
  • Add tag set
  • Remove required use from version
  • Edit tag set
  • Add to
  • Add 'split' attribute to
  • Add tag set
  • Extend CheckedType to accept unspecified values
  • Add PermissiveBoolean datatype
  • Add attributes 'ftype' and 'sort' to test
  • Add (undocumented) attributes to
  • Edit filter tag for output tag
  • Rewrite handling
  • Add (undocumented) 'dbkey' attribute to TestParam
  • Add 'from_file' value to ActionsOptionType
  • Use PermissiveBoolean for ParamOption 'selected'
  • Add attributes tool_type and URL_method to
  • use PermissiveBoolean on two attributes of
  • Add (undocumented) value 'no' to CheckedType
  • Add (undocumented) to
  • Add (undocumented) attributes to
  • Add tag
  • Add (undocumented) attribute metadata_name to
  • Add (undocumented) attributes to
  • Add (undocumented) 'ref_attribute' to
  • Add (undocumented) 'startswith' to
  • Add (undocumented) attribute 'display' to
  • Fix value baseurl in 'type' attribute
  • Fix type of min and max in
  • added doc and JXB bindings
  • updated README
  • Add Continuous integration with Travis for XSD linting
  • Fix dependency installation for Travis
  • Add Travis badge to ReadMe
  • Add me (@jmchilton) to authors list.
  • Fix CitationType attribute.
  • Add Shell test script
  • Update XSD for container tags.
  • Update XSD for very basic data collection usage.
  • Drop embeddedValue for test stuff.
  • Start work on implementing assert_contents tag in tests.
  • Add MIT license
  • Expand history in ReadMe
  • Fix authors in License file
  • first try
  • Improvements to XSD:
  • More progress...
  • Simplify schema and improve docs related to sanitization.
  • Improve docs on filter and change_format of outputs.
  • And more progress...
  • More progress...
  • More progress.
  • More progress...
  • More progress...
  • More progress...
  • Stub out environment variables and add a new tool type.
  • Embed citations right in galaxy.xsd.
  • Many more fixes derived from testing tools-iuc.
  • And more progress - now covers all Galaxy test tools and all IUC.
  • Doc updates
  • Doc improvements.
  • Much richer documentation for a few tags including command.
  • More work on documentation - revise last commit and add repeats and
    sections.
  • Work on docs.
  • More docs...
  • More docs for validators.
  • A lot more documentation around parameter options.
  • More documentation...
  • Mention code files as a way to generate dynamic options.
  • Flush out docs for requirements and child tags.
  • Flush out docs for sanitizers, configfiles, etc...
  • Working my way through basic parameter documentation.
  • More work on parameters and environment variables.
  • Finishing first pass at filling out wiki docs for param.
  • Starting to polish and flush out data outputs documentation.
  • More docs - conditional output actions and some more on discover
    datasets.
  • A bunch of work on cleaning up and flushing out documentation for
    test tags.
  • Document the test output_collection tag.
  • Fix missing annotation.
  • Revise automated testing, check against test tools and IUC tools.
  • Fixes of structure broken with docs.
  • Use the new action conditional filter.
  • Add a fictious XML schema with Galaxy stuff.
  • Remove embedded best practice stuff - the help will have links now.
  • Fill out some docs for test sections...
  • Document testing discovered datasets and metadata.
  • Fix a syntax error.
  • Update the link for multiple outputs.
  • Relocate Galaxy XSD files into Galaxy directory structure.
  • Cleanup XSD for move to Galaxy core.
  • Merge branch 'master' of Galaxy-XSD into 'dev'

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2923, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABuxqmsz9Joq-WOkCZhDcFzt-tSEfvx7ks5qqMhOgaJpZM4J9fDt
.

@bgruening
Copy link
Member

👍

@mvdbeek
Copy link
Member

mvdbeek commented Sep 15, 2016

I think present and future tool authors will be very happy with this. 👍

cd galaxy
git pull
cd ..

Copy link
Member

@nsoranzo nsoranzo Sep 15, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that lines 2-9 can be removed.

git pull
cd ..

count=`wc -l tool_files.list | cut -f1 -d' '`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no tool_files.list file.

- Merge in @bgruening's numerous awesome fixes.
- Remove test script with issues that make if not a fit for this repo in that form (noted by @nsoranzo).
- Incorporate other changes to the XSD suggested by @nsoranzo.
@jmchilton
Copy link
Member Author

Pushed babaf7a to address reviewer comments on XSD.

  • Merged in @bgruening's numerous awesome fixes from Galaxy-XSD downstream.
  • Removed test script with issues that make if not a fit for this repo in that form (noted by @nsoranzo).
  • Incorporate other changes to the XSD suggested by @nsoranzo.

@@ -0,0 +1,22 @@
The MIT License (MIT)

Copyright (c) 2011-2015 John Chilton, Jean-Frédéric Berthelot, Pierre Lindenbaum
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2016

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@bgruening
Copy link
Member

Thanks @jmchilton! Ready for merge?

@jmchilton
Copy link
Member Author

I consider it ready for merging. I'll open a new PR regarding the LICENSE (let's see if we can drop it as a special license inside of Galaxy) - though I did update the year.

@martenson
Copy link
Member

I would leave it open for a bit - it deserves to be open for more than 14 hours I believe.

@jmchilton
Copy link
Member Author

I'm pretty sure we can relocate this here and keep the LICENSE statement as it is, but it would be cleaner if we re-licensed it under Galaxy's license.

@JeanFred, @lindenb as co-copyright holders I was hoping to get your explicit permission to re-license this under the terms of Galaxy's license. It is a permissive (not copy-left) license very much in the spirit of the original MIT license I believe. Regardless, this PR adds both of your names to Galaxy's CONTRIBUTORS list.

xref galaxyproject/Galaxy-XSD#2

@martenson martenson merged commit f12723d into galaxyproject:dev Sep 16, 2016
@JeanFred
Copy link
Contributor

Wow, I’m really, really happy to see this integrated into core Galaxy!

@JeanFred, @lindenb as co-copyright holders I was hoping to get your explicit permission to re-license this under the terms of Galaxy's license. It is a permissive (not copy-left) license very much in the spirit of the original MIT license I believe. Regardless, this PR adds both of your names to Galaxy's CONTRIBUTORS list.

Sure, I am fine with re-licensing or dual-licensing my contributions to make it easier to manage for the Galaxy project. I was not very familiar with AFL but it looks good to me. To make it official sounding:

I hereby agree to license my contributions under the Academic Free License version 3.0

Thanks everyone!

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Sep 16, 2016
- Add CI script that lints the XSD definition and then validates all test tools against it.
- Add tox environment to run the aforementioned script.
- Add Travis check through tox to run the script.

This will hopefully ensure that all new tool features get added to the schema (as long as there is a test tool for the feature). xref galaxyproject#2923 galaxyproject#2903

Largely based on work done in Galaxy-XSD by JeanFred (e.g. galaxyproject/Galaxy-XSD@386ba08)
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Sep 16, 2016
- Add CI script that lints the XSD definition and then validates all test tools against it.
- Add tox environment to run the aforementioned script.
- Add Travis check through tox to run the script.
- Drop the multi_page.xml test tool - pages have been removed from the framework and are not part of the schema (xref galaxyproject#1776).

This will hopefully ensure that all new tool features get added to the schema (as long as there is a test tool for the feature). xref galaxyproject#2923 galaxyproject#2903

Largely based on work done in Galaxy-XSD by JeanFred (e.g. galaxyproject/Galaxy-XSD@386ba08)
@nsoranzo nsoranzo deleted the merge_xsd_into_core branch February 16, 2021 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants