Silence warnings by default when reading in VO Tables #8028
Description
TL;DR
Users often are given files they don't have control over, and those files aren't always standard-compliant. This is especially true of VO Tables. I'd like to suggest that we make the VO Table reader more forgiving, although the writer should continue to emit warnings. Obviously we should discuss this first before doing, but I just want to put the proposal out there.
Details
Taking the example of VO Tables, the following is an example of reading in one of the files in our test suite (which wasn't there to test warnings):
In [5]: parse('gemini.xml')
WARNING: W49: gemini.xml:37:12: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]
WARNING: W49: gemini.xml:49:12: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]
WARNING: W49: gemini.xml:61:12: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]
WARNING: W48: gemini.xml:78:10: W48: Unknown attribute 'value' on OPTION [astropy.io.votable.tree]
WARNING: W48: gemini.xml:79:10: W48: Unknown attribute 'value' on OPTION [astropy.io.votable.tree]
WARNING: W06: gemini.xml:98:6: W06: Invalid UCD 'obs.field': Secondary word 'obs.field' is not valid as a primary word [astropy.io.votable.tree]
WARNING: W06: gemini.xml:99:6: W06: Invalid UCD 'obs.field': Secondary word 'obs.field' is not valid as a primary word [astropy.io.votable.tree]
WARNING: E02: gemini.xml:99:6: E02: Incorrect number of elements in array. Expected multiple of 3, got 1 [astropy.io.votable.converters]
WARNING: W06: gemini.xml:100:6: W06: Invalid UCD 'obs.field': Secondary word 'obs.field' is not valid as a primary word [astropy.io.votable.tree]
WARNING: W06: gemini.xml:101:6: W06: Invalid UCD 'em.wl;stat.interval': Unknown word 'stat.interval' [astropy.io.votable.tree]
WARNING: E02: gemini.xml:101:6: E02: Incorrect number of elements in array. Expected multiple of 2, got 1 [astropy.io.votable.converters]
WARNING: W06: gemini.xml:102:6: W06: Invalid UCD 'time;stat.interval': Unknown word 'stat.interval' [astropy.io.votable.tree]
WARNING: E02: gemini.xml:102:6: E02: Incorrect number of elements in array. Expected multiple of 2, got 1 [astropy.io.votable.converters]
WARNING: W06: gemini.xml:112:6: W06: Invalid UCD 'obs.field': Secondary word 'obs.field' is not valid as a primary word [astropy.io.votable.tree]
WARNING: W06: gemini.xml:113:6: W06: Invalid UCD 'obs.field': Secondary word 'obs.field' is not valid as a primary word [astropy.io.votable.tree]
WARNING: E02: gemini.xml:113:6: E02: Incorrect number of elements in array. Expected multiple of 3, got 1 [astropy.io.votable.converters]
WARNING: W06: gemini.xml:114:6: W06: Invalid UCD 'obs.field': Secondary word 'obs.field' is not valid as a primary word [astropy.io.votable.tree]
WARNING: W06: gemini.xml:115:6: W06: Invalid UCD 'em.wl;stat.interval': Unknown word 'stat.interval' [astropy.io.votable.tree]
WARNING: E02: gemini.xml:115:6: E02: Incorrect number of elements in array. Expected multiple of 2, got 1 [astropy.io.votable.converters]
WARNING: W06: gemini.xml:116:6: W06: Invalid UCD 'time;stat.interval': Unknown word 'stat.interval' (suppressing further warnings of this type...) [astropy.io.votable.tree]
WARNING: E02: gemini.xml:116:6: E02: Incorrect number of elements in array. Expected multiple of 2, got 1 [astropy.io.votable.converters]
WARNING: E02: gemini.xml:127:6: E02: Incorrect number of elements in array. Expected multiple of 3, got 1 [astropy.io.votable.converters]
WARNING: E02: gemini.xml:137:6: E02: Incorrect number of elements in array. Expected multiple of 2, got 1 [astropy.io.votable.converters]
WARNING: E02: gemini.xml:151:6: E02: Incorrect number of elements in array. Expected multiple of 3, got 1 [astropy.io.votable.converters]
WARNING: E02: gemini.xml:161:6: E02: Incorrect number of elements in array. Expected multiple of 2, got 1 (suppressing further warnings of this type...) [astropy.io.votable.converters]
Out[5]: <VOTABLE>... 1 tables ...</VOTABLE>
This is a pretty typical number of warnings in my experience with VO Tables. I've never done anything about any of the warnings though...
Note that there is actually a way to be even more pedantic:
In [6]: parse('gemini.xml', pedantic=True)
---------------------------------------------------------------------------
W49 Traceback (most recent call last)
<ipython-input-6-70047e7af5ca> in <module>()
----> 1 parse('gemini.xml', pedantic=True)
~/Dropbox/Code/Astropy/astropy/astropy/io/votable/table.py in parse(source, columns, invalid, pedantic, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)
135 _debug_python_based_parser=_debug_python_based_parser) as iterator:
136 return tree.VOTableFile(
--> 137 config=config, pos=(1, 1)).parse(iterator, config)
138
139
...
~/Dropbox/Code/Astropy/astropy/astropy/io/votable/exceptions.py in vo_raise(exception_class, args, config, pos)
96 if config is None:
97 config = {}
---> 98 raise exception_class(args, config, pos)
99
100
W49: gemini.xml:37:12: W49: Empty cell illegal for integer fields.
But actually no way to be less pedantic and ignore the warnings (short of using warnings.catch_warnigns. I'd like to suggest that we add a verify key to the VO Table parse which can take different options as for FITS, including ignore, warn, exception (and possibly deprecate pendantic).
Furthermore, I think we might want to consider defaulting to 'ignore'.
We could also do something similar with FITS files - ignore warnings when reading but show them when writing?