Xmlschema rarely incorrectly matches substitution head element for validation instead of existing global element definition. #461
Description
Hi,
Sorry in advance for not being able to whittle this down into a more compact test form, I tried a lot, but every time I removed things or tried to write a simpler version, it would just work. The bug is extremely specific to some circumstances as you will see.
If you run
from xmlschema import XMLResource, XMLSchema10
schema = XMLSchema10("http://www.xbrl.org/2014/table.xsd")
resource = XMLResource("test_xml.xml")
schema.validate(resource)
with test_xml.xml as (which is a much stripped down version of the actual file that was failing. I wanted to attach it but github does not allow attaching .xml files).
<link:linkbase xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:link="http://www.xbrl.org/2003/linkbase" xmlns:gen="http://xbrl.org/2008/generic" xmlns:label="http://xbrl.org/2008/label" xmlns:formula="http://xbrl.org/2008/formula" xmlns:df="http://xbrl.org/2008/filter/dimension" xmlns:table="http://xbrl.org/2014/table">
<gen:link xlink:type="extended" xlink:role="http://www.eba.europa.eu/xbrl/crr/role/fws/ESG/its-2021-637/2023-05-30/tab/D_10.00">
<df:explicitDimension xlink:type="resource" xlink:label="eba_a3.root.filter" id="eba_a3.root.filter">
<df:dimension>
<df:qname>eba_dim:CTN</df:qname>
</df:dimension>
<df:member>
<df:qname>eba_exp:CT</df:qname>
<df:linkrole>http://www.eba.europa.eu/xbrl/crr/role/dict/dom/CT/CT54</df:linkrole>
<df:arcrole>http://xbrl.org/int/dim/arcrole/domain-member</df:arcrole>
<df:axis>descendant</df:axis>
</df:member>
</df:explicitDimension>
<table:aspectNodeFilterArc xlink:type="arc" xlink:arcrole="http://xbrl.org/arcrole/2014/aspect-node-filter" xlink:from="eba_a3.root" xlink:to="eba_a3.root.filter" complement="false" />
</gen:link>
</link:linkbase>
you get the following error: xmlschema.validators.exceptions.XMLSchemaValidationError: failed validating ('complement', 'false') with XsdAnyAttribute(namespace=['##other'], process_contents='lax')
If you debug/add some print statements, you see that this is because the XsdElement that the xml element is being validated against is not {http://xbrl.org/2014/table}aspectNodeFilterArc but rather {http://xbrl.org/2008/generic}arc, the head of its substitution group. However if you add a print(schema.maps.elements.get("{http://xbrl.org/2014/table}aspectNodeFilterArc")) you can clearly see that the global element definition {http://xbrl.org/2014/table}aspectNodeFilterArc is present.
The schema definition of {http://xbrl.org/2014/table}aspectNodeFilterArc is
<element id="xml-aspect-node-filter-arc" name="aspectNodeFilterArc" substitutionGroup="gen:arc">
<complexType>
<complexContent>
<extension base="gen:genericArcType">
<attribute default="false" name="complement" type="boolean" use="optional"/>
</extension>
</complexContent>
</complexType>
</element>
as can be seen in https://www.xbrl.org/2014/table.xsd
Additionally, this mismatch only happens under very specific circumstances. If you are wondering why I left in the df:explicitDimension element in the test xml, it's because it actually correctly validates if you remove that element, for some reason. On top of that, this file is one of about 20 files with table:aspectNodeFilterArc elements in this set of files I am validating, and only this one triggers this error. I tried to look for patterns or differences that explains why this one fails but I really haven't been able to find anything.
Specifically, if you add
if child.tag == "{http://xbrl.org/2014/table}aspectNodeFilterArc":
print(child)
print(xsd_element)
print(xsd_element.type)
to xmlschema.validators.groups.py at line 1035, within XsdGroup.raw_decode(), you get the print
<Element '{http://xbrl.org/2014/table}aspectNodeFilterArc' at 0x7f191bb36250>
XsdElement(ref='gen:arc', occurs=[1, 1])
XsdComplexType(name='gen:genericArcType')
and if you remove the df:explicitDimension element from the test xml, you instead get
<Element '{http://xbrl.org/2014/table}aspectNodeFilterArc' at 0x7f83ea966110>
XsdElement(name='table:aspectNodeFilterArc', occurs=[1, 1])
XsdComplexType(content='element-only', attributes=['complement', 'id', 'order', 'priority', 'use', 'xlink:actuate', 'xlink:arcrole', 'xlink:from', 'xlink:show', 'xlink:title', 'xlink:to', 'xlink:type', XsdAnyAttribute(namespace=['##other'], process_contents='lax')])
The reason it seems not to match in the first case is because XsdElement(name='table:aspectNodeFilterArc', occurs=[1, 1]) isnt yielded by self.iter_elements(): in line 922 match_element, but this is where I wasn't able to go further in understanding what exactly is going on, seems to have something to do with the construction of XsdGroups.
Thanks in advance,
Lorenzo