generateDS stops processing child elements

Question

I have an xml schema that describes e-invoice structure. I've used schema with generateDS to create a parser for that format. When parsing an invoice, it seems to work fine, but a section where is any-typed content, it stops processing childs where those elements appear.

Part of the schema that describes any element:

<!-- Elements to describe the invoice extensions -->
    <xs:complexType name="ExtensionRecord">
        <xs:sequence>
            <xs:element name="InformationName" type="NormalTextType" minOccurs="0"/>
            <xs:element name="InformationContent" type="LongTextType"/>
            <xs:element name="CustomContent" minOccurs="0">
                <xs:complexType>
                    <xs:sequence>
                        <xs:any processContents="skip"/>
                    </xs:sequence>
                </xs:complexType>
            </xs:element>
        </xs:sequence>
        <xs:attribute name="extensionId" type="ShortTextType" use="optional"/>
    </xs:complexType>

Relevant part of my implementation that uses the parser:

E_Invoice = einvoice111.parseString(xmlString, silence=True)

for ai in E_Invoice.Invoice.AdditionalInformation:
    print(dir(ai) )
    print(dir(ai.CustomContent))
    print(ai.CustomContent.export(sys.stdout, 0, name_='CustomContent'))

Part of payload XML:

    <AdditionalInformation extensionId="invoicePDFFormat">
      <InformationContent/>
      <CustomContent>
        <any>
          <Content>JVBERi0xLjQ........
                    <BASE64 coded binary>
        .....</Content>

and output of that code:

['CustomContent', 'InformationContent', 'InformationName', 'Tag_strip_pattern_', '_FixedOffsetTZ', '__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'build', 'buildAttributes', 'buildChildren', 'convert_unicode', 'export', 'exportAttributes', 'exportChildren', 'extensionId', 'factory', 'gds_build_any', 'gds_encode', 'gds_format_base64', 'gds_format_boolean', 'gds_format_boolean_list', 'gds_format_date', 'gds_format_datetime', 'gds_format_double', 'gds_format_double_list', 'gds_format_float', 'gds_format_float_list', 'gds_format_integer', 'gds_format_integer_list', 'gds_format_string', 'gds_format_time', 'gds_parse_date', 'gds_parse_datetime', 'gds_parse_time', 'gds_reverse_node_mapping', 'gds_str_lower', 'gds_validate_base64', 'gds_validate_boolean', 'gds_validate_boolean_list', 'gds_validate_date', 'gds_validate_datetime', 'gds_validate_double', 'gds_validate_double_list', 'gds_validate_float', 'gds_validate_float_list', 'gds_validate_integer', 'gds_validate_integer_list', 'gds_validate_simple_patterns', 'gds_validate_string', 'gds_validate_time', 'get_CustomContent', 'get_InformationContent', 'get_InformationName', 'get_class_obj_', 'get_extensionId', 'get_path_', 'get_path_list_', 'hasContent_', 'original_tagname_', 'set_CustomContent', 'set_InformationContent', 'set_InformationName', 'set_extensionId', 'subclass', 'superclass', 'tzoff_pattern', 'validate_LongTextType', 'validate_NormalTextType', 'validate_ShortTextType']
['Tag_strip_pattern_', '_FixedOffsetTZ', '__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'anytypeobjs_', 'build', 'buildAttributes', 'buildChildren', 'convert_unicode', 'export', 'exportAttributes', 'exportChildren', 'factory', 'gds_build_any', 'gds_encode', 'gds_format_base64', 'gds_format_boolean', 'gds_format_boolean_list', 'gds_format_date', 'gds_format_datetime', 'gds_format_double', 'gds_format_double_list', 'gds_format_float', 'gds_format_float_list', 'gds_format_integer', 'gds_format_integer_list', 'gds_format_string', 'gds_format_time', 'gds_parse_date', 'gds_parse_datetime', 'gds_parse_time', 'gds_reverse_node_mapping', 'gds_str_lower', 'gds_validate_base64', 'gds_validate_boolean', 'gds_validate_boolean_list', 'gds_validate_date', 'gds_validate_datetime', 'gds_validate_double', 'gds_validate_double_list', 'gds_validate_float', 'gds_validate_float_list', 'gds_validate_integer', 'gds_validate_integer_list', 'gds_validate_simple_patterns', 'gds_validate_string', 'gds_validate_time', 'get_anytypeobjs_', 'get_class_obj_', 'get_path_', 'get_path_list_', 'hasContent_', 'original_tagname_', 'set_anytypeobjs_', 'subclass', 'superclass', 'tzoff_pattern']
<CustomContent/>

CustomContent has omittag, showing that object structure ends there. I've also tried to export() whole document and it has the same situation.

So it's the part:

    <xs:complexType>
      <xs:sequence>
        <xs:any processContents="skip"/>
      </xs:sequence>
    </xs:complexType>

that does not appear in Python object-tree.

When I look into generated library that matches the schema, relevant part of the CustomContent class:

    def buildChildren(self, child_, node, nodeName_, fromsubclass_=False):
        obj_ = self.gds_build_any(child_, 'CustomContentType')
        if obj_ is not None:
            self.set_anytypeobjs_(obj_)

It uses gds_build_any() method, instead of creating a new instance of class that it would have generated from schema (that doesn't exist either).

When using Suds, I can access the any element and its content, but it breaks elsewhere.

Is there a way to configure generateDS so that it would:

• create classes to all schema's elements?
• use those classes to whole document?

Answer 1

Okay, tried all kind of approaches and came up with changes so that I can read the any content (not sure did I know what I was doing):

$ hg diff generateDS.py | wc -l
78

Only problem that my XML has base64 coded text inside . It looks like:

    <AdditionalInformation extensionId="invoicePDFFormat">
      <InformationContent/>
      <CustomContent>
        <any>
          <Content>JVBERi0xLjQNCiXi48/TDQoxIDAgb2JqDQo8PC9UeXBlIC9Gb250IC9TdWJ0eXBlIC9UeXBlMQ0K
L0VuY29kaW5nIC9XaW5BbnNpRW5jb2RpbmcgL0Jhc2VGb250IC9Db3VyaWVyID4+DQplbmRvYmoN
CjIgMCBvYmoNCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4NCnN0cmVhbQ0K
eJwz0DNVMIDionQFpxAuAwVDBV1DBQMFUwUTAwOFkFwu/WAPUyAvJA0oF1IMlAkpAhHJIKKcS8NV

but after GDS-processing, it looks like:

          JVBERi0xLjQNCiXi48/TDQoxIDAgb2JqDQo8PC9UeXBlIC9Gb250IC9TdWJ0eXBlIC9UeXBlMQ0K

L0VuY29kaW5nIC9XaW5BbnNpRW5jb2RpbmcgL0Jhc2VGb250IC9Db3VyaWVyID4+DQplbmRvYmoN

CjIgMCBvYmoNCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4NCnN0cmVhbQ0K

I get the text content inside the element with:

self.any = ''.join(node.itertext())

and it doesn't matter even I would use just node.text, there are empty lines inside it. That's still unsolved mystery.

Answer 2

Well, at least it appears that generateDS.py SAX-handler XschemaHandler at line 1652 tests:

    if name == AnyType:
        element = XschemaElement(attrs)
        element.type = AnyTypeIdentifier
        self.stack.append(element)

and that maches to tag (AnyType contains 'xs:any'). So GDS knows that any tag is special XML-grammar like 'sequence', 'simpleType', 'complexType' etc.

This is probably the reason why it does not try to create a class for it and map it to 'any'.

I modified it a bit:

    elif name == AnyType:
        print('opening AnyType')
        element = XschemaElement(attrs)
        element.name = 'any'
        #element.type = 'NormalTextType'
        element.type = 'StringType'
        self.inAnyType = 1
        self.stack.append(element)

and made similar to sax closing tag, and added it to tag content prosessing part:

    def characters(self, chrs):
        if self.inDocumentationType:
            # If there is an annotation/documentation element, save it.
            if len(self.stack) > 1 and len(chrs) > 0:
                self.stack[-1].documentation += chrs
        elif self.inAnyType:
            if len(self.stack) > 1 and len(chrs) > 0:
                self.stack[-1].any += chrs
        elif self.inElement:
            pass

Now i get output as:

    <CustomContent>
        <any>

              </any>
    </CustomContent>

Close but no cigar.