CODEWITHSUNDEEP
pythonxmllxmlsignxml-signature
Simon Capriles
Simon Capriles

Reputation: 183

Remove "ds:" in signed XML tree namespace in python's signxml

I am working with Python's lxml and signxml to generate an xml file and sign it with a pem certificate and private key.

I am required to validate the signed xml in the followign website validate XML. For some reason in this website the signed XML files with the "ds" namespace in signature tags do not recognize the file as signed.

I will not focus much on the generated xml file with lxml. The code to sign the xml file has the following form:

    def _get_xml_tree_root(self):
        root = ET.Element('facturaElectronicaCompraVenta' , attrib={location_attribute: invoice_sector + '.xsd'})
        xml_header = ET.SubElement(root, 'header')
        xml_detail = ET.SubElement(root, 'detail')
      return root

    def _get_signed_xml(self):
        signed_root = XMLSigner().sign(
            self._get_xml_tree_root(),
            key=base64.b64decode(io.TextIOWrapper(BytesIO(electronic_key)).read()),
            cert=base64.b64decode(io.TextIOWrapper(BytesIO(electronic_certificate)).read())
        )
        return signed_root

The problem is that the xml file that I generate in the signature section has following form:

<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header></header>
   <detail></detail>
    <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
        <ds:SignedInfo>
            <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
            <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
            <ds:Reference URI="">
                <ds:Transforms>
                    <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                    <ds:Transform Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
                </ds:Transforms>
                <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
                <ds:DigestValue>KvIMPxajMb98G3+HdSLg1/pgSyisLp4OWZt6Gxhe+/c=</ds:DigestValue>
            </ds:Reference>
        </ds:SignedInfo>
        <ds:SignatureValue>Bv9W9cGyXvX4QeDDb61YME8TbnFlBOVBw2Iiv+a+7VrxjoH4z8kLO4rgonXbqGuk2DKrR4ACqoFQNd/9/lJb31TDk2SjegURBsjP9gLvFWwfq99jh6zn6rPF/gwqd+lA1ruGpDT/Q+vxMXeNpXfk+nDcgdDJoP1bpDEPHbSHGkQu2SX1NQP1SlRZkNoJXxorFfbTDmm1/VFRsv5uBNQvf7hSxTEvvLW8WVYN271iTzHTpAnbyg7VTeys/Ca2FQsZ95hgCHfKsOHEX2/HtxpkGtXDjJKPHq43M2MR3Bp9+YUBAxcj5WMsGcs0lp7hFP6xADEJAcLdfta3SJCdNTa0Vw==</ds:SignatureValue>
        <ds:KeyInfo>
            <ds:X509Data>
                <ds:X509Certificate>
                   CertificateStuff...
                </ds:X509Certificate>
            </ds:X509Data>
        </ds:KeyInfo>
    </ds:Signature>
</facturaElectronicaCompraVenta>

I need to sign the xml file without the "ds" namespace like the following:

<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header></header>
   <detail></detail>
   <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
       <SignedInfo>
           <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
           <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
           <Reference URI="">
               <Transforms>
                   <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                   <Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/>
               </Transforms>
               <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
            <DigestValue>WmFvnKBZIr9D37PaYuxM3aoXVu9nDZT+2MI1I+RUh8s=  </DigestValue>
       </Reference>
       </SignedInfo>
       <SignatureValue> itb123fGGhh12DpFDFas34ASDAPpSSSSadasDasAS1smkRsj5ksdjasd8asdkasjd8asdkas8asdk21v a1qf+kBKLwF39mj+5zKo1qf+kBKLD42qD/+yxSMMS6DM5SywPxO1oyjnSZtObIe/45fdS4sE9+aNOn UncYUlSDAPpSSSSadasgIMWwlX2XMJ4SDAPpSSSSadas6qihJt/3dEIdta1RETSDAPpSSSSadas9S2W ALbT3VV8pjLqikVLcSDAPpSSSSadaseCKG8abcdssM0Wm8p+5grNNpSDAPpSSSSadasy4TvT4C3xS 70zSbKWeBUUglRcU8FECEcacu+UJaBCgRW0S3Q== </SignatureValue>
       <KeyInfo>
           <X509Data>
               <X509Certificate>
                  CertificateStuff..
               </X509Certificate>
           </X509Data>
       </KeyInfo>
   </Signature>
</facturaElectronicaCompraVenta>

I not sure why the site do not recognize the signature with the "ds:" namespace. I have previously struggled with xml namespaces and I do not understand them very well.

But, how could I sign the XML file without the "ds:" namespace without changing the signxml library source code?

Upvotes: 3

Views: 1492

Answers (3)

Alan Souza
Alan Souza

Reputation: 21

You can do something like this:

signer = XMLSigner()
ns = {None: signer.namespaces['ds']}
signer.namespaces = ns

ref: https://github.com/XML-Security/signxml/issues/171

Upvotes: 2

Frank
Frank

Reputation: 1004

If you use signxml the following way (slightly modified from your example - specifically lines 1 and 2 of the method get_signed_xml(...)):

from lxml import etree
from signxml import XMLSigner
import sys

def get_xml_tree_root():
        root = etree.Element('facturaElectronicaCompraVenta')
        xml_header = etree.SubElement(root, 'header')
        xml_detail = etree.SubElement(root, 'detail')
        return root

def get_signed_xml(root):
        signature = etree.SubElement(root,'{http://www.w3.org/2000/09/xmldsig#}Signature',Id='placeholder',nsmap={ None: 'http://www.w3.org/2000/09/xmldsig#' })
        signed_root = XMLSigner(c14n_algorithm='http://www.w3.org/2001/10/xml-exc-c14n#').sign(
            root,
            key=open('example.key').read(),
            cert=open('example.pem').read()
        )
        return signed_root

if __name__ == '__main__':
    root = get_xml_tree_root()
    signed_root = get_signed_xml(root)
    signed_xml_file = open('signed.xml','wb')
    signed_xml_file.write(etree.tostring(signed_root,encoding='UTF-8'))
    signed_xml_file.close()
    print(etree.tostring(signed_root,encoding='UTF-8',pretty_print=True).decode('UTF-8'))

you get the following output:

<facturaElectronicaCompraVenta>
  <header/>
  <detail/>
  <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
    <ds:SignedInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
      <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
      <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
      <ds:Reference URI="">
        <ds:Transforms>
          <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
          <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
        </ds:Transforms>
        <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
        <ds:DigestValue>hdNz66gEKYxWCR0+FfES7...</ds:DigestValue>
      </ds:Reference>
    </ds:SignedInfo>
    <SignatureValue>bSsNZJX4...</SignatureValue>
    <KeyInfo>
      <X509Data>
        <X509Certificate>MIIC/zCCAeegAwIBA...</X509Certificate>
      </X509Data>
    </KeyInfo>
  </Signature>
</facturaElectronicaCompraVenta>

which has the ds: prefix removed from everything but the SignedInfo element and it's children. Which to me actually looks like a bug in XMLSign().

Thus I am pretty sure, that signxml is not able to give you right now what you are looking for. Unfortunately I have no idea what you could do otherwise...

Edit: I just recognised, that the verification-site you linked to does some kind of caching so that it actually verified an old version of the file I created with earlier code than the above.

If you run the output of the code I posted above through that validation on that site, it actually recognises the xml document to be signed. It looks like the ds: prefix on the SignedInfo element and it's children doesn't matter.

The validation still fails, but it looks like it is only because I used a self-signed certificate. The verification result actually is:

Documento : signed.xml
 x test - Firma no válida
     v Documento auténtico
     x Cadena de confianza
     v Firmado en el periodo de vigencia
     v Firmado con certificado no revocado

So, maybe signxml can actually do what you want.

Upvotes: 2

hellohawaii
hellohawaii

Reputation: 3074

Solution 1:

I came up with a function that changes namespace ns_from to ns_to. Removing a namespace can just be realized by setting ns_to to "".

(However, this solution seems to be problematic according the comments by the OP)

def replace_namespace(root, ns_from, ns_to):
    """
    Merge the namespace ns_from to ns_to in the tree rooted at the root, everything that belongs to ns_from will be in
    ns_to
    To do so, I
    1. change the tag of root
        Change the namespace of root from ns_from to ns_to
    2. change the attribute
        Change the namespace of root from ns_from to ns_to
    3. change the nsmap
        delete the ns_from space
    4. keep other property
    """
    # change the tag of root
    tag = etree.QName(root.tag)
    # if there are attributes belong to namespace ns_from, update to namespace of the
    # attributes to namespace ns_to
    if tag.namespace == ns_from:
        root.tag = '{%s}%s' % (ns_to, tag.localname)

    # change the attribute of root
    # if there are attributes belong to namespace ns_from, update to namespace of the
    # attributes to namespace ns_to
    root_attrib_dict = dict(root.attrib)
    new_attrib_dict = {}
    for key, value in root_attrib_dict.items():
        key_QName = etree.QName(key)
        if key_QName.namespace == ns_from:
            new_key = '{%s}%s' % (ns_to, key_QName.localname)
            new_attrib_dict[new_key] = value
        else:
            new_attrib_dict[key] = value

    # set the new nsmap
    new_nsmap = root.nsmap.copy()
    for ns_key, ns_value in root.nsmap.items():
        if ns_value == ns_from:
            del new_nsmap[ns_key]

    # make the updated root
    new_root = etree.Element(root.tag, attrib=new_attrib_dict, nsmap=new_nsmap)

    # copy other properties
    new_root.text = root.text
    new_root.tail = root.tail

    # call recursively
    for old_root in root[:]:
        new_root.append(replace_namespace(old_root, ns_from, ns_to))

    return new_root

Test Codes:

input_xml_string = """
<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header></header>
   <detail></detail>
    <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
        <ds:SignedInfo>
            <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
            <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
            <ds:Reference URI="">
                <ds:Transforms>
                    <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                    <ds:Transform Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
                </ds:Transforms>
                <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
                <ds:DigestValue>KvIMPxajMb98G3+HdSLg1/pgSyisLp4OWZt6Gxhe+/c=</ds:DigestValue>
            </ds:Reference>
        </ds:SignedInfo>
        <ds:SignatureValue>Bv9W9cGyXvX4QeDDb61YME8TbnFlBOVBw2Iiv+a+7VrxjoH4z8kLO4rgonXbqGuk2DKrR4ACqoFQNd/9/lJb31TDk2SjegURBsjP9gLvFWwfq99jh6zn6rPF/gwqd+lA1ruGpDT/Q+vxMXeNpXfk+nDcgdDJoP1bpDEPHbSHGkQu2SX1NQP1SlRZkNoJXxorFfbTDmm1/VFRsv5uBNQvf7hSxTEvvLW8WVYN271iTzHTpAnbyg7VTeys/Ca2FQsZ95hgCHfKsOHEX2/HtxpkGtXDjJKPHq43M2MR3Bp9+YUBAxcj5WMsGcs0lp7hFP6xADEJAcLdfta3SJCdNTa0Vw==</ds:SignatureValue>
        <ds:KeyInfo>
            <ds:X509Data>
                <ds:X509Certificate>
                   CertificateStuff...
                </ds:X509Certificate>
            </ds:X509Data>
        </ds:KeyInfo>
    </ds:Signature>
</facturaElectronicaCompraVenta>
"""

root = etree.fromstring(input_xml_string)
ns_modify_from = "http://www.w3.org/2000/09/xmldsig#"
ns_modify_to = ""
new_root = replace_namespace(root, ns_modify_from, ns_modify_to)

# create a new elementtree with new_root so that we can use the
# .write method.
tree = etree.ElementTree()
tree._setroot(new_root)

tree.write('done.xml',
           pretty_print=True, xml_declaration=True, encoding='UTF-8')

Which gives:

<?xml version='1.0' encoding='UTF-8'?>
<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header/>
   <detail/>
    <Signature>
        <SignedInfo>
            <CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
            <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
            <Reference URI="">
                <Transforms>
                    <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                    <Transform Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
                </Transforms>
                <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
                <DigestValue>KvIMPxajMb98G3+HdSLg1/pgSyisLp4OWZt6Gxhe+/c=</DigestValue>
            </Reference>
        </SignedInfo>
        <SignatureValue>Bv9W9cGyXvX4QeDDb61YME8TbnFlBOVBw2Iiv+a+7VrxjoH4z8kLO4rgonXbqGuk2DKrR4ACqoFQNd/9/lJb31TDk2SjegURBsjP9gLvFWwfq99jh6zn6rPF/gwqd+lA1ruGpDT/Q+vxMXeNpXfk+nDcgdDJoP1bpDEPHbSHGkQu2SX1NQP1SlRZkNoJXxorFfbTDmm1/VFRsv5uBNQvf7hSxTEvvLW8WVYN271iTzHTpAnbyg7VTeys/Ca2FQsZ95hgCHfKsOHEX2/HtxpkGtXDjJKPHq43M2MR3Bp9+YUBAxcj5WMsGcs0lp7hFP6xADEJAcLdfta3SJCdNTa0Vw==</SignatureValue>
        <KeyInfo>
            <X509Data>
                <X509Certificate>
                   CertificateStuff...
                </X509Certificate>
            </X509Data>
        </KeyInfo>
    </Signature>
</facturaElectronicaCompraVenta>

Solution 2(Seems to be problematic):

(This is my initial solution, however, I find it problematic, so I modify it and finally get solution 1)

I modify the codes from this answser from the following two aspects:

  1. I use a recursive function. The original answer only reset nsmap for the root node. So there will be a xmlns:ds=***" left in the Signature Element(What you meet as you stated in the comments)
  2. I add codes to update the attribte of the root. If using the original answer, the xsi:noNamespaceSchemaLocation attribute will be deleted.
def set_namsespace(root, target_nt):
    """
    Only keep the namespace target_nt in the tree rooted at the root, other namespace will be removed,
     and the target_nt is set to the default name space.
    To do so, I
    1. change the tag of root
        Change the namespace of root to be target_nt (Sine target_nt is default, it is omitted from the tag)
    2. change the attribute
        Change the namespace of root to be target_nt
    3. change the nsmap
        set the target_nt as the default namespace, other namespace will be deleted
    4. keep other property
    """
    # change the tag of root
    tag = etree.QName(root.tag)
    if tag.namespace is not None:
        root.tag = '{%s}%s' % (target_nt, tag.localname)

    # change the attribute of root
    # if there are attributes belong to other namespace, update to namespace of the
    # attributes to target_nt
    root_attrib_dict = dict(root.attrib)
    new_attrib_dict = {}
    for key, value in root_attrib_dict.items():
        key_QName = etree.QName(key)
        if key_QName.namespace is not None:
            new_key = '{%s}%s' % (target_nt, key_QName.localname)
        else:
            new_key = key
        new_attrib_dict[new_key] = value

    # set the new nsmap
    # only keep the target_nt, set to default namespace
    new_nsmap = {None: target_nt}

    # make the updated root
    new_root = etree.Element(root.tag, attrib=new_attrib_dict, nsmap=new_nsmap)

    # copy other properties
    new_root.text = root.text
    new_root.tail = root.tail

    # call recursively
    for old_root in root[:]:
        new_root.append(set_namsespace(old_root, target_nt))

    return new_root

Test Codes:

root = etree.fromstring(input_xml_string)
target_ns = "http://www.w3.org/2001/XMLSchema-instance"
new_root = set_namsespace(root, target_ns)

# create a new elementtree with new_root so that we can use the
# .write method.
tree = etree.ElementTree()
tree._setroot(new_root)

tree.write('done.xml',
           pretty_print=True, xml_declaration=True, encoding='UTF-8')

Which gives:

<?xml version='1.0' encoding='UTF-8'?>
<facturaElectronicaCompraVenta xmlns="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header/>
   <detail/>
    <Signature>
        <SignedInfo>
            <CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
            <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
            <Reference URI="">
                <Transforms>
                    <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                    <Transform Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
                </Transforms>
                <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
                <DigestValue>KvIMPxajMb98G3+HdSLg1/pgSyisLp4OWZt6Gxhe+/c=</DigestValue>
            </Reference>
        </SignedInfo>
        <SignatureValue>Bv9W9cGyXvX4QeDDb61YME8TbnFlBOVBw2Iiv+a+7VrxjoH4z8kLO4rgonXbqGuk2DKrR4ACqoFQNd/9/lJb31TDk2SjegURBsjP9gLvFWwfq99jh6zn6rPF/gwqd+lA1ruGpDT/Q+vxMXeNpXfk+nDcgdDJoP1bpDEPHbSHGkQu2SX1NQP1SlRZkNoJXxorFfbTDmm1/VFRsv5uBNQvf7hSxTEvvLW8WVYN271iTzHTpAnbyg7VTeys/Ca2FQsZ95hgCHfKsOHEX2/HtxpkGtXDjJKPHq43M2MR3Bp9+YUBAxcj5WMsGcs0lp7hFP6xADEJAcLdfta3SJCdNTa0Vw==</SignatureValue>
        <KeyInfo>
            <X509Data>
                <X509Certificate>
                   CertificateStuff...
                </X509Certificate>
            </X509Data>
        </KeyInfo>
    </Signature>
</facturaElectronicaCompraVenta>

This version of codes is kind of problematic. The code set the default name space to what you want to keep, and delete other namespace. This will add an unwanted default namespace declaration xmlns="http://www.w3.org/2001/XMLSchema-instance".

Solution 3:

Perhaps I should just reset the namespace?

def set_namsespace(root, target_nt):
    """
    Set the target_nt to be default namespace
    """
    # set the target_nt to be default namespace
    new_nsmap = root.nsmap.copy()
    for ns_key, ns_value in root.nsmap.items():
        if ns_value == target_nt:
            del new_nsmap[ns_key]
            new_nsmap[None] = target_nt

    # make the updated root
    root_attrib_dict = dict(root.attrib)
    new_root = etree.Element(root.tag, attrib=root_attrib_dict, nsmap=new_nsmap)

    # copy other properties
    new_root.text = root.text
    new_root.tail = root.tail

    # call recursively
    for old_root in root[:]:
        new_root.append(set_namsespace(old_root, target_nt))

    return new_root

Using:

root = etree.fromstring(input_xml_string)
target_ns = "http://www.w3.org/2000/09/xmldsig#"
new_root = set_namsespace(root, target_ns)

Which gives:

<?xml version='1.0' encoding='UTF-8'?>
<facturaElectronicaCompraVenta xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="facturaElectronicaCompraVenta.xsd">
   <header/>
   <detail/>
    <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
        <SignedInfo>
            <CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
            <SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
            <Reference URI="">
                <Transforms>
                    <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
                    <Transform Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
                </Transforms>
                <DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
                <DigestValue>KvIMPxajMb98G3+HdSLg1/pgSyisLp4OWZt6Gxhe+/c=</DigestValue>
            </Reference>
        </SignedInfo>
        <SignatureValue>Bv9W9cGyXvX4QeDDb61YME8TbnFlBOVBw2Iiv+a+7VrxjoH4z8kLO4rgonXbqGuk2DKrR4ACqoFQNd/9/lJb31TDk2SjegURBsjP9gLvFWwfq99jh6zn6rPF/gwqd+lA1ruGpDT/Q+vxMXeNpXfk+nDcgdDJoP1bpDEPHbSHGkQu2SX1NQP1SlRZkNoJXxorFfbTDmm1/VFRsv5uBNQvf7hSxTEvvLW8WVYN271iTzHTpAnbyg7VTeys/Ca2FQsZ95hgCHfKsOHEX2/HtxpkGtXDjJKPHq43M2MR3Bp9+YUBAxcj5WMsGcs0lp7hFP6xADEJAcLdfta3SJCdNTa0Vw==</SignatureValue>
        <KeyInfo>
            <X509Data>
                <X509Certificate>
                   CertificateStuff...
                </X509Certificate>
            </X509Data>
        </KeyInfo>
    </Signature>
</facturaElectronicaCompraVenta>

Upvotes: 1

Related Questions