CODEWITHSUNDEEP
pythonxmllxmlelementtree
user2015487
user2015487

Reputation: 2185

how to check lxml element tree strings?

I have a list of lxml element trees. I would like to store in a dictionary the number of times a sub-tree appears in any of subtress of the list of trees. For example 

tree1='''<A attribute1="1"><B><C/></B></A>'''
tree2='''<A attribute1="1"><D><C attribute="2"/></D></A>'''
tree3='''<E attribute1="1"><B><C/></B></E>'''
list_trees=[tree1,tree2,tree3]
print list_trees
from collections import defaultdict
from lxml import etree as ET
mydict=defaultdict(int) 
for tree in list_trees:
    root=ET.fromstring(tree)
    for sub_root in root.iter():
        print ET.tostring(sub_root)
        mydict[ET.tostring(sub_root)]+=1
print mydict

I get the following correct result:

defaultdict(<type 'int'>, {'<E attribute1="1"><B><C/></B></E>': 1, '<C/>': 2, '<A attribute1="1"><D><C attribute="2"/></D></A>': 1, '<B><C/></B>': 2, '<C attribute="2"/>': 1, '<D><C attribute="2"/></D>': 1, '<A attribute1="1"><B><C/></B></A>': 1})

This only works in this particular example. However, In the general case, xmls can be identical but have different ordering of attributes, or extra white spaces or new lines that don't matter. However, this general case will break my system. I know that there have been posts about how to check 2 identical xml trees, however, i would like to convert the xmls into strings in order to do this particular application described above (easily keeping unique trees as string allows for easy comparisons and more flexibility in the future) and also be able to store it in sql nicely. How can an xml be made into a string in a consistent matter, regardless of orderings, or extra spaces, extra lines?

editing for giving the case that does not work: These 3 xml trees are identical, they just have different ordering of attributes or extra spaces or new lines.

tree4='''<A attribute1="1" attribute2="2"><B><C/></B></A>'''
tree5='''<A attribute1="1"     attribute2="2"   >
<B><C/></B></A>'''
tree6='''<A attribute2="2" attribute1="1"><B><C/></B></A>'''

My output gives the following:

defaultdict(<type 'int'>, {'<B><C/></B>': 3, '<A attribute1="1" attribute2="2"><B><C/></B></A>': 1, '<A attribute1="1" attribute2="2">\n<B><C/></B></A>': 1, '<C/>': 3, '<A attribute2="2" attribute1="1"><B><C/></B></A>': 1})

However, the output should be:

defaultdict(<type 'int'>, {'<B><C/></B>': 3, '<A attribute1="1" attribute2="2"><B><C/></B></A>': 3, '<C/>': 3})

Upvotes: 1

Views: 354

Answers (1)

supersam654
supersam654

Reputation: 3244

If you insist on comparing the string representation of XML trees, I recommend using BeautifulSoup on top of lxml. In particular, calling prettify() on any part of the tree creates a distinct representation that ignores whitespace and strange formatting from the input. The output strings are bit more verbose but they work. I went ahead and replaced newlines with "fake newlines" ('\n' -> '\\n') so the output is more compact.

from collections import defaultdict
from bs4 import BeautifulSoup as Soup

tree4='''<A attribute1="1" attribute2="2"><B><C/></B></A>'''
tree5='''<A attribute1="1"     attribute2="2"   >
<B><C/></B></A>'''
tree6='''<A attribute2="2" attribute1="1"><B><C/></B></A>'''
list_trees = [tree4, tree5, tree6]

mydict = defaultdict(int)
for tree in list_trees:
    root = Soup(tree, 'lxml-xml') # Use the LXML XML parser.
    for sub_root in root.find_all():
        print(sub_root)
        mydict[sub_root.prettify().replace('\n', '\\n')] += 1

print('Results')
for key, value in mydict.items():
    print(u'%s: %s' % (key, value))

Which prints out the desired results (with a few extra newlines and spaces):

$ python counter.py

<A attribute1="1" attribute2="2">\n <B>\n  <C/>\n </B>\n</A>: 3
<B>\n <C/>\n</B>: 3
<C/>\n: 3

Upvotes: 1

Related Questions