Parent node insertion in XML in Python not working like I wished

Question

I have a long XML file structured like this:

Attribute "bbox" contains four values, and I want to insert the tag "newline" every time there is a certain distance between the first number of the coordinates (example here, in the file they are all different) provided.

The concept is that if the distance between the first number of the current bbox and its preceding one is > 10, insert an open tag and close it when the distance comes again (in other words, close the tag when there is the need to open a new one). So the output would be:

[text tags]
[text tags]

etc.

My code so far is this:

import lxml.etree as etree
from lxml.builder import E

parser = etree.XMLParser(remove_blank_text=True)
tree = etree.parse('fe2.xml', parser)
root = tree.getroot()

def removeByIdx(parent, idx):
    currElem = parent[idx]   # The indicated element
    parent.remove(currElem)  # Remove it from the parent
    return currElem          # Return the index and element

def wrap(line, idxList):
    if len(idxList) == 0:
        return    # No elements to wrap
    # Take the first element from the original location
    idx = idxList.pop(0)     # Index of the first element
    elem = removeByIdx(line, idx) # The indicated element
    # Create "newline" element with "elem" inside
    nElem = E.newline(elem)
    line.insert(idx, nElem)  # Put it in place of "elem"
    while len(idxList) > 0:  # Process the rest of index list
        # Value not used, but must be removed
        idxList.pop(0)
        # Remove the current element from the original location
        currElem = removeByIdx(line, idx + 1)
        nElem.append(currElem)  # Append it to "newline"

for line in root.iter('textline'):
    idxList = []
    for x in line:
        bb = x.attrib.get('bbox')
        if bb is not None:
            bb = bb.split(',')
        #print('This: ', bb)
        this = bb
        xPrev = x.getprevious()
        bb = None
        if xPrev is not None:
            bb = xPrev.attrib.get('bbox')
            if bb is not None:
                bb = bb.split(',')
        if bb is not None:
            prev = bb
            #print(this, prev)
            if this is not None:
                distance = float(this[0]) - float(prev[0])
                print(distance)
                if distance > 10:
                    par = x.getparent()
                    idx = par.index(x)
                    idxList.append(idx)
                else:  # "Wrong" element, wrap elements "gathered" so far
                    wrap(line, idxList)
                    idxList = []
                    # Process "good" elements without any "bad" after them, if any
                wrap(line, idxList)


print(etree.tostring(root, encoding='unicode', pretty_print=True))

But it doesn't work as expected, I see no newline tag and if I shorten the distance it just wraps one text element. What am I missing?

EDIT: expected output

    


    APPARATO


    24a


    24b


    24c


    24d


    25a

    ****
    T
i
t
o
l
o
 I
l
 
 C
a
u
s
i
d
i
c
o
]
 
 s
p
s
.
 
 a
 Don  Rodrigo 
    
    ****
    ****
    1
    ... continues
****

Alexandre B. · Accepted Answer

You can try the following approach:

Find the text elements using xpath
Iterate over all elements:
- For all items, we keep the current value and previous values from the BBox. We keep the last valid value. That means that, if for a given element, the bbox tag is missing, the previous one will be used.
- Then we compute distance (e.g. |bb_current - bb_previous|)
- If the distance is bigger than 10: that means a new new_line tag needs to be opened. But first, we need to close the previous one. So the current_line is inserted in parent tag using insert (related topic)
- If the new_line is already open: we add current tags inside using append, else we let them as original.
Finally, we add the last new_line element outside the loop.

Notes: the others tag such as empty text elements or whitespace elemnts are move. They need to be handle in for loop if we want to keep their origin position.

Full code:

import lxml.etree as etree

parser = etree.XMLParser(remove_blank_text=True)
tree = etree.parse('data.xml', parser)
root = tree.getroot()

# Get the first BBox value as float
# Return null if not found
def getBBoxFirstValue(line):
    if line is not None:
        bb = line.attrib.get('bbox')
        if bb is not None:
            try:
                return float(bb.split(",")[0])
            except ValueError:
                pass
    return None


new_line        = None 
previous_bb     = None

for x in tree.xpath('//text'):
    # Get current bb value
    bb = getBBoxFirstValue(x)

    # Check current and past values aren't empty
    if bb is not None and previous_bb is not None:
        # If distance with preview bb > 10
        if abs(bb - previous_bb) > 10:
            # If new_line isn't empty: it's inserted into parent tag at position of current tag 
            if new_line is not None:
                x.getparent().insert(x.getparent().index(x), new_line)
            # A new "new_line" element is created
            new_line = etree.Element("new_line")

        # If the new line isn't not (e.g. one distance > 10 has been already found)
        if new_line is not None:
            new_line.append(x)

    # Keep latest non empty BBox 1st value
    if bb is not None:
        previous_bb = bb


# Add last new_line element if not null
if new_line is not None:
    tree.xpath('//text')[-1].getparent().append(new_line)


newtree = etree.tostring(root, encoding='utf-8', pretty_print=True)
newtree = newtree.decode("utf-8")

input.xml

output

Parent node insertion in XML in Python not working like I wished

Answers (1)

Related Questions