CODEWITHSUNDEEP
pythonxmlxml-parsing
jenhenry
jenhenry

Reputation: 23

Parse xml data with multiple roots in python

I'm making an API call that returns multiple xml responses as so-

<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BESAPI.xsd">
        <Action Resource="https://www.example.com">
                <Name> ABC </Name>
                <ID> 123 </ID>
        </Action>
</BESAPI>

<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BESAPI.xsd">
        <Action Resource="https://www.example.com">
                <Name> DEF </Name>
                <ID> 456 </ID>
        </Action>
</BESAPI>

<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BESAPI.xsd">
        <Action Resource="https://www.example.com">
                <Name> GHI </Name>
                <ID> 789 </ID>
        </Action>
</BESAPI>

I want to parse all the action IDs from the tag and add them to a list-

import xml.etree.ElementTree as ET
url = ""
payload = ""
headers = {}
response = requests.post(url, headers=headers, data=payload)

root = ET.fromstring(response.content)
actionidlist = []
for elem in root.iter('Action'):
    for subelem in elem.iter('ID'):
        actionidlist.append(subelem.text)
        print(actionidlist)

I get errors though because there are multiple roots. How do I parse this?

Edit: By errors I mean, actionidlist seems to only contain the last ID and not the rest of the IDs.

Upvotes: 0

Views: 692

Answers (2)

joao
joao

Reputation: 2293

ET.fromstring() only parses one XML section, if you try to parse your entire input data, with multiple roots, you get the error:

xml.etree.ElementTree.ParseError: junk after document element: line 9, column 0

So I suggest pre-processing the input data, to split it into a list of xml responses, then parse each one in turn:

import xml.etree.ElementTree as ET
url = ""
payload = ""
headers = {}
response = requests.post(url, headers=headers, data=payload)

# Split the input data into a list of strings (xml sections)
xml_sections = ['']
for line in response.content.splitlines():
    if len(line) != 0:
        xml_sections[-1] += line + '\n'
    else:
        xml_sections.append('')

# Parse each XML section separately
actionidlist = []
for s in xml_sections:
    root = ET.fromstring(s)
    for elem in root.iter('Action'):
        for subelem in elem.iter('ID'):
            actionidlist.append(subelem.text)
print(actionidlist)

This produces the following output:

[' 123 ', ' 456 ', ' 789 ']

Upvotes: 1

Harshal Taware
Harshal Taware

Reputation: 33

from pyspark.sql import SQLContext
sqlContext = SQLContext(sc)
file = "filepath/<xml_file_name.xml>"
schema_path = "filepath/<xml_schame_name.xml>"
"""
"""
XSD Schema
schema_path =
<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation="BESAPI.xsd">
        <Action Resource="https://www.example.com">
                <Name> string </Name>
                <ID> INT </ID>
        </Action>
</BESAPI>

<?xml version="1.0" encoding="UTF-8"?>
<BESAPI xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation="BESAPI.xsd">
        <Action Resource="https://www.example.com">
                <Name> string </Name>
                <ID> INT </ID>
        </Action>
</BESAPI>
"""


df_schema = sqlContext.read.format('com.databricks.spark.xml').options(rowTag='Resource').load(schema_path)
df =sqlContext.read.format('com.databricks.spark.xml').options(rowTag='Resource').load(path,schema=df_schema.schema)
#display(df)
df.createOrReplaceTempView("temptable")
structured_df =sqlContext.sql("select concat_ws(', ',Action.Name) as Name,concat_ws(', ',Action.ID) as ID from temptable")
display(structured_df)

Upvotes: 0

Related Questions