Having some challenges parsing XML with Python and xml.etree.ElementTree

Question



  Tbl_Projects_Tableau
  https://share.corp.com/sites/CPIBudget/_vti_bin/ListData.svc/Tbl_Projects_Tableau/
  2018-07-25T21:27:59Z
  
  
    https://share.corp.com/sites/CPIBudget/_vti_bin/ListData.svc/Tbl_Projects_Tableau(1)
    
    2018-06-14T17:15:27Z
    
      
    
    
    
    
    
    
    
      
        6
        
        Project Swoop
        ABC
        ABC9
        500000
        Nookie
        Smooth
        1
        0x0100FD279BEBCF3C4F45BB75D6147D315C09
        1
        Item
        2018-06-14T17:15:27
        2018-06-14T16:58:50
        2
        2
        8
        1.0
        /sites/SmoothBudget/Lists/Projects_Tableau1

here is a sample of the XML I am trying to parse to CSV.

here is my code so far:

import config
import csv
import pymysql
import requests
import xml.etree.ElementTree as ET
from requests_ntlm import HttpNtlmAuth

ssoUsername = config.username
ssoPassword = config.password

f = open(path+csvFile,'w',newline='')
csvwriter = csv.writer(f)
column_headers = ['FBN','Project_Name','Cluster','Site','OP2_USD','Type','Group']
csvwriter.writerow(column_headers)

rows = []
r2 = requests.get(project_url, auth=HttpNtlmAuth('ANT\'+ssoUsername,ssoPassword), verify=False)
projectData = r2.content
etree2 = ET.fromstring(projectData)

#print(etree2.findall('.****'))

for element in etree2.findall(".****") :
    print(element.find('{http://schemas.microsoft.com/ado/2007/08/dataservices}FBN_IDId'))
    fbnKey2 = element.find('{http://schemas.microsoft.com/ado/2007/08/dataservices}FBN_IDId')
    FBN = fbnMap.get(fbnKey2)

so at this point I cannot get the .text of the '{http://schemas.microsoft.com/ado/2007/08/dataservices}FBN_IDId' element. No matter what xPath I try it always gives me NoneType has not attribute text error.

Here are the results of print(etree2.findall('.****'))

[, , , , , , , , , , , , , , , , , , , , , , , , , , , , ]

It seems like I should be able to get the FBNIDId, but the best I have been able to do is get

None

and that results in the none type error. The only time I was able to get it to work at all was to do :

for element in etree2.findall(".//{http://schemas.microsoft.com/ado/2007/08/dataservices}FBN_IDId") :
        fbnKey2 = element.text
        FBN = fbnMap.get(fbnKey2)

but if I do that then I have to do that for each element I need and then figure out how to combine them all into a single row and then loop through for adding all the rows and that seems wrong.

suggestions?

Luke Woodward · Accepted Answer

In your loop

for element in etree2.findall(".****") :

you are iterating through all elements four levels down. For each such element you then search within its child elements for those named FBN_IDId. This will find any such elements that are five levels down.

However, there aren't any such elements. They only exist four levels down.

Perhaps you want to iterate through all elements three levels down and look for child elements of these, named FBN_IDId? This could be done by removing one of the *s in your loop:

for element in etree2.findall(".***") :

However this loop would also find the empty element.

Perhaps it would be better to write

for element in etree2.findall(".//{http://schemas.microsoft.com/ado/2007/08/dataservices/metadata}properties") :

which loops through all elements at any depth.

Having some challenges parsing XML with Python and xml.etree.ElementTree

Answers (2)

Related Questions