Parsing XML files with Python

Question

I'm trying to solve a problem where I have XML data that has 2 time series (sometimes more) of measurements in the same file. In this case temperature and wind speed measurements from the same time interval. See the XML file below:

   


        
                

                    
        
          2015-01-01T00:00:00Z
          2015-01-02T00:00:00Z
        
      
      
        
          2015-01-02T00:00:00Z
        
            

        
                    
                
                    
                    
            atmosphere
                    
                
            

                
                
                    
          
        
            
            
                101007
            Helsinki Rautatientori
            -16011680
            2934
                                                


            Helsinki

            
        
       
                        

                
                                Helsinki Rautatientori
                                60.17169 24.94460 
                            

                        
                    
                
          
                                             
                        
                             
                                      2015-01-01T00:00:00Z
                      4.6
                            
                                                  
                        
                             
                                      2015-01-01T01:00:00Z
                      4.5
                            
                                                  
                        
                             
                                      2015-01-01T02:00:00Z
                      4.2
                            
                                                  
                        
                             
                                      2015-01-01T03:00:00Z
                      4.0
                            
                                                  
                        
                             
                                      2015-01-01T04:00:00Z
                      4.1
                            
                                                  
                        
                             
                                      2015-01-01T05:00:00Z
                      4.1
                            
                                                  
                        
                             
                                      2015-01-01T06:00:00Z
                      4.5
                            
                                                  
                        
                             
                                      2015-01-01T07:00:00Z
                      4.1
                            
                                                  
                        
                             
                                      2015-01-01T08:00:00Z
                      3.3
                            
                                                  
                        
                             
                                      2015-01-01T09:00:00Z
                      2.9
                            
                                                  
                        
                             
                                      2015-01-01T10:00:00Z
                      2.5
                            
                                                  
                        
                             
                                      2015-01-01T11:00:00Z
                      2.4
                            
                                                  
                        
                             
                                      2015-01-01T12:00:00Z
                      2.5
                            
                                                  
                        
                             
                                      2015-01-01T13:00:00Z
                      2.7
                            
                                                  
                        
                             
                                      2015-01-01T14:00:00Z
                      2.8
                            
                                                  
                        
                             
                                      2015-01-01T15:00:00Z
                      3.1
                            
                                                  
                        
                             
                                      2015-01-01T16:00:00Z
                      3.3
                            
                                                  
                        
                             
                                      2015-01-01T17:00:00Z
                      3.7
                            
                                                  
                        
                             
                                      2015-01-01T18:00:00Z
                      4.1
                            
                                                  
                        
                             
                                      2015-01-01T19:00:00Z
                      4.1
                            
                                                  
                        
                             
                                      2015-01-01T20:00:00Z
                      4.4
                            
                                                  
                        
                             
                                      2015-01-01T21:00:00Z
                      4.7
                            
                                                  
                        
                             
                                      2015-01-01T22:00:00Z
                      4.6
                            
                                                  
                        
                             
                                      2015-01-01T23:00:00Z
                      4.7
                            
                                                  
                        
                             
                                      2015-01-02T00:00:00Z
                      4.2
                            
                                                 
                    
                

        
    
        
                


      
            

        
                    
                
                    
                    
            atmosphere
                    
                
            

                
                
                    
          
        
            
            
                101007
            Helsinki Rautatientori
            -16011680
            2934
                                                


            Helsinki

            
        
       
                        

                
                                Helsinki Rautatientori
                                60.17169 24.94460 
                            

                        
                    
                
          
                                             
                        
                             
                                      2015-01-01T00:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T01:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T02:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T03:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T04:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T05:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T06:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T07:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T08:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T09:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T10:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T11:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T12:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T13:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T14:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T15:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T16:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T17:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T18:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T19:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T20:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T21:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T22:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-01T23:00:00Z
                      NaN
                            
                                                  
                        
                             
                                      2015-01-02T00:00:00Z
                      NaN

The data is nested within for the temperature and within for the wind speed (NaN's in this file).

I would like to iterate through those series separately and save them to .txt file in tabular format. I'm able to save the times series data, but I cant separate them column wise eg. to time|temp|wind speed format.

My current code works if the XML file has only one measurement (temp OR wind speed separately) because I only need to iterate through the file.

import xml.etree.cElementTree as ET

tree = ET.ElementTree(file='XML_file.xml')
def parser(item1,item2):
    return item1.text,item2.text

data = [parser(item1,item2) for item1,item2 in zip(tree.iter(tag='{http://www.opengis.net/waterml/2.0}time'),tree.iter(tag='{http://www.opengis.net/waterml/2.0}value'))]



with open('data.txt', 'w') as fp:
        fp.write('
'.join('%s|%s' % x for x in data))

Because the measurements for temp and ws are the same length how could I iterate through them (different tag perhaps?) and return tuple for each time point eg. (time,temp value, winds speed value)

EDIT: This prints results. But if I add tag like this i.iter(tag='{http://www.opengis.net/waterml/2.0}MeasurementTimeseries') I get TypeError: iter() takes no keyword arguments.

for i in tree.iter(tag='{http://www.opengis.net/waterml/2.0}MeasurementTimeseries'):
    for j in i.iter():
        print j.text

This prints:

2015-01-01T00:00:00Z
4.6




2015-01-01T01:00:00Z
4.5

abarnert · Accepted Answer

I think what you're asking for is this: Instead of finding all time and value tags anywhere in the file, you want to find the time and value tags only under the temperature tag. (And then do the same thing for the time and value tags only under the windspeed tag, and then zip the two together, but you could do that part yourself if you had the first part.)

To do this, you need a nested search.

You can do that explicitly, with two searches—e.g., iterate over the two MeasurementTimeseries, then, within each one, iterate over its time and value descendants:

def parse_one_series(series):
    return [parser(item1,item2) for item1,item2 in
            zip(series.iter(tag='{http://www.opengis.net/waterml/2.0}time'),
                series.iter(tag='{http://www.opengis.net/waterml/2.0}value'))]

data = zip(*(parse_one_series(series) for series in
           tree.iter(tag='{http://www.opengis.net/waterml/2.0}MeasurementTimeseries')))

And now, each element in data is a pair of pairs, the first being time and temp, the second being time and windspeed. Although you might want to organize things a little more nicely:

data = [(time, temp, wind) for (time, temp), (_, wind) in data]

Parsing XML files with Python

Answers (1)

Related Questions