Opening a space(?) delimited text file in python 2.7?

Question

I have what I think is a space delimited text file that I would like to open and copy some of the data to lists (Python 2.7). This is a snippet of the data file:

    0.000000       11.00      737.09        1.00     1116.00
    0.001000       14.00      669.29       10.00      613.70
    0.002000       15.00      962.27        2.00      623.50
    0.003000        7.00      880.86        7.00      800.71
    0.004000        9.00      634.67        3.00     1045.00
    0.005000       12.00      614.67        3.00      913.33
    0.006000       12.00      782.58        6.00      841.00
    0.007000       13.00      860.08        6.00      354.00
    0.008000       14.00      541.07        4.00      665.25
    0.009000       14.00      763.00        6.00     1063.00
    0.010000        9.00      790.33        6.00      857.83
    0.011000        6.00      899.83        4.00     1070.75
    0.012000       16.00      710.88       10.00      809.90
    0.013000       12.00      863.50        7.00      923.14
    0.014000        9.00      591.67        6.00      633.17
    0.015000       12.00      740.58        6.00      837.00
    0.016000       10.00      727.60        7.00      758.00
    0.017000       12.00      838.75        4.00      638.75
    0.018000        9.00      991.33        7.00      731.57
    0.019000       12.00      680.75        5.00     1079.40
    0.020000       15.00      843.20        3.00      546.00
    0.021000       11.00      795.18        5.00     1317.20
    0.022000        9.00      943.33        5.00      911.00
    0.023000       13.00      711.23        3.00      981.67
    0.024000       11.00      922.73        5.00     1111.00
    0.025000     1112.00      683.58        6.00      542.83
    0.026000       15.00     1053.80        5.00     1144.40

Below is the code I have tried, which does not work. I would like to have two lists, one each from the second and the fourth column.

listb = []
listd = []
with open('data_file.txt', 'r') as file:        
     reader = csv.reader(file,delimiter=' ')
     for a,b,c,d,e in reader:   
         listb.append(int(b))
         listd.append(int(d))  

What am I doing wrong?

Answer 1

You can find all values you need, using regexp

import re

list_b = []
list_d = []

with open('C://data_file.txt', 'r') as f:
    for line in f:
        list_line = re.findall(r"[\d.\d+']+", line)
        list_b.append(float(list_line[1])) #appends second column
        list_d.append(float(list_line[3])) #appends fourth column

print list_b
print list_d

Answer 2

f=open("input.txt",'r')
x=f.readlines()
list1=[]
list2=[]
import re
for line in x:
  pattern=re.compile(r"(\d+)(?=\.)")
  li=pattern.findall(line)
  list1.append(li[1])
  list2.append(li[3])

You can use this if you only want to capture integersand not floats.

Answer 3

The problem is the multiple spaces between fields (columns).

CSV stands for comma-separated values. Imagine for a second that you are using commas instead of spaces. Line 1 in your file would then look like:

,,,,0.000000,,,,,,,11.00,,,,,,737.09,,,,,,,1.00,,,,,1116.00

So, the CSV reader sees more than 5 fields (columns) in that row.

You have two options:

1. Switch to using single space separators
2. Use a simple split() to deal with multiple whitespace:

:

 listb = []
 listd = []
 with open('text', 'r') as file:
    for row in file:
        a, b, c, d, e = row.split()
        listb.append(int(b))
        listd.append(int(d))

P.S: Once this part is working, you will run into a problem calling int() on strings like "11.00" which aren't really integers. So I recommend using something like:

int(float(b))

Answer 4

One alternative is to take advantage of the built-in str.split():

a, b, c, d, e = zip(*((map(float, line.split()) for line in open('data_file.txt'))))