How to deal with ValueError: invalid literal for float() in python

Question

My problem is different because it does not deal with regular expressions. so I think its a slightly different. I got this error.

ValueError: invalid literal for float(): 512220      0      20      34.4     
 2.4      0     10010      913        52      0.00

my csv file looks like

512220     0      20       34.4      2.4      0     10010      913        52      0.00
512221     1      30       34.6      2.3      0     10230      910.3      54      0.00
512222     2      50       34.8      2.1      0     10020      932        56      0.00
512223     3      60       35.4      2.5      0     10340      945.5      58      0.00

my code is

with open(item[1]) as f:
    lines = f.readlines()
    print 'lines', lines
for k, line in enumerate(lines):
    data_temporary = line.strip().split("\r\n")

when i print "lines" i got follwing

['512220     0      20       34.4      2.4      0     10010      913        52   
 0.00\n', '512221     1      30       34.6      2.3      0     10230      910.3   
 54      0.00\n', '512222     2      50       34.8      2.1      0     10020     
 932        56      0.00\n', '512223     3      60       35.4      2.5  
 0     10340      945.5      58      0.00'\n]

when I print data_temporary i got the following one line only.

['160129    29  0000     0      0.04       5.3      2.04  
  0.00     11758      9.13        52      0.00']

I tried these commands and results are as follows. . data_temporary = line.strip().split(" ")

['512220', '', '', '', '', '', '', '0', '', '', '', '', '', '20', '', '', '', 
 '', '', '', '34.4', '', '', '', '', '', '2.4', '', '', '', '', '', '0', '', '',
 '', '10010', '', '', '', '', '', '913', '', '','', '', '', '52', '', '',
 '', '', '', '0.00']

I tried to apply different solutions found on SO but couldn't work. like I try to use

  lines = map(lambda l: l.strip().split('\t'), lines) and some others.

I think I had to break list into string and then perform operation on it. could someone help me to solve this problem so that I understand better. thanks

Answer 1

If you iterate over a file with a for loop, you will get one line each iteration. Then you can call split() on that line to split it by whitespace into a list.

with open('filename.txt', 'r') as f:
    for line in f:
        data = line.split()
        print data

        z = float(data[3])

Output:

['512220', '0', '20', '34.4', '2.4', '0', '10010', '913', '52', '0.00']
['512221', '1', '30', '34.6', '2.3', '0', '10230', '910.3', '54', '0.00']
['512222', '2', '50', '34.8', '2.1', '0', '10020', '932', '56', '0.00']
['512223', '3', '60', '35.4', '2.5', '0', '10340', '945.5', '58', '0.00']

A lot of your elements look like integers so I wouldn't suggest converting every field to float. Instead, I would pick out the individual columns and convert them.

I don't know the name of your fields, so I made some up. Here's some code that will load this file into a list of dictionaries where the fields have been converted to the appropriate type:

from pprint import pprint

fields = [ 
    ('id', int),
    ('n', int),
    ('s', int),
    ('a', float),
    ('b', float),
    ('z', int),
    ('n2', int),
    ('top', float),
    ('x', int),
    ('bottom', float),
]

def read_data(path):
    with open(path, 'r') as f:
        for line in f:
            data = line.split()

            res = {}
            for n, field in enumerate(fields):
                name, _type = field
                res[name] = _type(data[n])
            yield res 

pprint(list(read_data('data.txt')))

Output:

[{'a': 34.4,
  'b': 2.4,
  'bottom': 0.0,
  'id': 512220,
  'n': 0,
  'n2': 10010,
  's': 20,
  'top': 913.0,
  'x': 52,
  'z': 0},
 {'a': 34.6,
  'b': 2.3,
  'bottom': 0.0,
  'id': 512221,
  'n': 1,
  'n2': 10230,
  's': 30,
  'top': 910.3,
  'x': 54,
  'z': 0},
 {'a': 34.8,
  'b': 2.1,
  'bottom': 0.0,
  'id': 512222,
  'n': 2,
  'n2': 10020,
  's': 50,
  'top': 932.0,
  'x': 56,
  'z': 0},
 {'a': 35.4,
  'b': 2.5,
  'bottom': 0.0,
  'id': 512223,
  'n': 3,
  'n2': 10340,
  's': 60,
  'top': 945.5,
  'x': 58,
  'z': 0}]

Answer 2

'' is not a valid value for float.

Try data_temporary = line.split() and see if that works.

Or, using a list comprehension:

values = [float(item) for item in line.split() if item]