CODEWITHSUNDEEP
pythonlistlogging2d
user1410747
user1410747

Reputation: 99

Reading from a file directly into a 2D list python

Here is the contents of my .log file

Current Date & Time, Trans, Elapsed Time, Response Time, Trans Rate,

Wed Jul 18 10:03:1, 5, 0.37 sec, 0.00 sec, 13.51 t/s,

Wed Jul 18 10:03:5, 5, 0.45 sec, 0.00 sec, 11.11 t/s,

Wed Jul 18 10:04:0, 5, 0.91 sec, 0.00 sec, 5.49 t/s,

Wed Jul 18 10:22:4, 12, 0.79 sec, 0.00 sec, 15.19 t/s,

Wed Jul 18 10:23:0, 12, 0.56 sec, 0.00 sec, 21.43 t/s,

Wed Jul 18 10:23:1, 12, 0.53 sec, 0.00 sec, 22.64 t/s,

I want to put these values directly into a 2D list in python, but every solution I have found just creates a list item for each row. I want the value of a list slot to be one comma separated value. EG row 3 column 2 in the list would store 5. It's probably quite simple but I've been searching and searching and found nothing to do it this way. Any help much appreciated. Cheers.

Update:

Traceback (most recent call last): File "StressTestCompare.py", line 45, in print data[2][0]
IndexError: list index out of range

[['Current Date & Time', '\tTrans', '\tElapsed Time', '\tResponse Time', '\tTrans Rate', '\tThroughput', '\tConc', '\tOKAY', '\tFailed', '\tData Transfer', '\tIP Address', ''], ['Wed Jul 18 10:03:1', '\t5', '\t0.37 sec', '\t0.00 sec', '\t13.51 t/s', '\t0.00 b/s', '\t0.00', '\t0', '\t0', '\t0 bytes', '\t10.2.2.55:8080'], [], ['Wed Jul 18 10:03:5', '\t5', '\t0.45 sec', '\t0.00 sec', '\t11.11 t/s', '\t0.00 b/s', '\t0.00', '\t0', '\t5', '\t0 bytes', '\t10.2.2.56:8080'], [], ['Wed Jul 18 10:04:0', '\t5', '\t0.91 sec', '\t0.00 sec', '\t5.49 t/s', '\t1609.89 b/s', '\t0.00', '\t5', '\t0', '\t1465 bytes', '\t10.2.2.57:8080'], [], ['Wed Jul 18 10:22:4', '\t12', '\t0.79 sec', '\t0.00 sec', '\t15.19 t/s', '\t0.00 b/s', '\t0.00', '\t0', '\t0', '\t0 bytes', '\t10.2.2.55:8080'], [], ['Wed Jul 18 10:23:0', '\t12', '\t0.56 sec', '\t0.00 sec', '\t21.43 t/s', '\t0.00 b/s', '\t0.00', '\t0', '\t12', '\t0 bytes', '\t10.2.2.56:8080'], [], ['Wed Jul 18 10:23:1', '\t12', '\t0.53 sec', '\t0.00 sec', '\t22.64 t/s', '\t0.00 b/s', '\t0.00', '\t0', '\t0', '\t0 bytes', '\t10.2.2.57:8080'], []]

Using print repr(data) It comes out like this, is it possible to get it in an easier to read format? Even still it looks like it's all there from what I can see, so I don't know why i'm getting the out of range error.

Upvotes: 3

Views: 2453

Answers (2)

Levon
Levon

Reputation: 143152

The csv module comes in handy for this:

import csv

with open('data.txt', 'rb') as inf:
     data = list(csv.reader(inf, skipinitialspace=True))
     data = [i for i in data if i] ## add to deal w/ blank lines in data file

would give you variable data with contents:

[['Current Date & Time',
  'Trans',
  'Elapsed Time',
  'Response Time',
  'Trans Rate',
  ''],
 ['Wed Jul 18 10:03:1', '5', '0.37 sec', '0.00 sec', '13.51 t/s', ''],
 ['Wed Jul 18 10:03:5', '5', '0.45 sec', '0.00 sec', '11.11 t/s', ''],
 ['Wed Jul 18 10:04:0', '5', '0.91 sec', '0.00 sec', '5.49 t/s', ''],
 ['Wed Jul 18 10:22:4', '12', '0.79 sec', '0.00 sec', '15.19 t/s', ''],
 ['Wed Jul 18 10:23:0', '12', '0.56 sec', '0.00 sec', '21.43 t/s', ''],
 ['Wed Jul 18 10:23:1', '12', '0.53 sec', '0.00 sec', '22.64 t/s', '']]

and data[2][1] (not data[3][2] due to zero-based indexing) would give you

'5'

a string, which you could convert to an integer 5 with int(data[2][1])

Update:

Added data = [i for i in data if i] to deal with possible blank lines in input causing problems in OP's updated post.

Upvotes: 6

eumiro
eumiro

Reputation: 213075

Have a look at pandas:

>>> import pandas
>>> a = pandas.read_csv('input.csv')
>>> print a
  Current Date & Time   Trans  Elapsed Time  Response Time  Trans Rate  Unnamed: 5
0  Wed Jul 18 10:03:1       5      0.37 sec       0.00 sec   13.51 t/s         NaN
1  Wed Jul 18 10:03:5       5      0.45 sec       0.00 sec   11.11 t/s         NaN
2  Wed Jul 18 10:04:0       5      0.91 sec       0.00 sec    5.49 t/s         NaN
3  Wed Jul 18 10:22:4      12      0.79 sec       0.00 sec   15.19 t/s         NaN
4  Wed Jul 18 10:23:0      12      0.56 sec       0.00 sec   21.43 t/s         NaN
5  Wed Jul 18 10:23:1      12      0.53 sec       0.00 sec   22.64 t/s         NaN

>>> print a[' Trans'] * 2
0    10
1    10
2    10
3    24
4    24
5    24
Name:  Trans

Upvotes: 3

Related Questions