CODEWITHSUNDEEP
pythoncsviterator
mary
mary

Reputation: 2607

Reading .csv in Python without looping through the whole file?

The only way I've seen Python's csv.reader used is in a for loop, which goes through the whole file without saving past values of the read in variables. I only need to work with 2 consecutive lines of the (enormous) file at a time. Using the csv.reader for loop, I only have 1 line at a time.

Is there a way to use Python's csv module for taking in only one line of a csv file without having to finish reading the file to the end?

I need to set variables to the values in the first line, set a second set of variables to the values of the next line, use the two sets of variables simultaneously for computations, then overwrite the first set of variables with the second set, and read a new line to overwrite the second set.

Upvotes: 27

Views: 48108

Answers (5)

Bryan Oakley
Bryan Oakley

Reputation: 386010

There's nothing forcing you to use the reader in a loop. Just read the first line, then read the second line.

import csv
rdr = csv.reader(open("data.csv"))
line1 = rdr.next() # in Python 2, or next(rdr) in Python 3
line2 = rdr.next()

Upvotes: 43

hatef
hatef

Reputation: 6199

Read CSV:

readCSV = csv.reader(csvFile, delimiter=',')

Read the next row in Python 2.7:

    row = readCSV.next()

Read the next row in Python 3.4:

    row = readCSV.__next__()

Upvotes: 5

Jeffrey Harris
Jeffrey Harris

Reputation: 3550

If you're always looking at exactly two consecutive lines, it sounds to me like you might benefit from using the pairwise recipe. From the itertools module:

from itertools import tee, izip
def pairwise(iterable):
   "s -> (s0,s1), (s1,s2), (s2, s3), ..."
   a, b = tee(iterable)
   next(b, None)
   return izip(a, b)

You would use this like so:

for first_dict, second_dict in pairwise(csv.DictReader(stream)):
    # do stuff with first_dict and second_dict

Upvotes: 5

dash-tom-bang
dash-tom-bang

Reputation: 17853

Blatant stealing from TK... ...mostly the question that remains is, what does the OP want to do with the first and last lines of the file?

prevLine = None

for x in csv.DictReader(stream):
   if prevLine is not None:
       DoWork(prevLine, x)
   else:
       Initialize(x)
   prevLine = x

Finalize(prevLine)

Upvotes: 4

TK.
TK.

Reputation: 417

The obvious answer seems to be to just store the previous line on each iteration.

>>> for x in csv.DictReader(stream):
...   print prevLine
...   print x
...   prevLine = x
....

Upvotes: 4

Related Questions