CODEWITHSUNDEEP
pythoncsv
pafpaf
pafpaf

Reputation: 283

Reading until specific line of CSV in Python

I am opening a big CSV file and during inserting the data into the array I got MemoryError. So how can i read specific lines of that CSV file (for example from line 1 till 10000)

here is the code:

datafile=open('test.csv','r')
datareader=csv.reader(datafile,delimiter=';')

for row in datareader:
    MyArray.append(row)

Upvotes: 1

Views: 1727

Answers (2)

Jon Clements
Jon Clements

Reputation: 142206

I'd use islice instead of enumerate:

from itertools import islice

# First 10000
MyArray.extend(islice(datareader, 10000))

# Or, specify start/stop ranges (10000-20000 (non inclusive))
MyArray.extend(islice(datareader, 10000, 20000))

# Or read in chunks of 10k
for chunk in iter(lambda: list(islice(datareader, 10000)), []):
    # do something with 10k rows

Upvotes: 2

Tim Pietzcker
Tim Pietzcker

Reputation: 336408

Use enumerate():

for i, row in enumerate(datareader):
    MyArray.append(row)
    if i == 10000:
        break

or, for any range:

start = 1000
stop = 2000
for i, row in enumerate(datareader):
    if i < start: 
        continue     # skip this row
    elif i > stop:
        break        # abort the loop
    else:            # ("else" is not strictly necessary here, but more explicit)
        MyArray.append(row)

Upvotes: 2

Related Questions