CODEWITHSUNDEEP
python
Whiskey
Whiskey

Reputation: 459

Memory Error while reading a Big text file

I have a Text File Of Around 36gb which contains words per line, i am trying to read the file, but it says Memory Error, at which I am not shocked, but how do I work around to read it?

I am trying this:

for words in open("hugefile.txt").readlines():
    #do something

I have 2gb RAM, OS: Windows XP, Python 2.7

Thanks.

Upvotes: 0

Views: 1046

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1121306

You are calling readlines() which loads the whole file into memory.

Iterate over the file instead:

for words in open("hugefile.txt"):

This'll iterate over lines one by one, reading more lines as needed.

Upvotes: 8

Related Questions