CODEWITHSUNDEEP
pythonfilememoryio
Ben Pickering
Ben Pickering

Reputation: 99

Open file I/O memory management in python

I'm writing a script which has a huge list of weather observations. For each weather observation in the list, the code opens a weather radar file for that time. It then pulls out some data and moves onto the next line/observation.

To be safe with the memory, I want to close the weather radar file when I move onto the next line. Here is a simplified version of what I do:

    synop = open(filename).read().splitlines()
    synop.close()

The first line is fine and I can do stuff later with synop just fine. Trying to close synop causes a problem though as it is type = list.

Is there an alternate way to close the file I am accessing? Or does python automatically close the file after the first line of code?

I will be running this on approx. 4.5m files so I need good memory management!

Upvotes: 1

Views: 92

Answers (1)

Gonzo
Gonzo

Reputation: 1553

The problem is that you're not keeping the file object. Python will close the file for you after the first line, but thats not correct nor pythonic, since it's not explicit.

One correct way is to keep the file object, like this:

f = open(filename)
synop = f.read().splitlines()
f.close()

But the best way is to use context management

with open(filename) as f:
    synop = f.read().splitlines()

Python will close file for you as soon this scope ends.

Upvotes: 1

Related Questions