CODEWITHSUNDEEP
pythonio
Mason.Sun
Mason.Sun

Reputation: 71

Using too many Open calls. How do I close all files?

I'm trying to change a lot of pdf-files. Because of this I must open a lot of files. I use the method open to many times. So python gives the error too many open files.

I hope my code is grace.writer many too similar 

readerbanner = PyPDF2.pdf.PdfFileReader(open('transafe.pdf', 'rb'))
readertestpages = PyPDF2.pdf.PdfFileReader(open(os.path.join(Cache_path, cache_file_name), 'rb'))
writeroutput.write(open(os.path.join(Output_path,cache_file_name), 'wb'))

or

writer_output.write(open(os.path.join(Cache_path, str(NumPage) + "_" + pdf_file_name), 'wb'))
reader_page_x = PyPDF2.pdf.PdfFileReader(open(os.path.join(PDF_path, pdf_file_name), 'rb'))

All the open methods do not use f_name = open("path","r").

because all open file have period. I know the position but not know how close all open files.

Upvotes: 1

Views: 6991

Answers (4)

Wasper
Wasper

Reputation: 113

ExitStack can be useful: https://docs.python.org/3/library/contextlib.html#contextlib.ExitStack

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

Upvotes: 0

Binnut
Binnut

Reputation: 90

As far as i know, this code should not open too many files. Unless it is run a lot of times.

Regardless, the problem consists of you calling:

PyPDF2.pdf.PdfFileReader(open('transafe.pdf', 'rb'))

and similar. This creates a file object, but saves no reference to it. What you need to do for all open calls is:

file = open('transafe.pdf', 'rb')
PyPDF2.pdf.PdfFileReader(file)

And then:

file.close()

when you do not use the file anymore. If you want to close many files at the same time, put them in a list.

Upvotes: 2

mjy
mjy

Reputation: 324

with statement

with open("abc.txt", "r") as file1, open("123.txt", "r") as file2:
    # use files
    foo = file1.readlines()

# they are closed automatically
print(file1.closed)
# -> True
print(file2.closed)
# -> True

wrapper function

files = []
def my_open(*args):
    f = open(*args)
    files.append(f)
    return f

# use my_open
foo = my_open("text.txt", "r")

# close all files
list(map(lambda f: f.close(), files))

wrapper class

class OpenFiles():
    def __init__(self):
        self.files = []

    def open(self, *args):
        f = open(*args)
        self.files.append(f)
        return f

    def close(self):
        list(map(lambda f: f.close(), self.files))


files = OpenFiles()

# use open method
foo = files.open("text.txt", "r")

# close all files
files.close()

Upvotes: 1

Caumons
Caumons

Reputation: 9595

To close a file just call close() on it.

You can also use a context manager which handles file closing for you:

with open('file.txt') as myfile:
    # do something with myfile here

# here myfile is automatically closed

Upvotes: 3

Related Questions