CODEWITHSUNDEEP
pythonfile
ZZMike
ZZMike

Reputation: 550

How to open a list of files in Python

I'm reading data file (text), and generating a number of reports, each one is written to a different output file (also text). I'm opening them the long way:

fP = open('file1','w')

invP = open('inventory','w')

orderP = open('orders','w')

... and so on, with a corresponding group of close() lines at the end.

If I could open them with a for loop, using a list of fP names and file names, I could guarantee closing the same files.

I tried using a dictionary of fp:filename, but that [obviously] didn't work, because either the fP variable is undefined, or a string 'fP' isn't a good file object name.

Since these are output files, I probably don't need to check for open errors - if I can't open one or more, I can't go on anyway.

Is there any way to open a group of files (not more than 10 or so) from a list of names, in a loop?

Upvotes: 12

Views: 31587

Answers (6)

Kos
Kos

Reputation: 72271

Good news! Python 3.3 brings in a standard safe way to do this:

contextlib.ExitStack

From the docs:

Each instance maintains a stack of registered callbacks that are called in reverse order when the instance is closed.
(...)
Since registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested with statements had been used with the registered set of callbacks.

Here's an example how to use it:

from contextlib import ExitStack

with ExitStack() as stack:
    files = [
        stack.enter_context(open(filename))
        for filename in filenames
    ]
    # ... use files ...

When the code leaves the with statement, all files that have already been opened will be closed.

This way you also know that if 2 files get opened and then third file fails to open, the two already-opened files will be closed correctly. Also if an exception is raised anytime inside the with block, you'll see correct cleanup.

Upvotes: 19

Victor Barrantes
Victor Barrantes

Reputation: 2318

Both answers above are good if you know or define ahead of time the list of files you will want to create. But, in case you want a more generic solution, you can build that list just in time, use your OS to create empty files on disk (this is done different ways depending on the OS you are), then create the list of files interactively this way:

import os
working_folder = input("Enter full path for the working folder/directory: ")
os.chdir(working_folder)
filenames_list = os.listdir()

#you can filter too, if you need so:
#filenames_list = [filename for filename in os.listdir() if '.txt' in filename]

#then you can do as Reut Sharabani and A.J. suggest above and make a list of file descriptors
file_descriptors = [open(filename, 'w') for filename in filenames_list]
#or a dictionary as Reut Sharabani suggests (I liked that one Reut :)

#do whatever you need to do with all those files already opened for writing

#then close the files
for fd in file_descriptors:
   fd.close()

It is ok to use "with"; as some suggest, if you work with only one file at the time (from start to finish), but if you want to work with all the files at the same time, it is better a list or dictionary of file descriptors.

Upvotes: 1

Ajay
Ajay

Reputation: 5347

Since you are saying there are many data files.Instead of entering filenames manually into a list.You can get the filenames into a list with this.

from os import listdir
from os.path import isfile, join
files_in_dir = [ f for f in listdir('/home/cam/Desktop') if isfile(join('/home/cam/Desktop',f)) ]

Now you can 

for file in files_in_dir:
    with open(file, 'w') as f:
        f.do_something

Upvotes: 5

grayshirt
grayshirt

Reputation: 643

Use the with keyword to guarantee that opened files (and other similar resources, known as "context managers") are closed:

with open(file_path, 'w') as output_file:
    output_file.write('whatever')

Upon exiting the with block, the file will be properly closed -- even if an exception occurs.

You could easily loop over a list of paths to the desired files:

files = ['fp1.txt', 'inventory', 'orders']
for file in files:
    with open(file, 'w') as current_file:
        current_file.do_some_stuff()

Upvotes: 5

Reut Sharabani
Reut Sharabani

Reputation: 31339

You can open as many files as you want and keep them in a list to close them later:

fds = [open(path, 'w') for path in paths]

# ... do stuff with files

# close files
for fd in fds:
    fd.close()

Or you could use a dictionary for better readability:

# map each file name to a file descriptor
files = {path: open(path, 'w') for path in paths}

# use file descriptors through the mapping
files['inventory'].write("Something")

# close files
for path in files:
    files[path].close()

Upvotes: 3

A.J. Uppal
A.J. Uppal

Reputation: 19264

Yes, you can use a list comprehension:

filenames = ['file1.txt', 'file2.txt', 'file3.txt'...]
filedata = {filename: open(filename, 'w') for filename in filenames}

Now, all of the opened instances are saved in filedata, assigned to the name of the file.

To close them:

for file in filedata.values():
    file.close()

Upvotes: 10

Related Questions