CODEWITHSUNDEEP
pythonwith-statementcontextmanager
Ryan Haining
Ryan Haining

Reputation: 36882

Open a list of files using with/as context manager

Note: I am aware of the

with open('f1') as f1, open('f2') as f2:
    ...

syntax. This is a different question.

Given a list of strings file_names is there a way using with/as to open every file name in that using a single line. Something such as:

with [open(fn) for fn in file_names] as files:
    # use the list of files

which of course doesn't work as it attempts to use the context manager on a list. The length of the list is not known until runtime

Upvotes: 24

Views: 5080

Answers (3)

Max
Max

Reputation: 3445

How about this?

class ListContext:
    def __init__(self, l):
        self.l = l

    def __enter__(self):
        for x in self.l:
            x.__enter__()
        return self.l

    def __exit__(self, type, value, traceback):
        for x in self.l:
            x.__exit__(type, value, traceback)

arr = ['a', 'b', 'c']

with ListContext([open(fn, 'w') for fn in arr]) as files:
    print files

print files

Output is:

[<open file 'a', mode 'w' at 0x7f43d655e390>, <open file 'b', mode 'w' at 0x7f43d655e420>, <open file 'c', mode 'w' at 0x7f43d655e4b0>]
[<closed file 'a', mode 'w' at 0x7f43d655e390>, <closed file 'b', mode 'w' at 0x7f43d655e420>, <closed file 'c', mode 'w' at 0x7f43d655e4b0>]

Notice, they are open inside the with context and closed outside.

This is using the Python context manager API.

EDIT: It seems like this already exists but is deprecated: See contextlib and this SO question. Use it like this:

import contextlib

with contextlib.nested(*[open(fn, 'w') for fn in arr]) as files:
    print files
print files

Upvotes: 6

Henry Keiter
Henry Keiter

Reputation: 17188

If you have access to Python 3.3+, there is a special class designed exactly for this purpose: the ExitStack. It works just like you'd expect:

with contextlib.ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

Upvotes: 28

Andrew Clark
Andrew Clark

Reputation: 208565

It sounds like you are basically looking for contextlib.nested(), this was deprecated in Python 2.7 in favor of the multiple manager form of the with statement but as noted in the docs:

The one advantage of this function over the multiple manager form of the with statement is that argument unpacking allows it to be used with a variable number of context managers

In case you are on Python 3.x, here is the code from the Python 2.7 source:

from contextlib import contextmanager

@contextmanager
def nested(*managers):
    """Combine multiple context managers into a single nested context manager.                                                                                                               
   This function has been deprecated in favour of the multiple manager form
   of the with statement.

   The one advantage of this function over the multiple manager form of the
   with statement is that argument unpacking allows it to be
   used with a variable number of context managers as follows:

      with nested(*managers):
          do_something()

    """
    warn("With-statements now directly support multiple context managers",
         DeprecationWarning, 3)                                                                                                                                                                  exits = []
    vars = []
    exc = (None, None, None)
    try:
        for mgr in managers:
            exit = mgr.__exit__
            enter = mgr.__enter__
            vars.append(enter())
            exits.append(exit)
        yield vars
    except:
        exc = sys.exc_info()
    finally:
        while exits:
            exit = exits.pop()
            try:
                if exit(*exc):
                    exc = (None, None, None)
            except:
                exc = sys.exc_info()
        if exc != (None, None, None):
            # Don't rely on sys.exc_info() still containing
            # the right information. Another exception may
            # have been raised and caught by an exit method
            raise exc[0], exc[1], exc[2]

Upvotes: 0

Related Questions