CODEWITHSUNDEEP
pythoncontextmanager
Ibolit
Ibolit

Reputation: 9720

python: `with open()` and unknown number of files

Say, I want to open a number of files and process them line by line. For the sake of argument, let's assume I want to concatenate the nth lines in each file into one line and write that into yet another file. I don't want to bloat my memory, so I don't want to read them in in their entirety. And I don't know how many files my function will get.

If I knew the number of files, I'd do something like this: 

with open(file_a) as in_a, open(file_b) as in_b, open(file_c, "w") as out:
    try:
        while True:
            line_a = next(in_a)
            line_b = next(in_b)
            out.write(line_a + line_b)
    except StopIteration:
        pass

Is there a way to do something similar with an unknown number of inputs (or other context managers)?

Upvotes: 3

Views: 903

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1123720

Use the contextlib.ExitStack() object to track an arbitrary number of context managers:

from contextlib import ExitStack

with ExitStack() as stack, open(file_c, "w") as out:
    infiles = [stack.enter_context(open(fname)) for fname in filenames]
    for lines in zip(*infiles):
        out.writelines(lines)

Upvotes: 9

Related Questions