Elegant way of combining files - Python

Question

Hi just a quick question. I have a concatenation of files that works flawlessly, but its a bit of a mess. I wondered if there was just a more elegant way to write this:

path = path/to/file/location
with open(path + 'result.txt', 'w') as result, \
        open(path + 'file1.txt') as f1, \
            open(path + 'file2.txt' ) as f2, \
                open(path + 'file3.txt' ) as f3, \
                    open(path + 'file4.txt' ) as f4, \
                        open(path + 'file5.txt' ) as f5, \
                            open(path + 'file6.txt' ) as f6, \
                                open(path + 'file7.txt' ) as f7, \
                                    open(path + 'file8.txt' ) as f8, \
                                        open(path + 'file9.txt' ) as f9, \
                                            open(path + 'file10.txt' ) as f10, \
                                                open(path + 'file11.txt' ) as f11, \
                                                    open(path + 'file12.txt' ) as f12, \
                                                        open(path + 'file13.txt' ) as f13, \
                                                            open(path + 'file14.txt' ) as f14, \
                                                                open(path + 'file15.txt' ) as f15, \
                                                                    open(path + 'file16.txt' ) as f16:
    for line1, line2, line3, line4, line5, line6, line7, line8, \ 
        line9, line10, line11, line12, line13, line14, line15, line16 \
        in zip(f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13,f14,f15,f16):

        result.write('{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, \
        {}, {}, {}\n'.format(line1.rstrip(), line2.rstrip(), line3.rstrip(), line4.rstrip(), \
        line5.rstrip(), line6.rstrip(), line7.rstrip(), line8.rstrip(), line9.rstrip(), \
        line10.rstrip(), line11.rstrip(), line12.rstrip(), line13.rstrip(), line14.rstrip(), \
        line15.rstrip(), line16.rstrip()))

Thanks

Answer 1

You could always do it without with, putting the files in a list and closing them manually, in a loop, when everything is done. This will also make the format line much simpler:

path = "path/to/file/location/"
with open(path + 'result.txt', 'w') as result:
    files = [open(path + 'file%d.txt' % (n+1)) for n in range(16)]
    form = ", ".join('{}' for f in files) + '\n'
    for lines in zip(*files):
        result.write(form.format(*map(str.rstrip, lines)))
    for f in files:
        f.close()

Or using contextlib.ExitStack, as suggested in comments. This way, the opened files are passed to the stack and that will take care of closing the files after the with block.

with open(path + 'result.txt', 'w') as result, contextlib.ExitStack() as stack:
    files = [stack.enter_context(open(path + 'file%d.txt' % (n+1))) for n in range(16)]
    form = ", ".join('{}' for f in files) + '\n'
    for lines in zip(*files):
        result.write(form.format(*map(str.rstrip, lines)))

Answer 2

You could process the files serially (which gets around the potential problem of having too many files open at once):

result_names = ['result1','result2']
result_index = 0
old_result_path = path + "file1.txt"
for n in xrange(2,17):
    new_result_path = path + (result_names[result_index] if n<16 else 'result.txt')
    input_path = path + "file%d.txt" % n
    with open( old_result_path, 'r' ) as old_input, \
        open( input_path, 'r' ) as new_input, 
            open( new_result_path, 'w' ) as result:
       for line1, line2 in zip( old_input, new_input ):
           result.write('{}, {}\n'.format(line1.rstrip(), line2.rstrip())
    old_result_path = new_result_path
    result_index = 1 - result_index

This will leave result1.txt and result2.txt lying about, which you may or may not care about cleaning up.