CODEWITHSUNDEEP
python
user17451
user17451

Reputation: 665

How to read and write multiple files?

I want to write a program for this: In a folder I have n number of files; first read one file and perform some operation then store result in a separate file. Then read 2nd file, perform operation again and save result in new 2nd file. Do the same procedure for n number of files. The program reads all files one by one and stores results of each file separately. Please give examples how I can do it.

Upvotes: 18

Views: 103572

Answers (8)

tldr
tldr

Reputation: 116

I know I saw this double with open() somewhere but couldn't remember where. So I built a small example in case someone needs.

""" A module to clean code(js, py, json or whatever) files saved as .txt files to 
be used in HTML code blocks.  """
from os import listdir
from os.path import abspath, dirname, splitext
from re import sub, MULTILINE

def cleanForHTML():
    """ This function will search a directory text files to be edited. """

    ## define some regex for our search and replace. We are looking for <, > and &
    ## To replaced with &ls;, &gt; and &amp;. We might want to replace proper whitespace
    ## chars to as well? (r'\t', '    ') and (f'\n', '<br>')
    search_ = ((r'(<)', '&lt;'), (r'(>)', '&gt;'), (r'(&)', '&amp;'))

    ## Read  and loop our file location. Our location is the same one that our python file is in.
    for loc in listdir(abspath(dirname(__file__))):

        ## Here we split our filename into it's parts ('fileName', '.txt')
        name = splitext(loc)

        if name[1] == '.txt':
            ## we found our .txt file so we can start file operations.
            with open(loc, 'r') as file_1, open(f'{name[0]}(fixed){name[1]}', 'w') as file_2:

                ## read our first file
                retFile = file_1.read()

                ## find and replace some text.
                for find_ in search_:
                    retFile = sub(find_[0], find_[1], retFile, 0, MULTILINE)

                ## finally we can write to our newly created text file.
                file_2.write(retFile)

Upvotes: 0

codearena
codearena

Reputation: 11

This thing also works for reading multiple files, my file name is fedaralist_1.txt and federalist_2.txt and like this, I have 84 files till fedaralist_84.txt

And I'm reading the files as f.

for file in filename:
        with open(f'federalist_{file}.txt','r') as f:
             f.read()

Upvotes: 0

Alex Coventry
Alex Coventry

Reputation: 70819

You may find the fileinput module useful. It is designed for exactly this problem.

Upvotes: 9

pramod
pramod

Reputation: 1

from pylab import * 
import csv 
import os 
import glob 
import re 
x=[] 
y=[]

f=open("one.txt",'w')

for infile in glob.glob(('*.csv')):
    #   print "" +infile
    csv23=csv2rec(""+infile,'rb',delimiter=',')
    for line in csv23:      
        x.append(line[1])
        #  print len(x)
    for i in range(3000,8000):
        y.append(x[i])
    print ""+infile,"\t",mean(y)
    print >>f,""+infile,"\t\t",mean(y)
    del y[:len(y)]
    del x[:len(x)]

Upvotes: 0

monkut
monkut

Reputation: 43832

I've just learned of the os.walk() command recently, and it may help you here. It allows you to walk down a directory tree structure.

import os
OUTPUT_DIR = 'C:\\RESULTS'
for path, dirs, files in os.walk('.'):
    for file in files:
        read_f = open(os.join(path,file),'r')
        write_f = open(os.path.join(OUTPUT_DIR,file))

        # Do stuff

Upvotes: 2

michaeljoseph
michaeljoseph

Reputation: 7153

Combined answer incorporating directory or specific list of filenames arguments:

import sys
import os.path
import glob

def processFile(filename):
    fileHandle = open(filename, "r")
    for line in fileHandle:
        # do some processing
        pass
    fileHandle.close()

def outputResults(filename):
    output_filemask = "out"
    fileHandle = open("%s.%s" % (filename, output_filemask), "w")
    # do some processing
    fileHandle.write('processed\n')
    fileHandle.close()

def processFiles(args):
    input_filemask = "log"
    directory = args[1]
    if os.path.isdir(directory):
        print "processing a directory"
        list_of_files = glob.glob('%s/*.%s' % (directory, input_filemask))
    else:
        print "processing a list of files"
        list_of_files = sys.argv[1:]

    for file_name in list_of_files:
        print file_name
        processFile(file_name)
        outputResults(file_name)

if __name__ == '__main__':
    if (len(sys.argv) > 1):
        processFiles(sys.argv)
    else:
        print 'usage message'

Upvotes: 1

Mapad
Mapad

Reputation: 8537

I think what you miss is how to retrieve all the files in that directory. To do so, use the glob module. Here is an example which will duplicate all the files with extension *.txt to files with extension *.out

import glob

list_of_files = glob.glob('./*.txt')           # create the list of file
for file_name in list_of_files:
  FI = open(file_name, 'r')
  FO = open(file_name.replace('txt', 'out'), 'w') 
  for line in FI:
    FO.write(line)

  FI.close()
  FO.close()

Upvotes: 19

Matthew Scharley
Matthew Scharley

Reputation: 132244

import sys

# argv is your commandline arguments, argv[0] is your program name, so skip it
for n in sys.argv[1:]:
    print(n) #print out the filename we are currently processing
    input = open(n, "r")
    output = open(n + ".out", "w")
    # do some processing
    input.close()
    output.close()

Then call it like:

./foo.py bar.txt baz.txt

Upvotes: 13

Related Questions