CODEWITHSUNDEEP
pythoncsv
newbie
newbie

Reputation: 95

append content of one csv file to another using python

I have 2 csv files:

output.csv has a 5 columns of titles.
output1.csv has about 40 columns of different types of data.

I need to append all the content of output1.csv to output.csv. How can I do this?
could somebody please give me a hint on how to go about it ???
i have the following code :

reader=csv.DictReader(open("test.csv","r"))
allrows = list(reader)

keepcols = [c for c in allrows[0] if all(r[c] != '0' for r in allrows)]

print keepcols
writer=csv.DictWriter(open("output.csv","w"),fieldnames='keepcols',extrasaction='ignore')
writer.writerows(allrows)

    with open("test1.csv","r") as f:
        fields=next(f).split()
        # print(fields)
        allrows=[]
        for line in f:
            line=line.split()
            row=dict(zip(fields,line))
            allrows.append(row)
            # print(row)
        keepcols = [c for c in fields if any(row[c] != '0' for row in allrows)]
        print keepcols
        writer=csv.DictWriter(open("output1.csv","w"),fieldnames=keepcols,extrasaction='ignore')
        writer.writerows(allrows)

test.csv generates output.csv
test1.csv generates output1.csv

i m trying to see if i can make both files generate my output in the same file..

Upvotes: 7

Views: 31365

Answers (4)

Timothee Heller
Timothee Heller

Reputation: 36

You could also use a generator from the reader if you want to pass a condition:

import csv
def read_generator(filepath:str):
   with open(filepath, 'rb'):
     reader = csv.reader(f)
     for row in reader:
       if row[0] == condition:
         yield row

and then write from that with:

writer = csv.writer(open("process.csv", "rb"))
write.writerow(read_generator(file_to_read.csv))

Upvotes: 0

vpit3833
vpit3833

Reputation: 7951

This recent discussion looks very similar to what you are looking for except that the OP there wanted to concatenate mp3 files.

EDIT:

import os, sys
target = '/path/to/target'
src1 = '/path/to/source1.csv'
src2 = '/path/to/source2.csv'
tf = open(target, 'a')
tf.write(open(src1).read())
tf.write(open(src2).read())
tf.close()

try this, this should work since you simply want to do the equivalent of cat src1 src2 > target of shell command

Upvotes: 4

Lior
Lior

Reputation: 2631

If I understand your question correctly, you want to create a csv with 41 columns - the 1 from output.csv followed by the 40 from output1.csv. I assume they have the same number of rows (if not - what is the necessary behavior?)

Try using the csv module:

import csv
reader = csv.reader(open('output.csv', 'rb'))
reader1 = csv.reader(open('output1.csv', 'rb'))
writer = csv.writer(open('appended_output.csv', 'wb'))
for row in reader:
    row1 = reader1.next()
    writer.writerow(row + row1)

If your csv files are formatted with special delimiters or quoting characters, you can use the optional keyword arguments for the csv.reader and csv.writer objects. See Python's csv module documentation for details...

EDIT: Added 'b' flag, as suggested.

Upvotes: 8

John Machin
John Machin

Reputation: 82924

"I need to append all the content of output1.csv to output.csv." ... taken literally that would mean write each row in the first file followed by each row in the second file. Is that what you want??

titles of what? the 40 columns in the other file?? If this is so, then assuming that you want the titles written as a row of column headings:

import csv
titles = [x[0] for x in csv.reader(open('titles.csv', 'rb'))]
writer = csv.writer(open('merged.csv', 'wb'))
writer.writerow(titles)
for row in csv.reader(open('data.csv', 'rb')):
    writer.writerow(row)

Upvotes: 0

Related Questions