CODEWITHSUNDEEP
pythoncsvwriter
Drublic
Drublic

Reputation: 709

Python, empty file after csv writer.. again

My python program loops through a bunch of csv-files, read them, and write specific columns in the file to another csv file. While the program runs, i can see the files being written in the correct manner, but once the program is finished, all the files i've just written become empty.

The solution to all the other similar threads seems to be closing the file you write to properly, but i cant seem to figure out what im doing wrong. Anyone?

import os
import csv

def ensure_dir(f):
    d = os.path.dirname(f)
    if not os.path.exists(d):
        os.makedirs(d)

readpath = os.path.join("d:\\", "project")
savepath=os.path.join("d:\\", "save")
ensure_dir(savepath)
contents_1=os.listdir(readpath)
for i in contents_1[1:len(contents_1)]:
    readpath_2=os.path.join(readpath, i)
    if os.path.isdir(readpath_2)== True :
        contents_2=os.listdir(readpath_2)
        for i in contents_2:
            readpath_3=os.path.join(readpath_2, i)
            if os.path.isfile(readpath_3)== True :
                savefile=savepath + "\\" + i
                savefile = open(savefile, 'wb')
                writer = csv.writer(savefile, delimiter=';')
                readfile=open(readpath_3, 'rb')
                reader = csv.reader(readfile, delimiter=';')
                try:
                    for row in reader:
                        writer.writerow([row[0], row[3]])
                except:
                    print(i)
                finally:
                    savefile.close()
                    readfile.close()

Upvotes: 2

Views: 2358

Answers (3)

Steven Rumbalski
Steven Rumbalski

Reputation: 45552

savefile=savepath + "\\" + i is the error. If both "d:\\project\a\x.csv" and "d:\\project\b\x.csv" exist, then you will write to savepath + "\\" + i more than once. If the second path as an empty "x.csv", then it would overwrite the result with an empty file.

Try this instead:

import os
import csv

def ensure_dir(f):
    d = os.path.dirname(f)
    if not os.path.exists(d):
        os.makedirs(d)

readpath = os.path.join("d:\\", "project")
savepath = os.path.join("d:\\", "save")

ensure_dir(savepath)

for dname in os.listdir(readpath)[1:]:
    readpath_2 = os.path.join(dname, fname)
    if not os.path.isdir(readpath_2):
        continue
    for fname in os.listdir(readpath_2)
        fullfname = os.path.join(readpath_2, fname)
        if not os.path.isfile(fullfname):
            continue
        savefile = open(savepath + "\\" + dname + "_" + fname, wb)
        writer = csv.writer(savefile, delimiter=';')
        readfile=open(fullfname, 'rb')
        reader = csv.reader(readfile, delimiter=';')
        try:
            for row in reader:
                writer.writerow([row[0], row[3]])
        except:
            print(i)
        finally:
            savefile.close()
            readfile.close()

This code could be greatly improved with os.walk

Upvotes: 2

John Machin
John Machin

Reputation: 82992

(1) Your outer loop AND your inner loop both use i as the loop variable. This has no hope of (a) being understood by a human (b) working properly.

(2) except: print(i) ... What??? I'd suggest you remove the try/except and fix any bugs that you come across.

Upvotes: 0

Zack Bloom
Zack Bloom

Reputation: 8417

Quoting from the python documentation:

If csvfile is a file object, it must be opened with the ‘b’ flag on platforms where that makes a difference.

Change the 'w' and 'r' flags to 'wb' and 'rb'.

Upvotes: 0

Related Questions