CODEWITHSUNDEEP
pythonpython-2.7csvparsing
Dr.Tautology
Dr.Tautology

Reputation: 462

Python csv reader incomplete file line iteration

Here is my problem. I need to parse a comma separated file and I've got my code working how I would like, however while testing it and attempting to break things I've come across a problem.

Here is the example code: 

import csv
compareList=["testfield1","testfield2","testfield3","testfield4"]
z=open("testFile",'r')
x=csv.reader(z,quotechar='\'')
testDic={}
iter=0
for lineList in x:
    try:
        for item in compareList:
            testDic[item]=lineList[iter]
            iter+=1
        iter=0
    except IndexError:
        iter=0
        lineList=[]
        for item in compareList:
            testList.append("")
            testDic[item]=lineList[iter]
            iter+=1
        iter=0

    for item in compareList:
        testFile.write(testDic[item])
        if compareList.index(item)!=len(compareList)-1
            testFile.write(",")
    testFile.write('\n')
testFile.close()
z.close()

So what this is supposed to do is check and make sure that each line of the csv file matches the length of a list. If the length of the line does not match the length of the list, then the line is converted to null values(commas) that equal the length of compareList. Here is an example of what is in the file:

,,"sometext",343434
,,"moretext",343434
,,"stuff",4543343
,,"morestuff",3434354

The code works just fine if the line is missing an item. So the output of at file containing:

,"sometext",343434
,,"moretext",343434
,,"stuff",4543343
,,"morestuff",3434354

will look like this:

,,,,
,,"moretext",343434
,,"stuff",4543343
,,"morestuff",3434354

The problem I have induced is when the line looks something like this:

,"sometext",343434
,,"moretext",343434
,,"St,'",uff",4543343
,,"morestuff",3434354

The ouput of this file will be:

,,,,
,,"moretext",343434
,,,,

So it will apply the change as expected and null out lines 1 and 3, but it just stops processing at that line. I've been pulling my hair out trying to figure out what is going on here, with no luck.

As always I greatly appreciate any help you are willing to give.

Upvotes: 1

Views: 757

Answers (1)

Thomas Lehoux
Thomas Lehoux

Reputation: 1196

Just print each line returned by csv.reader to understand what is the problem:

>>> import csv
>>> z=open("testFile",'r')
>>> x=csv.reader(z,quotechar='\'')
>>> for lineList in x:
...     print lineList
...
['', '"sometext"', '343434']
['', '', '"moretext"', '343434']
['', '', '"St', '",uff",4543343\n,,"morestuff",3434354\n']

The last 2 lines are just one line for csv.reader. Now, just remove quotechar='\''

>>> import csv
>>> z=open("testFile",'r')
>>> x=csv.reader(z)
>>> for lineList in x:
...     print lineList
...
['', 'sometext', '343434']
['', '', 'moretext', '343434']
['', '', "St,'", 'uff"', '4543343']
['', '', 'morestuff', '3434354']

Upvotes: 1

Related Questions