CODEWITHSUNDEEP
pythoncsv
Todd Curry
Todd Curry

Reputation: 1083

Proper iteration and exit in python csv reader loop

I'm struggling a bit with two things:

  1. I've got a huge block of stuff to do under for row in reader that I'm worried about everything being executed properly. There seems to be some superstition about long instruction blocks, so I'm channeling that, ignorantly.

  2. I'm not sure how best to advance to the next record, or even if I have to do so. When I've reached an error condition, the line is written to a discard file, so I can really just wait until the "end" of the loop. But what do I need to do to say "next record, please?" Do I have to make an explicit next() at the end of the for row... loop?

Appreciate some guidance here. TC

with open('data.csv',rb) as f:
    reader = csv.reader(f)
    next(reader, None) # skip the header
    for row in reader:
        foo
        foo
        lots more foo
        if bar1 == errorvalue1:
            writediscard (row)
        else:
            if bar2 == errorvalue2:
                writediscard (row)
            else:
                writegoodstufftothedatabase(data)
   # in all cases, want to advance to the next record

Note that I've def writediscard and writegoodstufftothedatabase above. I can handle writing and committing writes to a sqlitedb. This darn iterating through a CSV is another story.

Upvotes: 0

Views: 1266

Answers (1)

C. Ross
C. Ross

Reputation: 31848

  1. The code should execute just fine. The "superstition" about long instruction blocks is not out of concern that something won't execute but rather that it is difficult to read, understand, maintain and modify. Consider breaking up your "foo" logic into defined functions and calling those.

  2. In your case the next record logic happens in the iterator for row in reader:. You don't need to call anything to get the next record, you only need to make sure you don't execute any code you don't want to on a failure, which you do with your conditional structure. You could use the continue to skip the code in the loop and return to the top, but that's not necessary the way you have your code structured.

Upvotes: 1

Related Questions