CSV file row incrementing

Question

Currently I am working on a CSV file program where I have to partition the CSV file into non overlapping parts (row-wise), but should not create any extra files in the disk.

Currently I am using this program to read particular parts of the CSV file row-wise:

def dataFromFile(fname):
    record = []
    count=0
    with open(fname, 'rb') as f:
         reader = csv.reader(f)
         for row in reader:
             while '' in row:
                   row.remove('')
             count+=1
             record.append(row)
    print count
    return record[0:4]

Here I am able to read the rows within the range 0-4. But is there a possibility to automate this i.e can we without giving that range manually everytime read the next 4 rows and so-on till the end of CSV file?

The CSV file contains:

apple   beer    rice    chicken
apple   beer    rice    
apple   beer        
apple   mango       
milk    beer    rice    chicken
milk    beer    rice    
milk    beer        
milk    mango       

Answer 1

You can use yield:

def dataFromFile(fname):
    record = []
    print'Opening the file is executed once'
    count = 0
    with open(fname) as f:
        for row in csv.reader(f, delimiter=' '):
            count += 1
            fields = [field for field in row if field]
            if fields:
                record.append(fields)
            if len(record) == 4:
                print 'Last yielded row:', count
                yield(record)
                record = []
    if record:
        yield record

And the call:

for row in dataFromFile('your.csv'):
   print row

From the interpreter

>>> import csv
>>> 
>>> def dataFromFile(fname):
    record = []
    print 'Opening the file is executed once'
    count = 0
    with open(fname) as f:
        for row in csv.reader(f, delimiter=' '):
            count += 1

            fields = [field for field in row if field]
            if fields:
                record.append(fields)
            if len(record) == 4:
                print 'Last yielded row:', count
                yield(record)
                record = []
    if record:
        yield record 
... 
>>> for row in dataFromFile('your.csv'):
...     print row
... 
Opening the file is executed once
Last yielded row: 4
[['apple', 'beer', 'rice', 'chicken'], ['apple', 'beer', 'rice'], ['apple', 'beer'], ['apple', 'mango']]
Last yielded row: 8
[['milk', 'beer', 'rice', 'chicken'], ['milk', 'beer', 'rice'], ['milk', 'beer'], ['milk', 'mango']]
>>>