CODEWITHSUNDEEP
pythoncsvpandas
Jonathan Roth
Jonathan Roth

Reputation: 1271

How to get the number of rows in a Pandas chunk?

I'm reading a huge csv file by iterating over chunks. How can I get the size of the currently processed chunk? Especially the last chunk may have smaller number of rows than defined with the parameter chunksize.

reader = pd.read_table('myFile.csv', sep=';', chunksize=100)

Upvotes: 6

Views: 3975

Answers (1)

jezrael
jezrael

Reputation: 863156

You need check length of DataFrame:

for x in reader:
    print (len(x.index))
    print (len(x))
    print (x.shape[0])

Upvotes: 6

Related Questions