CODEWITHSUNDEEP
pythonpandas
Igor K.
Igor K.

Reputation: 877

Start iterating dataframe from a specific row

My DataFrame:

                  dt_object     price
0       1990-01-01 00:00:00  1.050067
1       1990-01-01 00:05:00  1.049700
2       1990-01-01 00:10:00  1.049933
3       1990-01-01 00:15:00  1.049733
4       1990-01-01 00:20:00  1.050033

I iterate it using

for index, row in df.iterrows():
    if index >= 9000000:
       #do_the_job

The data file is very large. I need to iterate over from index 9000000. Can I start iterating over once from this index? It takes a long time before the check from 0 to 9000000 passes.
I cannot truncate data from 0 to 9000000 because I am doing the calculation using past data. For the current row the result is calculated based on past values.

Upvotes: 0

Views: 1802

Answers (3)

Rawan AlKhalawi
Rawan AlKhalawi

Reputation: 43

df = pandas.read_excel(file_path, engine='openpyxl', usecols=[col])
for i, row in df[239:].iterrows():
    for j, column in row.iteritems():

Upvotes: 0

Alfredo Maussa
Alfredo Maussa

Reputation: 545

You could "pre-index" in the for loop:

Instead calling df, you could just call df[idx_start:]

for index, row in df[idx_start:].iterrows():
    #do_the_job

Note Slicing in pandas take the columns, then instead df[idx_start:] it should be df.loc[key_start:,:] (with key map, it could be different from numbers) or df.loc[idx_start:,:] (with indexing from zero)

Upvotes: 1

Shiva
Shiva

Reputation: 2838

You can slice the DataFrame:

for index, row in df.iloc[9000000:].iterrows():
    pass

Upvotes: 2

Related Questions