Pandas DataFrame ValueError is not in range

Question

In this script I'm reading in a csv file and doing some basic calculations, to create another csv file with all of this modified data. The problem I'm running into is that I'm getting a ValueError saying "-1" is not in range. The "Condition" can either be 0 or -1, when the condition changes from a 0 to -1 the script is grabbing all the data points needed for the calculations in the row and storing them into arrays for later use. This CSV file usually starts and ends with the condition in the 0 position, however it might not always be the case as this file is constantly updated. So this might not always be the case. Can anyone see why I'm getting an out of range error? I'm not very used to python, so there could be a better way of doing this, any help on this is much appreciated. Also please ignore whitespace errors in the code below, the actual code has none.

import pandas as pd

df = pd.read_csv(csv_file_path)


#clean csv file, drop empty rows
df.dropna(subset = [list_of_columns], inplace=True)

#reset index
df.reset_index(drop=True, inplace=True)

#create empty lists for each field 

avg_m_temp = []

avg_temp_A = []
avg_temp_B = []
avg_temp_C = []

p_delta_A =[]
p_delta_B =[]
p_delta_C =[]


avg_temp_tank_A = []
avg_temp_tank_B = []
avg_temp_tank_C = []


for i in range(len(df)):


try:
    
    #check for the start of a pour condition changes from 0 to -1
    if df[condition][i] == 0 and df[condition][i+1] == -1:
        
        m_temp = []
        
        temp_A = []
        temp_B = []
        temp_C = []
        
        idle_A = df['Press_1'][i]
        idle_B = df['Press_2'][i]
        idle_C = df['Press_3'][i]

        A_arr=[]
        B_arr=[]
        C_arr=[]

        avg_A = []
        avg_B = []
        avg_C = []
        
        tank_A = []
        tank_B = []
        tank_C = []
        
        continue
        
    #check that condition is continuing     
    elif df[condition][i] == -1 and df[condition][i+1] == -1:
        
        m_temp.append(df[''][i])
        
        temp_A.append(df['temp_1'][i])
        temp_B.append(df['temp_2'][i])
        temp_C.append(df['temp_3'][i])
        
        A_arr.append(df['Press_1'][i])
        B_arr.append(df['Press_2'][i])
        C_arr.append(df['Press_3'][i])
        
        tank_A.append(df['tank_1'][i])
        tank_B.append(df['tank_2'][i])
        tank_C.append(df['tank_3'][i])
        
        
        continue

    #check if condition is finished either loop ends or condition changes from -1 to 0    
    elif df[condition][i] == 0 and df[condition][i-1] == -1:
        
        avg_m_temp.append((sum(m_temp)/len(m_temp)))
        
        avg_temp_A.append((sum(temp_A)/len(temp_A)))
        avg_temp_B.append((sum(temp_B)/len(temp_B)))
        avg_temp_C.append((sum(temp_C)/len(temp_C)))
        
        avg_A = sum(A_arr)/len(A_arr)
        avg_B = sum(B_arr)/len(B_arr)
        avg_C = sum(C_arr)/len(C_arr)

        delta_A = idle_A - avg_A
        delta_B = idle_B - avg_B
        delta_C = idle_C - avg_C
        
        p_delta_A.append(delta_A)
        p_delta_B.append(delta_B)
        p_delta_C.append(delta_C)
    
        avg_temp_tank_A.append((sum(tank_A)/len(tank_A)))
        avg_temp_tank_B.append((sum(tank_B)/len(tank_B)))
        avg_temp_tank_C.append((sum(tank_C)/len(tank_C)))
    
        continue
        
except:
    
    
    avg_m_temp.append((sum(m_temp)/len(m_temp)))
        
        avg_temp_A.append((sum(temp_A)/len(temp_A)))
        avg_temp_B.append((sum(temp_B)/len(temp_B)))
        avg_temp_C.append((sum(temp_C)/len(temp_C)))
        
        avg_A = sum(A_arr)/len(A_arr)
        avg_B = sum(B_arr)/len(B_arr)
        avg_C = sum(C_arr)/len(C_arr)

        delta_A = idle_A - avg_A
        delta_B = idle_B - avg_B
        delta_C = idle_C - avg_C
        
        p_delta_A.append(delta_A)
        p_delta_B.append(delta_B)
        p_delta_C.append(delta_C)
    
        avg_temp_tank_A.append((sum(tank_A)/len(tank_A)))
        avg_temp_tank_B.append((sum(tank_B)/len(tank_B)))
        avg_temp_tank_C.append((sum(tank_C)/len(tank_C)))

    continue

Answer 1

when using

for i in range(len(df)):

the first i is 0, so when you sometimes get to this check

elif df[condition][i] == 0 and df[condition][i-1] == -1:

it's really

elif df[condition][**0**] == 0 and df[condition][**-1**] == -1:

so the error doesn't care that you are testing 0 or -1, it doesn't like that you trying to find index -1, which doesn't exist, so it's out of range.

consider reworking your code with the following or something similar. You'll have to modify the if and elif statements so the right row references exist

for i in range(1, len(df))

or if needed

for i in range(1, len(df)-1)