CODEWITHSUNDEEP
pythonpandasdataframe
Muhammad
Muhammad

Reputation: 305

Storing csv file's contents into data Frames [Python Pandas]

I have written a function that reads csv files, stores them as a data frame and resample them on hourly basis. Below is my code;

def ABC(path1,path2):
    df1=pd.read_csv(path1, sep='\t',names = ["Datetime", "Value"])
    df2=pd.read_csv(path2, sep='\t',names = ["Datetime", "Value"])
    df1['Datetime']=pd.to_datetime(df1['Datetime'])
    df1=df1.set_index('Datetime')
    df1=df1.resample('H',how='sum')
    df2['Datetime']=pd.to_datetime(df2['Datetime'])
    df2=df2.set_index('Datetime')
    df2=df2.resample('H',how='sum')
    ABC = pd.DataFrame(df1['Value'] + df2['Value'])
    ABCD = ABC * 0.519
    return ABC, ABCD
ABC, ABCD= ABC('C:\Users\Desktop\B1.tsv'
                             ,'C:\Users\Desktop\B2.tsv')

This program works well but what if I have 30 file paths then it will be difficult to make 30 data frames and do this process. I was thinking of following way of doing the above;

def ABC():
    Path= ['B1','B2','B3']
    general = []
    for i in Path:
        url = ('C:\Users\Desktop\%s.tsv'%i)
        X = pd.read_csv(url,sep='\t',names = ["Datetime", "Value"])
        X['Datetime']=pd.to_datetime(X['Datetime'])
        X=X.set_index('Datetime')
        X=X.resample('H',how='sum')
        general.append(X)
        return general
df=ABC()
print df

The above code just output one dataFrame and not outputting what the first script is doing. Any idea what I am doing wrong?

Upvotes: 1

Views: 234

Answers (1)

IanS
IanS

Reputation: 16241

You return too early (after the very first iteration). It's an indentation problem. The function should read:

def ABC():
    Path= ['B1','B2','B3']
    general = pd.DataFrame()
    for i in Path:
        url = ('C:\Users\Desktop\%s.tsv'%i)
        X = pd.read_csv(url,sep='\t',names = ["Datetime", "Value"])
        X['Datetime']=pd.to_datetime(X['Datetime'])
        X=X.set_index('Datetime')
        X=X.resample('H',how='sum')
        if len(general) == 0:
            general = X
        else:
            general = general['Values'] + X['Values']
    return general

Notice the indentation of the last line.

EDIT: added code to sum the dataframes inside the loop as requested in a comment.

First, create an empty dataframe called general. At the first iteration (when the length of the empty dataframe is 0), assign the current dataframe X to general. At subsequent iterations, add the values of the current dataframe to the sum of all previous dataframes' values, stored in general.

Upvotes: 1

Related Questions