CODEWITHSUNDEEP
pythonloopscsvdataframejupyter-notebook
Renaldo Moon
Renaldo Moon

Reputation: 165

Looping through Multiple CSV files and producing Multiple Outputs

I am writing some python script that opens a .csv file, defines the dataframe, run some analysis (e.g. aggregate data, splitting columns, finding averages etc..) and plot the output of the analysis on a graph. The outputs will be both a graph (.png file) AND a csv file with the word "_ANALYSIS" added to the original file name at the end.

I have set it up as a loop function in Jupyter Notebook: 

#import multiple csv files

import glob
import pandas as pd
import numpy as np
from pytz import all_timezones
import matplotlib.pyplot as plt

files = glob.glob('folder/*.csv')
for file in files:
    df = pd.read_csv(file)

    #START OF THE ANALYSIS
    #Multiple lines of code starts here


    #GRAPH some outputs from the analysis
    df2 = df.replace(0, np.nan)
    fig, ax = plt.subplots()
    df2.groupby('Day_type').plot(x = 'Time', y = 'avg_vt', ax=ax, grid=True)


    #OUTPUT FILES: graph + csv file
    plt.savefig('*.png', index = False)
    file_name="file"+str(i+1)+"_ANALYSIS"
    df.to_csv('file1_ANALYSIS.csv', index = False)

Unfortunately, it isn't producing any outputs. There is no problem with the analysis code itself as I tried it before I added the loop function.

Thanks, R

Upvotes: 2

Views: 7203

Answers (1)

Joost Döbken
Joost Döbken

Reputation: 4017

slightly more elegant with pathlib

from pathlib import Path

folder="C:\Users\Renaldo.Moonu\Desktop\folder name"
for file in Path(folder).glob('*.csv'):
    df = pd.read_csv(file)
    df.fillna(0, inplace=True)
    fig, ax = plt.subplots()
    df.groupby('Day_type').plot(x = 'Time', y = 'avg_vt', ax=ax, grid=True)

    plt.savefig(file.with_suffix('.png'), index = False)
    df.to_csv(file.with_suffix('.csv'), index = False)

Upvotes: 2

Related Questions