CODEWITHSUNDEEP
pythonpandasdatetime
till Kadabra
till Kadabra

Reputation: 488

pandas - get a dataframe for every day

I have a DataFrame with dates in the index. I make a Subset of the DataFrame for every Day. Is there any way to write a function or a loop to generate these steps automatically?

import json
import requests
import pandas as pd
from pandas.io.json import json_normalize
import datetime as dt

#Get the channel feeds from Thinkspeak
response = requests.get("https://api.thingspeak.com/channels/518038/feeds.json?api_key=XXXXXX&results=500")

#Convert Json object to Python object
response_data = response.json()
channel_head = response_data["channel"]
channel_bottom = response_data["feeds"]

#Create DataFrame with Pandas
df = pd.DataFrame(channel_bottom)

#rename Parameters
df = df.rename(columns={"field1":"PM 2.5","field2":"PM 10"})

#Drop all entrys with at least on nan
df = df.dropna(how="any")

#Convert time to datetime object
df["created_at"] = df["created_at"].apply(lambda x:dt.datetime.strptime(x,"%Y-%m-%dT%H:%M:%SZ"))

#Set dates as Index
df = df.set_index(keys="created_at")

#Make a DataFrame for every day
df_2018_12_07 = df.loc['2018-12-07']
df_2018_12_06 = df.loc['2018-12-06']
df_2018_12_05 = df.loc['2018-12-05']
df_2018_12_04 = df.loc['2018-12-04']
df_2018_12_03 = df.loc['2018-12-03']
df_2018_12_02 = df.loc['2018-12-02']

Upvotes: 0

Views: 787

Answers (3)

Matina G
Matina G

Reputation: 1582

Supposing that you do that on the first day of next week (so, exporting monday to sunday next monday, you can do that as follows:

from datetime import date, timedelta
day = date.today() - timedelta(days=7)   # so, if today is monday, we start monday before
df = df.loc[today]
while day < today:
    df1 = df.loc[str(day)]
    df1.to_csv('mypath'+str(day)+'.csv')  #so that export files have different names 
    day = day+ timedelta(days=1)

Upvotes: 2

jezrael
jezrael

Reputation: 863236

You can create dictionary of DataFrames - then select by keys for DataFrame:

dfs = dict(tuple(df.groupby(df.index.strftime('%Y-%m-%d'))))

print (dfs['2018-12-07'])

Upvotes: 1

anky
anky

Reputation: 75100

you can use:

from datetime import date
today = str(date.today())
df = df.loc[today]

and schedule the script using any scheduler such as crontab.

Upvotes: 1

Related Questions