CODEWITHSUNDEEP
pythonpandas
Gere
Gere

Reputation: 12687

How to combine daily data with monthly data in Pandas?

I have daily data, and also monthly numbers. I would like to normalize the daily data by the monthly number - so for example the first 31 days of 2017 are all divided by the number corresponding to January 2017 from another data set.

import pandas as pd
import datetime as dt

N=100
start=dt.datetime(2017,1,1)
df_daily=pd.DataFrame({"a":range(N)}, index=pd.date_range(start, start+dt.timedelta(N-1)))

df_monthly=pd.Series([1, 2, 3], index=pd.PeriodIndex(["2017-1", "2017-2", "2017-3"], freq="M"))

df_daily["a"] / df_monthly # ???

I was hoping the time series data would align in a one-to-many fashion and do the required operation, but instead I get a lot of NaN.

How would you do this one-to-many data alignment correctly in Pandas?

I might also want to concat the data, in which case I expect the monthly data to duplicate values within one month.

Upvotes: 0

Views: 3387

Answers (2)

Tai
Tai

Reputation: 7994

You can extract the information with to_period('M') and then use map. 

df_daily["month"] = df_daily.index.to_period('M')
df_daily['a'] / df_daily["month"].map(df_monthly)

Without creating the month column, you can use

df_daily['a'] / df_daily.index.to_period('M').to_series().map(df_monthly)

Upvotes: 3

Bharath M Shetty
Bharath M Shetty

Reputation: 30605

You can create a temporary key from the index's month, then merge both the dataframe on the key i.e 

df_monthly = df_monthly.to_frame().assign(key=df_monthly.index.month)
df_daily = df_daily.assign(key=df_daily.index.month)

df_new = df_daily.merge(df_monthly,how='left').set_index(df_daily.index).drop('key',1)

            a    0
2017-01-01  0  1.0
2017-01-02  1  1.0
2017-01-03  2  1.0
2017-01-04  3  1.0
2017-01-05  4  1.0

For division you can then simply do :

df_new['b'] = df_new['a'] / df_new[0]

Upvotes: 1

Related Questions