CODEWITHSUNDEEP
pythonpandastimestampdayofmonth
Arsalan
Arsalan

Reputation: 373

finding first and last available days of a month in pandas

I have a pandas dataframe from 2007 to 2017. The data is like this:

date      closing_price
2007-12-03  728.73
2007-12-04  728.83
2007-12-05  728.83
2007-12-07  728.93
2007-12-10  728.22
2007-12-11  728.50
2007-12-12  728.51
2007-12-13  728.65
2007-12-14  728.65
2007-12-17  728.70
2007-12-18  728.73
2007-12-19  728.73
2007-12-20  728.73
2007-12-21  728.52
2007-12-24  728.52
2007-12-26  728.90
2007-12-27  728.90
2007-12-28  728.91
2008-01-05  728.88
2008-01-08  728.86
2008-01-09  728.84
2008-01-10  728.85
2008-01-11  728.85
2008-01-15  728.86
2008-01-16  728.89

As you can see, some days are missing for each month. I want to take the first and last 'available' days of each month, and calculate the difference of their closing_price, and put the results in a new dataframe. For example for the first month, the days will be 2007-12-03 and 2007-12-28, and the closing prices would be 728.73 and 728.91, so the result would be 0.18. How can I do this?

Upvotes: 0

Views: 1553

Answers (3)

Hunaphu
Hunaphu

Reputation: 701

Problem: Get first or last date of indexed dataframe

Solution: Resample the index and then extract the data.

lom    = pd.Series(x.index, index = x.index).resample('m').last()
xlast  = x[x.index.isin(lom)] # .resample('m').last() to get monthly freq

fom    = pd.Series(x.index, index = x.index).resample('m').first()
xfirst = x[x.index.isin(fom)]

Upvotes: 0

Hsgao
Hsgao

Reputation: 563

you can group df by month and apply a function to do it. Notice the to_period, this function convert DataFrame from DatetimeIndex to PeriodIndex with desired frequency.

def calculate(x):
    start_closing_price = x.loc[x.index.min(), "closing_price"]
    end_closing_price = x.loc[x.index.max(), "closing_price"]
    return end_closing_price-start_closing_price

result = df.groupby(df["date"].dt.to_period("M")).apply(calculate)

# result
date
2007-12    0.18
2008-01    0.01
Freq: M, dtype: float64

Upvotes: 3

ALollz
ALollz

Reputation: 59519

First make sure they are datetime and sorted:

import pandas as pd

df['date'] = pd.to_datetime(df.date)
df = df.sort_values('date')

Groupby

gp = df.groupby([df.date.dt.year.rename('year'), df.date.dt.month.rename('month')])
gp.closing_price.last() - gp.closing_price.first()

#year  month
#2007  12       0.18
#2008  1        0.01
#Name: closing_price, dtype: float64

or

gp = df.groupby(pd.Grouper(key='date', freq='1M'))
gp.last() - gp.first()

#            closing_price
#date                     
#2007-12-31           0.18
#2008-01-31           0.01

Resample

gp = df.set_index('date').resample('1M')
gp.last() - gp.first()

#            closing_price
#date                     
#2007-12-31           0.18
#2008-01-31           0.01

Upvotes: 2

Related Questions