Blaztix
Blaztix

Reputation: 1294

Pandas group values and get mean by date range

I have a DataFrame like this

df = pd.DataFrame( data = numpy_data, columns=['value','date'])

    value   date
0   64.885  2018-01-11
1   74.839  2018-01-15
2   41.481  2018-01-17
3   22.027  2018-01-17
4   53.747  2018-01-18
...  ...        ...
514 61.017  2018-12-22
515 68.376  2018-12-21
516 79.079  2018-12-26
517 73.975  2018-12-26
518 76.923  2018-12-26

519 rows × 2 columns

And I want to plot this value vs date and I am using this

df.plot( x='date',y='value')

And I get this

enter image description here

The point here, this plot have to many fluctuation, and I want to soften this, my idea is group the values by date intervals and get the mean, for example 10 days, the mean between July 1 and July 10, and create de point in July 5

A long way is, get date range, separate in N ranges with start and end dates, filter data with date calculate the mean, and put in other DataFrame

Is there a short way to do that?

PD: Ignore the peaks

Upvotes: 0

Views: 3179

Answers (2)

Blaztix
Blaztix

Reputation: 1294

Based on yatu answer

The problem with his answer, is the rolling function considere values as index, not as date, with some transformations rolling can read Timestamp as use time as window [ pandas.rolling ]

df = pd.DataFrame( data = numpy_data, columns=['value','date'])

df['date'] = df.apply(lambda row: pd.Timestamp(row.date), axis=1 )
df = df.set_index(df.date).drop('date', axis=1)
df.sort_index(inplace=True)

df.rolling('10d').mean().plot( ylim=(30,100) , figsize=(16,5),grid='true')
Final results

Final result

Upvotes: 0

yatu
yatu

Reputation: 88236

One thing you could do for instance is to take the rolling mean of the dataframe, using DataFrame.rolling along with mean:

df = df.set_index(df.date).drop('date', axis=1)
df.rolling(3).mean().plot()

For the example dataframe you have, directly plotting the dataframe would result in:

enter image description here

And having taking the rolling mean, you would have:

enter image description here

Here I chose a window of 3, but his will depend on how wmooth you want it to be

Upvotes: 1

Related Questions