CODEWITHSUNDEEP
pythonpandasdatetimemean
mortysporty
mortysporty

Reputation: 2889

Why does pandas mean, on datetime, work on a series but not on a groupby object

I am trying to take the mean of dates in by groups.

import pandas as pd

df = pd.DataFrame({'Id': ['A', 'A', 'B', 'B'],
                   'Date': [pd.datetime(2000, 12, 31), pd.datetime(2002, 12, 31),
                            pd.datetime(2000, 6, 30), pd.datetime(2002, 6, 30)]})

This has always been a pain to do, so I was pleased to learn that this had apparntly been fixed in pandas 0.25 Datetime objects with pandas mean function.

df['Date'].mean()
Out[45]: Timestamp('2001-09-30 00:00:00') # This works

However, this cant be done using ´groupby´

df.groupby('Id')['Date'].mean()

Traceback (most recent call last):

  File "<ipython-input-46-5fae5ffac6c6>", line 1, in <module>
    df.groupby('Id')['Date'].mean()

  File "C:\Users\xxx\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 1205, in mean
"mean", alt=lambda x, axis: Series(x).mean(**kwargs), **kwargs

  File "C:\Users\xxx\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 888, in _cython_agg_general
raise DataError("No numeric types to aggregate")

DataError: No numeric types to aggregate

What is going on here, and is there an easy workaround?

Upvotes: 1

Views: 487

Answers (1)

jezrael
jezrael

Reputation: 863531

Use lambda function with GroupBy.agg or GroupBy.apply:

print (df.groupby('Id')['Date'].agg(lambda x: x.mean()))
print (df.groupby('Id')['Date'].agg(pd.Series.mean))
print (df.groupby('Id')['Date'].apply(lambda x: x.mean()))
print (df.groupby('Id')['Date'].apply(pd.Series.mean))

Id
A   2001-12-31
B   2001-06-30
Name: Date, dtype: datetime64[ns]

Difference is if multiple columns:

df = pd.DataFrame({'Id': ['A', 'A', 'B', 'B'],
                   'Date': [pd.datetime(2000, 12, 31), pd.datetime(2002, 12, 31),
                            pd.datetime(2000, 6, 30), pd.datetime(2002, 6, 30)]})
df['Date1'] = df['Date']
print (df.groupby('Id').agg(lambda x: x.mean()))
         Date      Date1
Id                      
A  2001-12-31 2001-12-31
B  2001-06-30 2001-06-30
print (df.groupby('Id').agg(pd.Series.mean))
         Date      Date1
Id                      
A  2001-12-31 2001-12-31
B  2001-06-30 2001-06-30

print (df.groupby('Id').apply(lambda x: x.mean()))
Empty DataFrame
Columns: []
Index: []

print (df.groupby('Id').apply(pd.Series.mean))
Empty DataFrame
Columns: []
Index: []

Why does pandas mean, on datetime, work on a series but not on a groupby object

Some time ago it was problem with mean for Series, Datetimes, check this, so possible in some next versions of pandas this should working well.

Upvotes: 2

Related Questions