Pandas resample raises type error for index consisting of datetime objects

Question

I am trying to resample a very simple dataframe, which fails with the following exception:

TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'

I read the pandas API docs and looked at dozens of examples and I cannot figure out what I am doing wrong.

# %%
import pandas as pd
print(f"pandas version: {pd.__version__}\n\n")

data = pd.DataFrame({"created": ['2019-03-07T11:01:07.361+0000',
                                 '2019-06-05T15:09:51.203+0100',
                                 '2019-06-05T15:09:51.203+0100'],
                     "value": [10, 20, 30]})

# %%
print(f"original type: {type(data.created[0])}\n")
data.info()

# %%
data.created = pd.to_datetime(data.created)

# %%
print(f"updated type: {type(data.created[0])}\n")
data.info()

# %%
data.set_index("created", inplace=True)
data.info()

# %%
data.resample("D").mean()

Here is the result

pandas version: 0.24.2

original type: <class 'str'>

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 2 columns):
created    3 non-null object
value      3 non-null int64
dtypes: int64(1), object(1)
memory usage: 128.0+ bytes

updated type: <class 'datetime.datetime'>

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 2 columns):
created    3 non-null object
value      3 non-null int64
dtypes: int64(1), object(1)
memory usage: 128.0+ bytes

<class 'pandas.core.frame.DataFrame'>
Index: 3 entries, 2019-03-07 11:01:07.361000+00:00 to 2019-06-05 15:09:51.203000+01:00
Data columns (total 1 columns):
value    3 non-null int64
dtypes: int64(1)
memory usage: 48.0+ bytes

Traceback (most recent call last):
  File "c:/Users/me/dev/misc/index.py", line 32, in <module>
    data.resample("D").mean()
  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\generic.py", line 8155, in resample
    base=base, key=on, level=level)
  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\resample.py", line 1250, in resample
    return tg._get_resampler(obj, kind=kind)
  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\resample.py", line 1380, in _get_resampler
    "but got an instance of %r" % type(ax).__name__)
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'Index, but got an instance of 'Index'

Answer 1

Let's start from some principles:

1. To do a resample, the source Series or DataFrame must have e.g. a DatetimeIndex (not an "ordinary" index).

2. You could set_index to this column, but to do so, all Datetime elements must be in the same time zone (your data are not).

So you can proceed as follows:

1. While converting created column to Datetime (a part of your code), pass utc=True to "unify" the timezone:

   data.created = pd.to_datetime(data.created, utc=True)
   

2. Set the index and then you are free to resample:

   data.set_index('created').resample("D").mean()
   

Another option: Instead of set_index, you can pass on parameter, specifying a Datetime(-like) column:

data.resample("D", on='created').mean()

but this column still has to have all entries in the same time zone.