CODEWITHSUNDEEP
pythonpython-2.7datetimepandasfilter
Edamame
Edamame

Reputation: 25376

pandas filter date time: TypeError: can't compare offset-naive and offset-aware datetimes

I have a pandas dataframe:

    name    my_timestamp
------------------------------------------
0   a1      2016-07-28 09:27:07.536963-07:00
1   a2      2016-07-28 09:27:07.536963-07:00
2   a3      2016-08-15 13:05:54.924185-07:00
3   a4      2016-08-30 04:04:18.971667-07:00
4   a5      2016-03-22 14:36:22.999825-07:00
5   a6      2016-08-30 04:04:18.971667-07:00

I am trying to filter some rows in my pandas data frame like below:

import datetime
my_df[my_df.my_timestamp > datetime.datetime(2016, 7, 1)]

But get the following errors:

TypeErrorTraceback (most recent call last)
<ipython-input-21-35be746f191d> in <module>()
      1 import datetime
----> 2 my_df[my_df.my_timestamp > datetime.datetime(2016, 7, 1)]

/usr/local/lib/python2.7/dist-packages/pandas/core/ops.pyc in wrapper(self, other, axis)
    761                 other = np.asarray(other)
    762 
--> 763             res = na_op(values, other)
    764             if isscalar(res):
    765                 raise TypeError('Could not compare %s type with Series' %

/usr/local/lib/python2.7/dist-packages/pandas/core/ops.pyc in na_op(x, y)
    681                     result = lib.vec_compare(x, y, op)
    682             else:
--> 683                 result = lib.scalar_compare(x, y, op)
    684         else:
    685 

pandas/lib.pyx in pandas.lib.scalar_compare (pandas/lib.c:14261)()

TypeError: can't compare offset-naive and offset-aware date times

It seems to be the timezone issue. What would be the best way to ignore time-zone here? Thanks!

Upvotes: 2

Views: 1440

Answers (1)

Dennis Golomazov
Dennis Golomazov

Reputation: 17349

Assuming that all timestamps in the dataframe are in the same timezone:

tz_info = my_df.iloc[0].my_timestamp.tzinfo
my_df[my_df.my_timestamp > datetime.datetime(2016, 7, 1, tzinfo=tz_info)]

Upvotes: 3

Related Questions