CODEWITHSUNDEEP
python-3.xpandastimestamppython-datetime
Kraxi
Kraxi

Reputation: 103

Pandas and datetime coercion. Can't convert whole column to Timestamp

So, I have an issue. Pandas keeps telling me that

'datetime.date' is coerced to a datetime. In the future pandas will not coerce, and a TypeError will be raised. To >retain the current behavior, convert the 'datetime.date' to a datetime with >'pd.Timestamp'. I'd like to get rid of this warning So until now I had a dataframe with some data, I was doing some filtration and manipulation. At some point I have a column with dates in string format. I don't care about timzeones etc. It's all about day accuracy. I'm getting a warning mentioned above, when I convert the strings to datetime, like below:

df['Some_date'] = pd.to_datetime(df['Some_date'], format='%m/%d/%Y')

So I tried to do something like that:

df['Some_date'] = pd.Timestamp(df['Some_date'])

But it fails as pd.Timestamp doesn't accept Series as an argument. I'm looking for a quickest way to convert those strings to Timestamp.

=====================================

EDIT

I'm so sorry, for confusion. I'm getting my error at another place. It happens when I try to filtrate my data like this: df = df[(df['Some_date'] > firstday)] Where firstday is being calculated basing on datetime. Like here:

import datetime

def get_dates_filter():
    lastday = datetime.date.today().replace(day=1) - datetime.timedelta(days=1)
    firstday = lastday.replace(day=1)
    return firstday, lastday

So probably the issue is comparing two different types of date representation

Upvotes: 1

Views: 3872

Answers (1)

jezrael
jezrael

Reputation: 863611

In pandas python dates are still poor supported, the best is working with datetimes with no times.

If there are python dates you can convert to strings before to_datetime:

df['Some_date'] = pd.to_datetime(df['Some_date'].astype(str))

If need remove times from datetimes in column use:

df['Some_date'] = pd.to_datetime(df['Some_date'].astype(str)).dt.floor('d')

Test:

rng = pd.date_range('2017-04-03', periods=3).date
df = pd.DataFrame({'Some_date': rng})  
print (df)
    Some_date
0  2017-04-03
1  2017-04-04
2  2017-04-05

print (type(df.loc[0, 'Some_date']))
<class 'datetime.date'>

df['Some_date'] = pd.to_datetime(df['Some_date'].astype(str))
print (df)
   Some_date
0 2017-04-03
1 2017-04-04
2 2017-04-05

print (type(df.loc[0, 'Some_date']))
<class 'pandas._libs.tslibs.timestamps.Timestamp'>

print (df['Some_date'].dtype)
datetime64[ns]

Upvotes: 1

Related Questions