CODEWITHSUNDEEP
pythonpandasdatetime
Will.Evo
Will.Evo

Reputation: 1177

Prevent pandas from converting datetime objects into pandas Timestamps

I have a very simple problem. I want to parse datetime strings as python datetime objects and store them in a pandas dataframe. However, I can't figure out how to prevent pandas from converting the datetime object into a pandas Timestamp. For example:

import dateutil
import pandas as pd
from io import StringIO

time = '2020-11-03T15:55:21'

no_pandas = dateutil.parser.parse(time)
print (type(no_pandas))

#Create a simple csv
convert = {'Time': lambda x: dateutil.parser.parse(x)}
csv = StringIO(f'Time\n{time}')
df = pd.read_csv(csv, parse_dates=False, converters=convert)

print (type(df.iloc[0]['Time']))

which prints: <class 'datetime.datetime'> <class 'pandas._libs.tslibs.timestamps.Timestamp'>

Why is pandas overriding my preference for datetime objects and how can I prevent it from doing so?

Upvotes: 7

Views: 2592

Answers (3)

spikelucky
spikelucky

Reputation: 3

I can't even turn the timestamp back into my datetime afterwards but I have a workaround: save the datetime object as string and then turn it back into a datetime object when unpacking

df['Time'] = str(datetimeObject)
string = df.iloc[0]['Time']
datetimeObject = datetime.datetime.strptime(string, '%Y-%m-%d %H:%M:%S')

at least that works for me, if anyone has a more elegant solution i'm all ears

Upvotes: 0

Thiago Dantas
Thiago Dantas

Reputation: 61

For some reason pandas seems to override the type to pd.Timestamp. You should make sure pandas will read the column as a Series of dtype="object". So you should do something like:

df['Time'] = pd.Series(df['Time'].dt.to_pydatetime(), dtype="object")

Upvotes: 2

BENY
BENY

Reputation: 323226

After you load into pandas add to_pydatetime

df['Time'] = df['Time'].dt.to_pydatetime()

Out

type(pd.to_datetime(df['Time']).dt.to_pydatetime()[0])
Out[46]: datetime.datetime

Upvotes: 2

Related Questions