CODEWITHSUNDEEP
pythonpandasdatetimetime-series
chintan s
chintan s

Reputation: 6488

Extract date and time from pandas timestamp

I have a timestamp column where the timestamp is in the following format

2016-06-16T21:35:17.098+01:00

I want to extract date and time from it. I have done the following:

import datetime as dt

df['timestamp'] = df['timestamp'].apply(lambda x : pd.to_datetime(str(x)))

df['dates'] = df['timestamp'].dt.date

This worked for a while. But suddenly it does not.

If I again do df['dates'] = df['timestamp'].dt.date I get the following error 

Can only use .dt accessor with datetimelike values

Luckily, I have saved the data frame with dates in the csv but I now want to create another column time in the format 23:00:00.051

EDIT

From the raw data file (15 million samples), the timestamp column looks like following (first 5 samples):

            timestamp

0           2016-06-13T00:00:00.051+01:00
1           2016-06-13T00:00:00.718+01:00
2           2016-06-13T00:00:00.985+01:00
3           2016-06-13T00:00:02.431+01:00
4           2016-06-13T00:00:02.737+01:00

After the following command 

df['timestamp'] = df['timestamp'].apply(lambda x : pd.to_datetime(str(x)))

the timestamp column looks like with dtype as dtype: datetime64[ns]

0    2016-06-12 23:00:00.051
1    2016-06-12 23:00:00.718
2    2016-06-12 23:00:00.985
3    2016-06-12 23:00:02.431
4    2016-06-12 23:00:02.737

Then finally

df['dates'] = df['timestamp'].dt.date

0           2016-06-12
1           2016-06-12
2           2016-06-12
3           2016-06-12
4           2016-06-12

EDIT 2

Found the mistake. I had cleaned the data and saved the data frame in a csv file, so I don't have to do the cleaning again. When I read the csv, the timestamp dtype changes to object. Now how do I fix this?

Upvotes: 43

Views: 161613

Answers (5)

Muyiwa J. Obadara
Muyiwa J. Obadara

Reputation: 159

This are simple lines that works for me right now

# To extract the date
df["date"] = pd.to_datetime(df["timestamp"]).dt.strftime("%d-%m-%Y")

# Extract Time
df["time"] = pd.to_datetime(df["timestamp"]).dt.time

Source DataCamp.

Upvotes: 0

Colin Anthony
Colin Anthony

Reputation: 1227

You can use pandas built-in to_datetime object for this

df['timestamp'] = pd.to_datetime(df['timestamp'])
df['date'] = df['timestamp'].dt.date
df['time'] = df['timestamp'].dt.time

Upvotes: 3

sudesh
sudesh

Reputation: 43

When you are importing your csv, then use parse_dates parameter of pandas.read_csv(). For example, to import a column utc_datetime as datetime:

parse_dates = ['utc_datetime']
df = pandas.read_csv('file.csv', parse_dates=parse_dates)

To extract date from timestamp, use numpy instead of pandas:

df['utc_date'] = numpy.array(df['utc_datetime'].values, dtype='datetime64[D]')

Numpy datetime operations are significantly faster than pandas datetime operations.

Upvotes: 1

Ajay Goyal
Ajay Goyal

Reputation: 396

If date is in string form then:

import datetime

# this line converts the string object in Timestamp object
df['DateTime'] = [datetime.datetime.strptime(d, "%Y-%m-%d %H:%M") for d in df["DateTime"]]

# extracting date from timestamp
df['Date'] = [datetime.datetime.date(d) for d in df['DateTime']] 

# extracting time from timestamp
df['Time'] = [datetime.datetime.time(d) for d in df['DateTime']]

If the object is already in the Timestamp format then skip the first line of code.

%Y-%m-%d %H:%M this means your timestamp object must be in the form like 2016-05-16 12:35:00.

Upvotes: 24

Gursel Karacor
Gursel Karacor

Reputation: 1167

Do this first:

df['time'] = pd.to_datetime(df['timestamp'])

Before you do your extraction as usual:

df['dates'] = df['time'].dt.date

Upvotes: 56

Related Questions