CODEWITHSUNDEEP
pythonpandasdatedatetimetime
Tom
Tom

Reputation: 685

Splitting timestamp column into separate date and time columns

I have a pandas dataframe with over 1000 timestamps (below) that I would like to loop through:

2016-02-22 14:59:44.561776

I'm having a hard time splitting this time stamp into 2 columns- 'date' and 'time'. The date format can stay the same, but the time needs to be converted to CST (including milliseconds).

Thanks for the help

Upvotes: 62

Views: 175403

Answers (9)

Alexander
Alexander

Reputation: 109528

I'm not sure why you would want to do this in the first place, but if you really must...

df = pd.DataFrame({'my_timestamp': pd.date_range('2016-1-1 15:00', periods=5)})

>>> df
         my_timestamp
0 2016-01-01 15:00:00
1 2016-01-02 15:00:00
2 2016-01-03 15:00:00
3 2016-01-04 15:00:00
4 2016-01-05 15:00:00

df['new_date'] = [d.date() for d in df['my_timestamp']]
df['new_time'] = [d.time() for d in df['my_timestamp']]

>>> df
         my_timestamp    new_date  new_time
0 2016-01-01 15:00:00  2016-01-01  15:00:00
1 2016-01-02 15:00:00  2016-01-02  15:00:00
2 2016-01-03 15:00:00  2016-01-03  15:00:00
3 2016-01-04 15:00:00  2016-01-04  15:00:00
4 2016-01-05 15:00:00  2016-01-05  15:00:00

The conversion to CST is more tricky. I assume that the current timestamps are 'unaware', i.e. they do not have a timezone attached? If not, how would you expect to convert them?

For more details:

https://docs.python.org/2/library/datetime.html

How to make a datetime object aware (not naive)

EDIT

An alternative method that only loops once across the timestamps instead of twice:

new_dates, new_times = zip(*[(d.date(), d.time()) for d in df['my_timestamp']])
df = df.assign(new_date=new_dates, new_time=new_times)

EDIT 2023

This is how would do this now:

df = df.assign(
    new_date=df["my_timestamp"].dt.date, 
    new_time=df["my_timestamp"].dt.time
)

In fairness, the assign method was introduced in version 0.16.0 released March 22, 2015, now long before when I originally replied to this post back in 2016.

Upvotes: 51

sule mohammed
sule mohammed

Reputation: 137

If your timestamp is a string, you can convert it to pandas timestamp before splitting it.

#convert to pandas timestamp
data["old_date"] = pd.to_datetime(data.old_date)

#split columns
data["new_date"] = data["old_date"].dt.date
data["new_time"] = data["old_date"].dt.time

Upvotes: 0

Anton Protopopov
Anton Protopopov

Reputation: 31662

The easiest way is to use the pandas.Series dt accessor, which works on columns with a datetime dtype (see pd.to_datetime). For this case, pd.date_range creates an example column with a datetime dtype, therefore use .dt.date and .dt.time:

df = pd.DataFrame({'full_date': pd.date_range('2016-1-1 10:00:00.123', periods=10, freq='5H')})
df['date'] = df['full_date'].dt.date
df['time'] = df['full_date'].dt.time

In [166]: df
Out[166]:
                full_date        date             time
0 2016-01-01 10:00:00.123  2016-01-01  10:00:00.123000
1 2016-01-01 15:00:00.123  2016-01-01  15:00:00.123000
2 2016-01-01 20:00:00.123  2016-01-01  20:00:00.123000
3 2016-01-02 01:00:00.123  2016-01-02  01:00:00.123000
4 2016-01-02 06:00:00.123  2016-01-02  06:00:00.123000
5 2016-01-02 11:00:00.123  2016-01-02  11:00:00.123000
6 2016-01-02 16:00:00.123  2016-01-02  16:00:00.123000
7 2016-01-02 21:00:00.123  2016-01-02  21:00:00.123000
8 2016-01-03 02:00:00.123  2016-01-03  02:00:00.123000
9 2016-01-03 07:00:00.123  2016-01-03  07:00:00.123000

Upvotes: 18

arun
arun

Reputation: 521

In addition to @Alexander if you want a single liner

df['new_date'],df['new_time'] = zip(*[(d.date(), d.time()) for d in df['my_timestamp']])

Upvotes: 0

Okroshiashvili
Okroshiashvili

Reputation: 4149

Had same problem and this worked for me.

Suppose the date column in your dataset is called "date"

import pandas as pd
df = pd.read_csv(file_path)

df['Dates'] = pd.to_datetime(df['date']).dt.date
df['Time'] = pd.to_datetime(df['date']).dt.time

This will give you two columns "Dates" and "Time" with splited dates.

Upvotes: 74

pp_
pp_

Reputation: 3489

If your timestamp is a string, you can convert it to a datetime object:

from datetime import datetime

timestamp = '2016-02-22 14:59:44.561776'
dt = datetime.strptime(timestamp, '%Y-%m-%d %H:%M:%S.%f')

From then on you can bring it to whatever format you like.

Upvotes: 3

Nitin Sanghi
Nitin Sanghi

Reputation: 32

try this:

def time_date(datetime_obj):
    date_time = datetime_obj.split(' ')
    time = date_time[1].split('.')
    return date_time[0], time[0]

Upvotes: 0

Mathieu B
Mathieu B

Reputation: 471

If your timestamps are already in pandas format (not string), then:

df["date"] = df["timestamp"].date
dt["time"] = dt["timestamp"].time

If your timestamp is a string, you can parse it using the datetime module:

from datetime import datetime
data1["timestamp"] = df["timestamp"].apply(lambda x: \
    datetime.strptime(x,"%Y-%m-%d %H:%M:%S.%f"))

Source: http://pandas.pydata.org/pandas-docs/stable/timeseries.html

Upvotes: 4

wrkyle
wrkyle

Reputation: 559

Try

s = '2016-02-22 14:59:44.561776'

date,time = s.split()

then convert time as needed.

If you want to further split the time,

hour, minute, second = time.split(':')

Upvotes: 1

Related Questions