CODEWITHSUNDEEP
pythonpandassumtimedelta
Vivek Kalyanarangan
Vivek Kalyanarangan

Reputation: 9081

Sum of Timedeltas column in Pandas Python

A similar question has been asked here before

But when I tried all the available solutions it was giving me error.

Code:

print sum(data['Activity_Duration'],datetime.timedelta())  
#import operator  
#print reduce(operator.add, data['Activity_Duration'])

Error:

OverflowError

1 #print sum(data['Activity_Duration'],datetime.timedelta())
2 import operator
----> 3 print reduce(operator.add, data['Activity_Duration'])

OverflowError: long too big to convert

Am i missing something or can we come up with a more scalable solution?

Info: My data has 436746 rows.

I am working on 8 gig machine and data size is 650MB

Upvotes: 1

Views: 6823

Answers (1)

jezrael
jezrael

Reputation: 862661

I think you need sum:

print (df['Activity_Duration'].sum())

Sample:

import pandas as pd

start = pd.to_datetime('2015-02-24')
end = pd.to_datetime('2016-04-25')
rng = pd.date_range(start, end, freq='6D')

start = pd.to_datetime('2015-02-26')
end = pd.to_datetime('2016-04-27')
rng1 = pd.date_range(start, end, freq='6D')

df = pd.DataFrame({'Date1': rng, 'Date2': rng1})  

df['Activity_Duration'] = df.Date2 - df.Date1
print (df)
        Date1      Date2  Activity_Duration
0  2015-02-24 2015-02-26             2 days
1  2015-03-02 2015-03-04             2 days
2  2015-03-08 2015-03-10             2 days
3  2015-03-14 2015-03-16             2 days
4  2015-03-20 2015-03-22             2 days
5  2015-03-26 2015-03-28             2 days
6  2015-04-01 2015-04-03             2 days
7  2015-04-07 2015-04-09             2 days
8  2015-04-13 2015-04-15             2 days
9  2015-04-19 2015-04-21             2 days
...
...


print (df['Activity_Duration'].sum())
144 days 00:00:00

If need output in float:

import numpy as np

df['Activity_Duration'] = (df.Date2 - df.Date1) / np.timedelta64(1, 'D')
print (df)
        Date1      Date2  Activity_Duration
0  2015-02-24 2015-02-26                2.0
1  2015-03-02 2015-03-04                2.0
2  2015-03-08 2015-03-10                2.0
3  2015-03-14 2015-03-16                2.0
4  2015-03-20 2015-03-22                2.0
...
...
...

print (df['Activity_Duration'].sum())
144.0

Another solution is dt.days - output is int:

print (df['Activity_Duration'].dt.days.sum())
144

Timedelta limitations.

Upvotes: 4

Related Questions