CODEWITHSUNDEEP
pythonpandastimestamptime-seriesaggregation
Ram
Ram

Reputation: 13

Aggregate a time interval on individual dates

I have pandas dataframe with two timestamps columns start and end

    start                      end
2014-08-28 17:00:00 | 2014-08-29 22:00:00
2014-08-29 10:45:00 | 2014-09-01 17:00:00
2014-09-01 15:00:00 | 2014-09-01 19:00:00

The intention is to aggregate the number of hours that were logged on a given date. So in the case of my example.

I would be creating date range and aggreating the hours over multiple entries. 

2014-08-28 -> 7 hrs
2014-08-29 -> 10 hrs + 1 hr 15 min => 11 hrs 15 mins
2014-08-30 -> 24 hrs
2014-08-31 -> 24 hrs
2014-09-01 -> 17 hrs + 4 hrs => 21 hrs

I've tried using timedelta but it only splits in absolute hours, not on a per day basis.

I've also tried to explode the rows(i.e split the row on a day basis but I could only get it to works at a date level, not at a time stamp level)

Any suggestion are greatly appreciated.

Upvotes: 1

Views: 671

Answers (2)

Naga kiran
Naga kiran

Reputation: 4607

you can use of pd.date_range to create a minute to minute interval of each day that spent, after that you can count the spent minutes and convert it to time delta

start   end
0   2014-08-28 17:00:00 2014-08-29 22:00:00
1   2014-08-29 10:45:00 2014-09-01 17:00:00
2   2014-09-01 15:00:00 2014-09-01 19:00:00


#Creating the minute to minute time intervals from start to end date of each line and creating as one series of dates 
a = pd.Series(sum(df.apply(lambda x: pd.date_range(x['start'],x['end'],freq='min').tolist(),1).tolist(),[])).dt.date
# Counting the each mintue intervals and converting to time stamps
a.value_counts().apply(lambda x: pd.to_timedelta(x,'m'))

Out:

2014-08-29   1 days 11:16:00
2014-08-30   1 days 00:00:00
2014-08-31   1 days 00:00:00
2014-09-01   0 days 21:02:00
2014-08-28   0 days 07:00:00
dtype: timedelta64[ns]

Upvotes: 1

Sergii
Sergii

Reputation: 109

Hope that would be useful. I guess you'll be able to adjust to serve your purpose. Way to thinking is the following - store day and corresponding time in dict. if it's the same day - just write difference. Otherwise write time till first midnight, iterate whenever days needed and write time from last midnight till end. FYI... I guess for 2014-09-01 result might be 21 hrs.

from datetime import datetime, timedelta
from collections import defaultdict


s = [('2014-08-28 17:00:00', '2014-08-29 22:00:00'),
     ('2014-08-29 10:45:00', '2014-09-01 17:00:00'),
     ('2014-09-01 15:00:00', '2014-09-01 19:00:00') ]


def aggreate(time):
    store = defaultdict(timedelta)

    for slice in time:
        start = datetime.strptime(slice[0], "%Y-%m-%d %H:%M:%S")
        end = datetime.strptime(slice[1], "%Y-%m-%d %H:%M:%S")

        start_date = start.date()
        end_date = end.date()

        if start_date == end_date:
            store[start_date] += end - start

        else:
            midnight = datetime(start.year, start.month, start.day + 1, 0, 0, 0)
            part1 = midnight - start
            store[start_date] += part1

            for i in range(1, (end_date - start_date).days):
                next_date = start_date + timedelta(days=i)
                store[next_date] += timedelta(hours=24)

            last_midnight = datetime(end_date.year, end_date.month, end_date.day, 0, 0, 0)
            store[end_date] += end - last_midnight

    return store


r = aggreate(s)

for i in r:
    print(i, r[i])

2014-08-28 7:00:00
2014-08-29 1 day, 11:15:00
2014-08-30 1 day, 0:00:00
2014-08-31 1 day, 0:00:00
2014-09-01 21:00:00

Upvotes: 0

Related Questions