CODEWITHSUNDEEP
pythonpython-3.xpandas
luisfer
luisfer

Reputation: 2120

Pandas: Grouping by date, aggregating on other column

I have this dataframe. It's information about license usage:

    usuario feature     fini                    ffin                    delta
0   USER-1  PROGRAM-1   2016-06-30 21:03:21     2016-06-30 21:03:34     00:00:13
2   USER-1  PROGRAM-1   2016-06-30 21:09:20     2016-06-30 21:09:32     00:00:12
4   USER-1  PROGRAM-1   2016-06-30 21:14:40     2016-06-30 21:15:34     00:00:54
6   USER-1  PROGRAM-1   2016-06-30 21:16:42     2016-06-30 21:17:24     00:00:42
8   USER-1  PROGRAM-1   2016-06-30 21:18:09     2016-06-30 21:18:21     00:00:12

Sorry for the fields in spanish, but you get the idea. fini means fecha inicial (inital date) and ffin fecha final (ending date), as you have guess delta is ffin-fini

So, I want to know how much time USER-1 has spent in whatever program he has been working (PROGRAM-1) in this case.

If I do a table['delta'].sum() I get what I want, it says he used it 00:02:13.

Now suppose I have more users, more features, and I want to group them by days (maybe hours), to see how people are using their licenses

I tried the resample, but I really don't understand how it works. I saw there's a Grouper function but I don't have it installed.

Upvotes: 0

Views: 86

Answers (2)

James Dellinger
James Dellinger

Reputation: 1261

The line below will help you group by user and date and hour (fyi. if you were to instead use df['fini'].dt.hour it would sum up values for the same hour across multiple days):

df.groupby([df['usuario'], df['fini'].apply(lambda x: x.round('h'))]).delta.sum()

Applying this to an extended version of your example:

d = {
    'usuario':['USER-1','USER-1','USER-1','USER-1','USER-1','USER-1','USER-1','USER-1','USER-1','USER-1','USER-2','USER-2'],
    'feature':['PROGRAM-1','PROGRAM-1','PROGRAM-1','PROGRAM-1','PROGRAM-1','PROGRAM-1','PROGRAM-1','PROGRAM-1','PROGRAM-2','PROGRAM-2','PROGRAM-1','PROGRAM-1'],
    'fini':['2016-06-30 21:03:21','2016-06-30 21:09:20','2016-06-30 21:14:40','2016-06-30 21:16:42','2016-06-30 21:18:09', '2016-06-30 22:03:21','2016-06-30 22:09:20','2016-07-01 21:03:21','2016-07-01 22:09:20','2016-07-01 23:14:40','2016-06-30 17:16:42','2016-06-30 18:18:09'],
    'ffin':['2016-06-30 21:03:34','2016-06-30 21:09:32','2016-06-30 21:15:34','2016-06-30 21:17:24','2016-06-30 21:18:21', '2016-06-30 22:04:02','2016-06-30 22:09:51','2016-07-01 21:03:43','2016-07-01 22:10:12','2016-07-01 23:15:03','2016-06-30 17:17:23','2016-06-30 18:18:19']
}
df = pd.DataFrame(data=d)

date_cols = ['fini', 'ffin']
for col in date_cols:
    df[col] = pd.to_datetime(df[col])

df['delta'] = df['ffin'] - df['fini']

df.groupby([df['usuario'], df['fini'].apply(lambda x: x.round('h'))]).delta.sum()

Outputs the following:

usuario  fini               
USER-1   2016-06-30 21:00:00   00:02:13
         2016-06-30 22:00:00   00:01:12
         2016-07-01 21:00:00   00:00:22
         2016-07-01 22:00:00   00:00:52
         2016-07-01 23:00:00   00:00:23
USER-2   2016-06-30 17:00:00   00:00:41
         2016-06-30 18:00:00   00:00:10
Name: delta, dtype: timedelta64[ns]

Also, if you wanted, adding feature to the groupby is trivial:

df.groupby([df['usuario'], df['feature'], df['fini'].apply(lambda x: x.round('h'))]).delta.sum()

Outputs:

usuario  feature    fini               
USER-1   PROGRAM-1  2016-06-30 21:00:00   00:02:13
                    2016-06-30 22:00:00   00:01:12
                    2016-07-01 21:00:00   00:00:22
         PROGRAM-2  2016-07-01 22:00:00   00:00:52
                    2016-07-01 23:00:00   00:00:23
USER-2   PROGRAM-1  2016-06-30 17:00:00   00:00:41
                    2016-06-30 18:00:00   00:00:10
Name: delta, dtype: timedelta64[ns]

Upvotes: 2

mommermi
mommermi

Reputation: 1052

This is code is grouping the data by usuario and date (as provided infini). If you want a different grouping scheme (e.g., based on date and hour), you can modify it accordingly:

import pandas as pd

df = pd.DataFrame({'usuario': ['USER-1']*5,
                   'feature': ['PROGRAM-1']*5,
                   'fini': ['2016-06-30 21:03:21',
                            '2016-06-30 21:09:20',
                            '2016-06-30 21:14:40',
                            '2016-07-30 21:16:42',
                            '2016-07-30 21:18:09'],
                   'ffin': ['2016-06-30 21:03:34',
                            '2016-06-30 21:09:32',
                            '2016-06-30 21:15:34',
                            '2016-07-30 21:17:24',
                            '2016-07-30 21:18:21'],
                   'delta': ['00:00:13',
                             '00:00:12',
                             '00:00:54',
                             '00:00:42',
                             '00:00:12']})

# proper formatting for columns
df.fini = pd.to_datetime(df.fini)
df.ffin = pd.to_datetime(df.ffin)
df.delta = pd.to_timedelta(df.delta)

print(df.groupby([df.usuario, df.fini.dt.date]).delta.sum())
#usuario  fini      
#USER-1   2016-06-30   00:01:19
#         2016-07-30   00:00:54
#Name: delta, dtype: timedelta64[ns]

Upvotes: 0

Related Questions