Pandas: customed aggregate functions for DataFrameGroupBy

Question

I have the following data frame my_df:

name   date          A_score      B_score
------------------------------------------
John   2017-01-01     5            6
John   2017-01-10     10           8
John   2017-02-04     3            5
Andy   2017-01-25     8            9 
Andy   2017 02-05     7            1
Andy   2017-02-12     9            9

For each name, we want to find the absolute delta of A_score and B_score. The absolute delta is defined as the absolute value difference between the earliest date and the second earliest date.

The resulting data frame should be like:

 name    A_score_result      B_score_result
----------------------------------------------
 John           5               2
 Andy           1               8

To achieve this, I tried:

new_df = my_df.groupby('name').apply(lambda x:myFun(x))

and

new_df = my_df.groupby('name').agg(['myFun'])    

where myFun is:

def myFun(x):
    y = x[2]-x[1]
    return y

---

However, both approaches have errors like below:

/usr/local/lib/python3.4/dist-packages/pandas/core/frame.py in __getitem__(self, key)
   2057             return self._getitem_multilevel(key)
   2058         else:
-> 2059             return self._getitem_column(key)
   2060 
   2061     def _getitem_column(self, key):

/usr/local/lib/python3.4/dist-packages/pandas/core/frame.py in _getitem_column(self, key)
   2064         # get column
   2065         if self.columns.is_unique:
-> 2066             return self._get_item_cache(key)
   2067 
   2068         # duplicate columns & possible reduce dimensionality

/usr/local/lib/python3.4/dist-packages/pandas/core/generic.py in _get_item_cache(self, item)
   1384         res = cache.get(item)
   1385         if res is None:
-> 1386             values = self._data.get(item)
   1387             res = self._box_item_values(item, values)
   1388             cache[item] = res

/usr/local/lib/python3.4/dist-packages/pandas/core/internals.py in get(self, item, fastpath)
   3541 
   3542             if not isnull(item):
-> 3543                 loc = self.items.get_loc(item)
   3544             else:
   3545                 indexer = np.arange(len(self.items))[isnull(self.items)]

/usr/local/lib/python3.4/dist-packages/pandas/indexes/base.py in get_loc(self, key, method, tolerance)
   2134                 return self._engine.get_loc(key)
   2135             except KeyError:
-> 2136                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2137 
   2138         indexer = self.get_indexer([key], method=method, tolerance=tolerance)

pandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4145)()

pandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4009)()

pandas/src/hashtable_class_helper.pxi in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:13166)()

pandas/src/hashtable_class_helper.pxi in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:13120)()

KeyError: 0

---

Any suggestion on how to fix this problem? Thanks a lot!

Answer 1

Try this:

In [358]: df.drop('date',1).groupby('name').agg(lambda x: abs(x.iloc[1] - x.iloc[0]))
Out[358]:
      A_score  B_score
name
Andy        1        8
John        5        2