CODEWITHSUNDEEP
pythonpandasdataframegroup-by
Mansumen
Mansumen

Reputation: 383

Pandas get rows after groupby

Suppose I have the following dataset:

uid iid val
 1   1   2
 1   2   3
 1   3   4
 1   4  4.5
 1   5  5.5
 2   1   3
 2   2   3
 2   3   4
 3   4  4.5
 3   5  5.5

From this data, I want to first groupby uid, then get last 20% of number of rows from each uid.

That is, since uid=1 has 5 rows, I want to obtain last 1 row (20% of 5) from uid=1.

The following is what I want to do:

df.groupby('uid').tail([20% of each uid])

Can anyone help me?

Upvotes: 2

Views: 1545

Answers (2)

piRSquared
piRSquared

Reputation: 294218

I'd use floor division

df.groupby('uid').apply(lambda x: x.tail(len(x) // 5))

       uid  iid  val
uid                 
1   4    1    5  5.5

You can avoid including the uid in the index in the first place by passing group_keys=False to the groupby

df.groupby('uid', group_keys=False).apply(lambda x: x.tail(len(x) // 5))

   uid  iid  val
4    1    5  5.5

Upvotes: 1

gereleth
gereleth

Reputation: 2482

You can try applying a custom function to groupby object. Inside the function calculate how many rows should be taken and take the group's tail with that number of rows. int rounds toward 0, so any groups with less than 5 rows will not contribute any rows to the result.

df.groupby('uid').apply(lambda x: x.tail(int(0.2*x.shape[0])))

Upvotes: 2

Related Questions