CODEWITHSUNDEEP
pythonpandas
plasmon360
plasmon360

Reputation: 4199

sorting in pandas groupby with two columns

I am trying to group a dataframe with two columns and avoid default sorting using 'sort = False'. However, I am unable to achieve this.

Here is the simplified example 

df = pd.DataFrame([
        ['zebra', 1, 10],
        ['zebra', 2, 10],
        ['apple', 3, 20],
        ['apple', 4, 20],
    ],
    columns=['ColA','ColB','ColC'])

df is therefore 

    ColA  ColB  ColC
0  zebra     1    10
1  zebra     2    10
2  apple     3    20
3  apple     4    20

I am using pandas (1.0.3) groupby and disabling sorting of the keys

df_agg = df.groupby(by=['ColA','ColB'], sort = False)

df_agg.groups

results in 

{('apple', 3): Int64Index([2], dtype='int64'),
 ('apple', 4): Int64Index([3], dtype='int64'),
 ('zebra', 1): Int64Index([0], dtype='int64'),
 ('zebra', 2): Int64Index([1], dtype='int64')}

which is the same as "sort = True" (default)

However, what I would like is as following 

{
 ('zebra', 1): Int64Index([0], dtype='int64'),
 ('zebra', 2): Int64Index([1], dtype='int64'),
 ('apple', 3): Int64Index([2], dtype='int64'),
 ('apple', 4): Int64Index([3], dtype='int64')
}

'sort = False' when grouping by one column seems to be working fine.

df_agg = df.groupby(by=['ColA'], sort = False)
df_agg.groups

results in 

{'zebra': Int64Index([0, 1], dtype='int64'),
 'apple': Int64Index([2, 3], dtype='int64')}

If sorting only works on one column and not on tuples. I could sort the groups dict based on the tuple, but I am using an application that is expecting a groupby object. I appreciate any pointers on how this can be addressed.

Upvotes: 3

Views: 1536

Answers (2)

piRSquared
piRSquared

Reputation: 294218

The groups attribute is a dictionary and NOT where order of groups is determined. You must "resolve" the groupby object with some operation to determine what the order is/was.

df.groupby(['ColA', 'ColB'], sort=False, as_index=False).first()

    ColA  ColB  ColC
0  zebra     1    10
1  zebra     2    10
2  apple     3    20
3  apple     4    20

Versus

df.groupby(['ColA', 'ColB'], as_index=False).first()

    ColA  ColB  ColC
0  apple     3    20
1  apple     4    20
2  zebra     1    10
3  zebra     2    10

The ACTUAL place to look is the groupby object's ngroup method

g1 = df.groupby(['ColA', 'ColB'], sort=False, as_index=False)
g1.ngroup()

0    0
1    1
2    2
3    3
dtype: int64

Versus

g2 = df.groupby(['ColA', 'ColB'], as_index=False)
g2.ngroup()

0    2
1    3
2    0
3    1
dtype: int64

Upvotes: 6

Scott Boston
Scott Boston

Reputation: 153460

Let's use a psuedo sort key, here I create one using pd.factorize:

df.assign(sortkey=pd.factorize(df['ColA'])[0]).groupby(['sortkey', 'ColA', 'ColB']).groups

Output:

{(0, 'zebra', 1): Int64Index([0], dtype='int64'),
 (0, 'zebra', 2): Int64Index([1], dtype='int64'),
 (1, 'apple', 3): Int64Index([2], dtype='int64'),
 (1, 'apple', 4): Int64Index([3], dtype='int64')}

Upvotes: 4

Related Questions