CODEWITHSUNDEEP
pythonpython-3.xpandasgroup-bypandas-groupby
KubiK888
KubiK888

Reputation: 4723

Convert column values to column of list based on condition

Dataset as:

   id  col2  col3
0   1     1   123
1   1     1   234
2   1     0   345
3   2     1   456
4   2     0  1243
5   2     0   346
6   3     0   888
7   3     0   999
8   3     0   777

I would like to aggregate data by id, and append the values of col3 into a list only if its corresponding value at col2 is 1. Additionally, for people (of different id) who only have 0 in col2, I like the aggregated value to be 0 for col2 and empty list for col3.

Here is the current code:

df_test = pd.DataFrame({'id':[1, 1, 1, 2, 2, 2, 3, 3, 3], 'col2':[1, 1, 0, 1, 0, 0, 0, 0, 0], 'col3':[123, 234, 345, 456, 1243, 346, 888, 999, 777]})

df_test_agg = pd.pivot_table(df_test, index=['id'], values=['col2', 'col3'], aggfunc={'col2':np.max, 'col3':(lambda x:list(x))})

print (df_test_agg)

    col2              col3
id                        
1      1   [123, 234, 345]
2      1  [456, 1243, 346]
3      0   [888, 999, 777]

The desired output should be (ideally in one-step in Pandas):

    col2              col3
id                        
1      1            [123, 234]
2      1            [456]
3      0            []

///////////////////////////////////////////////////////////////////////////////////////

Edit - Trying out ColdSpeed's solution

df_test = pd.DataFrame({'id':[1, 1, 1, 2, 2, 2, 3, 3, 3], 'col2':[1, 1, 0, 1, 0, 0, 0, 0, 0], 'col3':[123, 234, 345, 456, 1243, 346, 888, 999, 777]})

print (df_test)

df_test_agg = (df_test.where(df_test.col2 > 0)
    .assign(id=df_test.id)
    .groupby('id')
    .agg({'col2': 'max', 'col3': lambda x: x.dropna().tolist()}))

print (df_test_agg)

   id  col2  col3
0   1     1   123
1   1     1   234
2   1     0   345
3   2     1   456
4   2     0  1243
5   2     0   346
6   3     0   888
7   3     0   999
8   3     0   777
    col2            col3
id                      
1    1.0  [123.0, 234.0]
2    1.0         [456.0]
3    NaN              []

///////////////////////////////////////////////////////////////////////////////////////

Edited original post to present more scenarios.

Upvotes: 2

Views: 1020

Answers (1)

cs95
cs95

Reputation: 402814

You can filter beforehand, then use groupby:

df_test.query('col2 > 0').groupby('id').agg({'col2': 'max', 'col3': list})

    col2        col3
id                  
1      1  [123, 234]
2      1       [456]

The caveat here is that if a group has only zeros, that group will be missing in the result. So, to fix that, you can mask with where:

(df_test.where(df_test.col2 > 0)
        .assign(id=df_test.id)
        .groupby('id')
        .agg({'col2': 'max', 'col3'lambda x: x.dropna().tolist()}))

    col2            col3
id                      
1    1.0  [123.0, 234.0]
2    1.0         [456.0]

To handle 0 groups in "col2", we can use 

(df.assign(col3=df.col3.where(df.col2.astype(bool)))
   .groupby('id')
   .agg({'col2':'max', 'col3': lambda x: x.dropna().astype(int).tolist()}))

    col2        col3
id                  
1      1  [123, 234]
2      1       [456]
3      0          []

Upvotes: 1

Related Questions