CODEWITHSUNDEEP
pythonpandasdataframepivot-table
MaMo
MaMo

Reputation: 585

Keep pandas dataframe columns and their order in pivot table

I have a dataframe:

df = pd.DataFrame({'No': [123,123,123,523,523,523,765], 
                  'Type': ['A','B','C','A','C','D','A'],
                  'Task': ['First','Second','First','Second','Third','First','Fifth'],
                  'Color': ['blue','red','blue','black','red','red','red'],
                  'Price': [10,5,1,12,12,12,18],
                  'Unit': ['E','E','E','E','E','E','E'],
                  'Pers.ID': [45,6,6,43,1,9,2]
                  })

So it looks like this:

df
+-----+------+--------+-------+-------+------+---------+
| No  | Type |  Task  | Color | Price | Unit | Pers.ID |
+-----+------+--------+-------+-------+------+---------+
| 123 | A    | First  | blue  |    10 | E    |      45 |
| 123 | B    | Second | red   |     5 | E    |       6 |
| 123 | C    | First  | blue  |     1 | E    |       6 |
| 523 | A    | Second | black |    12 | E    |      43 |
| 523 | C    | Third  | red   |    12 | E    |       1 |
| 523 | D    | First  | red   |    12 | E    |       9 |
| 765 | A    | First  | red   |    18 | E    |       2 |
+-----+------+--------+-------+-------+------+---------+

then I created a pivot table:

piv = pd.pivot_table(df, index=['No','Type','Task'])

Result:

                 Pers.ID  Price
No  Type Task                  
123 A    First        45     10
    B    Second        6      5
    C    First         6      1
523 A    Second       43     12
    C    Third         1     12
    D    First         9     12
765 A    Fifth         2     18

As you can see, problems are:

I tried to fix this by executing:

cols = list(df.columns)
piv = pd.pivot_table(df, index=['No','Type','Task'], values = cols)

but the result is the same.

I read other posts but none of them matched my problem in a way that I could use it.

Thank you!

EDIT: desired output

                   Color  Price   Unit  Pers.ID
No  Type Task                  
123 A    First      blue     10      E       45
    B    Second      red      5      E        6
    C    First      blue      1      E        6
523 A    Second    black     12      E       43
    C    Third       red     12      E        1
    D    First       red     12      E        9
765 A    Fifth       red     18      E        2

Upvotes: 3

Views: 7060

Answers (1)

jezrael
jezrael

Reputation: 863166

I think problem is in pivot_table default aggregate function is mean, so strings columns are excluded. So need custom function, also order is changed, so reindex is necessary: 

f = lambda x: x.sum() if np.issubdtype(x.dtype, np.number) else ', '.join(x)
cols = df.columns[~df.columns.isin(['No','Type','Task'])].tolist()

piv = (pd.pivot_table(df, 
                     index=['No','Type','Task'], 
                     values = cols,
                     aggfunc=f).reindex(columns=cols))
print (piv)
                 Color  Price Unit  Pers.ID
No  Type Task                              
123 A    First    blue     10    E       45
    B    Second    red      5    E        6
    C    First    blue      1    E        6
523 A    Second  black     12    E       43
    C    Third     red     12    E        1
    D    First     red     12    E        9
765 A    Fifth     red     18    E        2

Another solution with groupby and same aggregation function, ordering is not problem:

df = (df.groupby(['No','Type','Task'])
       .agg(lambda x: x.sum() if np.issubdtype(x.dtype, np.number) else ', '.join(x)))
print (df)
                 Color  Price Unit  Pers.ID
No  Type Task                              
123 A    First    blue     10    E       45
    B    Second    red      5    E        6
    C    First    blue      1    E        6
523 A    Second  black     12    E       43
    C    Third     red     12    E        1
    D    First     red     12    E        9
765 A    Fifth     red     18    E        2

But if need set first 3 columns to MultiIndex only:

df = df.set_index(['No','Type','Task'])
print (df)
                 Color  Price Unit  Pers.ID
No  Type Task                              
123 A    First    blue     10    E       45
    B    Second    red      5    E        6
    C    First    blue      1    E        6
523 A    Second  black     12    E       43
    C    Third     red     12    E        1
    D    First     red     12    E        9
765 A    Fifth     red     18    E        2

Upvotes: 4

Related Questions