CODEWITHSUNDEEP
pythonpandaspivotmulti-index
Chubaka
Chubaka

Reputation: 3135

How to select a specific column from a multiIndex dataframe?

Playing the kaggle beer review datasets

https://www.kaggle.com/rdoume/beerreviews

df.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 1504037 entries, 1586613 to 39648
Data columns (total 13 columns):
brewery_id            1504037 non-null int64
brewery_name          1504037 non-null object
review_time           1504037 non-null int64
review_overall        1504037 non-null float64
review_aroma          1504037 non-null float64
review_appearance     1504037 non-null float64
review_profilename    1504037 non-null object
beer_style            1504037 non-null object
review_palate         1504037 non-null float64
review_taste          1504037 non-null float64
beer_name             1504037 non-null object
beer_abv              1504037 non-null float64
beer_beerid           1504037 non-null int64
dtypes: float64(6), int64(3), object(4)
memory usage: 160.6+ MB

I just did a pivot table and returns the following results

review_stat_by_beer = df[['beer_name','review_overall','review_aroma','review_appearance','review_palate','review_taste']]\
    .drop_duplicates(['beer_name'])\
    .pivot_table(index="beer_name", aggfunc=("count",'mean','median'))


review_stat_by_beer.info()

<class 'pandas.core.frame.DataFrame'>
Index: 44075 entries, ! (Old Ale) to 葉山ビール (Hayama Beer)
Data columns (total 15 columns):
(review_appearance, count)     44075 non-null int64
(review_appearance, mean)      44075 non-null float64
(review_appearance, median)    44075 non-null float64
(review_aroma, count)          44075 non-null int64
(review_aroma, mean)           44075 non-null float64
(review_aroma, median)         44075 non-null float64
(review_overall, count)        44075 non-null int64
(review_overall, mean)         44075 non-null float64
(review_overall, median)       44075 non-null float64
(review_palate, count)         44075 non-null int64
(review_palate, mean)          44075 non-null float64
(review_palate, median)        44075 non-null float64
(review_taste, count)          44075 non-null int64
(review_taste, mean)           44075 non-null float64
(review_taste, median)         44075 non-null float64
dtypes: float64(10), int64(5)
memory usage: 5.4+ MB

Trying to choose these columns

review_stat_by_beer.(review_appearance, count)  # SyntaxError: invalid syntax

review_stat_by_beer[(review_appearance, count)] #NameError: name 'review_appearance' is not defined

review_stat_by_beer['(review_appearance, count)'] #KeyError: '(review_appearance, count)'

how do I select these pivot table results? My ultimate goal is to do the math between 2 columns:

(review_overall, mean) minus (review_taste, mean)

Any thoughts? Thanks!

Upvotes: 2

Views: 92

Answers (1)

cs95
cs95

Reputation: 402493

There are a few options for selecting a specific result from a multiIndex:

# Setup
df =  pd.DataFrame(np.arange(9).reshape(3, 3))
df.columns = [['A', 'A', 'B'], ['a', 'b', 'c']]
df

   A     B
   a  b  c
0  0  1  2
1  3  4  5
2  6  7  8

Direct selection,

df[('A', 'a')]

0    0
1    3
2    6
Name: (A, a), dtype: int64

Via loc,

df.loc[:, ('A', 'a')]
# or 
# df.loc(axis=1)[('A', 'a')]  

0    0
1    3
2    6
Name: (A, a), dtype: int64

And also with xs,

df.xs(('A', 'a'), axis=1)

0    0
1    3
2    6
Name: (A, a), dtype: int64

The idea in all these cases is to pass a tuple of strings which signifies the first and second levels respectively (your column index has 2 levels). In your case that would look like 

review_stat_by_beer[('review_appearance', 'count')]

There are more methods, but these are the best ones.

Upvotes: 2

Related Questions