CODEWITHSUNDEEP
pythonpandas
MarkNS
MarkNS

Reputation: 4021

Pandas merge hierarchical dataframes and maintain hierarchy

This github issue describes merging dataframes that have a mixed hierachy, and gives the solution of flattening the hierarchy to tuples.

df = pd.DataFrame([(1, 2, 3), (4, 5, 6)], columns=['a', 'b', 'c'])
new_df = df.groupby(['a']).agg({'b': [np.mean, np.sum]})
other_df = df = pd.DataFrame([(1, 2, 3), (7, 10, 6)], columns=['a', 'b', 'd'])
other_df.set_index('a', inplace=True)
print new_df
print other_df
p = pd.merge(new_df, other_df, left_index=True, right_index=True)
print p

output:

      b     
   mean  sum
a           
1     2    2
4     5    5

    b  d
a       
1   2  3
7  10  6

   (b, mean)  (b, sum)  b  d
a                           
1          2         2  2  3

However I would like to maintain the hierarchy, with a result as follows:

       b        b  d
    mean  sum
x                           
y      .    .   .  .

I just made the values dots here as they don't really make sense in this scenario, but hopefully the idea is clear.... any help gratefully received..

Upvotes: 2

Views: 996

Answers (1)

alko
alko

Reputation: 48317

Are you looking for something like this:

>>> other_to_tup = [(x, 'val') for x in other_df.columns]
>>> other_df.columns = pd.MultiIndex.from_tuples(other_to_tup)
>>> p = pd.merge(new_df, other_df, left_index=True, right_index=True)
>>> p
      b              d
   mean  sum  val  val
a
1     2    2    2    3

Upvotes: 4

Related Questions