CODEWITHSUNDEEP
pythonpandasdictionary
Jan G-M
Jan G-M

Reputation: 103

Dict of dict of dicts to pandas dataframe - changing multiindex rows to be columns

I have a dictionary like this:

my_dict = {'Key': {'Service': {'Number': 61, 'Percent': 2.54 }, 'Service2': {'Number': 42, 'Percent': 2.2 } }, 'Key2': {'Service3': {'Number': 8, 'Percent': 2.74}, 'Service2': {'Number': 52, 'Percent': 2.5 } }}

I'm trying to convert this to a pandas dataframe. I got this solution to work

pandas.concat(map(pandas.DataFrame, my_dict.itervalues()), keys=my_dict.keys()).stack().unstack(0)

However, my problem is that that I get a table where the row index is a multindex of Service & Number/Percent. Instead, I want the index to be only the different Services that come up (not a multiindex), and want the columns to be the Keys like they are now, but with 1 column section being Number and the 2nd column section being all the Keys with percent, if that makes sense. Transposing is not what I want, because I don't want the entire index to change, just the Number/Percent part. I want it to look like this, after converting it to a dataframe from the dictionary I wrote above:

          Number         Percent
          Key    Key2    Key     Key2
Service   61     NaN     2.54    NaN
Service2  42     52      2.2     2.5
Service3  NaN    8       NaN     2.74

Any suggestions on this?

Upvotes: 4

Views: 2561

Answers (1)

piRSquared
piRSquared

Reputation: 294218

pd.concat({k: pd.DataFrame(v) for k, v in my_dict.items()})

              Service  Service2  Service3
Key  Number     61.00      42.0       NaN
     Percent     2.54       2.2       NaN
Key2 Number       NaN      52.0      8.00
     Percent      NaN       2.5      2.74

pd.concat({k: pd.DataFrame(v) for k, v in my_dict.items()}, axis=1).stack(0).T

         Number       Percent      
            Key  Key2     Key  Key2
Service    61.0   NaN    2.54   NaN
Service2   42.0  52.0    2.20  2.50
Service3    NaN   8.0     NaN  2.74

This doesn't rely on comprehensions 

pd.DataFrame(my_dict).stack().apply(pd.Series).unstack()
# pandas.DataFrame(i).stack().apply(pandas.Series).unstack()

         Number       Percent      
            Key  Key2     Key  Key2
Service    61.0   NaN    2.54   NaN
Service2   42.0  52.0    2.20  2.50
Service3    NaN   8.0     NaN  2.74

Upvotes: 9

Related Questions