CODEWITHSUNDEEP
pythonpandasdictionary
user308827
user308827

Reputation: 21961

Convert a list of dictionary containing dictionaries into Pandas dataframe

I have the following list of python dictionaries:

[{'date': '2019-02-21', 'basicStats': {'min': -0.9994264245033264, 'max': -0.41181543469429016, 'mean': -0.4983844268421697, 'stDev': 0.071324608484601}}, {'date': '2019-02-16', 'basicStats': {'min': -0.9990605711936951, 'max': -0.09592325985431671, 'mean': -0.385945735727586, 'stDev': 0.0640801258659954}}, {'date': '2019-02-01', 'basicStats': {'min': -0.9989479184150696, 'max': -0.21808761358261108, 'mean': -0.4007919550689754, 'stDev': 0.07135259658292871}}]

I want to convert it into a pandas dataframe with a column for date and more columns for 'min', 'max', 'mean' and 'stdev'. However, when I do:

pd.DataFrame(dict)

I get:

date                                         basicStats
0   2019-02-21  {'min': -0.9994264245033264, 'max': -0.4118154...
1   2019-02-16  {'min': -0.9990605711936951, 'max': -0.0959232...
2   2019-02-01  {'min': -0.9989479184150696, 'max': -0.2180876...

How can I fix this?

Upvotes: 1

Views: 73

Answers (1)

jezrael
jezrael

Reputation: 862511

Use json.json_normalize:

from pandas.io.json import json_normalize

df = json_normalize(d)
print (df)
        date  basicStats.min  basicStats.max  basicStats.mean  \
0  2019-02-21       -0.999426       -0.411815        -0.498384   
1  2019-02-16       -0.999061       -0.095923        -0.385946   
2  2019-02-01       -0.998948       -0.218088        -0.400792   

   basicStats.stDev  
0          0.071325  
1          0.064080  
2          0.071353

Another idea is expand dictionary - extract key basicStats and merge all another keys:

df = pd.DataFrame([{**x, **x.pop('basicStats')}  for x in d])
print (df)
         date       min       max      mean     stDev
0  2019-02-21 -0.999426 -0.411815 -0.498384  0.071325
1  2019-02-16 -0.999061 -0.095923 -0.385946  0.064080
2  2019-02-01 -0.998948 -0.218088 -0.400792  0.071353

Upvotes: 4

Related Questions