CODEWITHSUNDEEP
pythonpandas
user266003
user266003

Reputation:

Reorganizing the data in a dataframe

I have data in the following format:

data = 
[
  {'data1': [{'sub_data1': 0}, {'sub_data2': 4}, {'sub_data3': 1}, {'sub_data4': -5}]},
  {'data2': [{'sub_data1': 1}, {'sub_data2': 1}, {'sub_data3': 1}, {'sub_data4': 12}]},
  {'data3': [{'sub_data1': 3}, {'sub_data2': 0}, {'sub_data3': 1}, {'sub_data4': 7}]},

]

How should I reorganize it so that when save it to hdf by 

a = pd.DataFrame(data, columns=map(lambda x: x.name, ['data1', 'data2', 'data3']))
a.to_hdf('my_data.hdf')

I get a dataframe in the following format:

            data1       data2     data3
_________________________________________
sub_data1   0           1           1
sub_data2   4           1           0
sub_data3   1           1           1
sub_data4   -5          12          7

update1: after following advice given me below and saving it an hdf file and reading it, I got this which is not what I want:

       data1                        data2                      data3   
0      {u'sub_data1': 22}           {u'sub_data1': 33}          {u'sub_data1': 44}   
1      {u'sub_data2': 0}            {u'sub_data2': 11}          {u'sub_data2': 44}   
2      {u'sub_data3': 12}           {u'sub_data3': 16}          {u'sub_data3': 19}   
3      {u'sub_data4': 0}            {u'sub_data4': 0}           {u'sub_data4': 0}

Upvotes: 2

Views: 120

Answers (1)

roman
roman

Reputation: 117636

Well if you convert your data into dictionary of dictionaries, you can then just create DataFrame very easily:

In [25]: data2 = {k: {m: n for i in v for m, n in i.iteritems()} for x in data for k, v in x.iteritems()}

In [26]: data2
Out[26]: 
{'data1': {'sub_data1': 0, 'sub_data2': 4, 'sub_data3': 1, 'sub_data4': -5},
 'data2': {'sub_data1': 1, 'sub_data2': 1, 'sub_data3': 1, 'sub_data4': 12},
 'data3': {'sub_data1': 3, 'sub_data2': 0, 'sub_data3': 1, 'sub_data4': 7}}

In [27]: pd.DataFrame(data2)
Out[27]: 
           data1  data2  data3
sub_data1      0      1      3
sub_data2      4      1      0
sub_data3      1      1      1
sub_data4     -5     12      7

Upvotes: 1

Related Questions