CODEWITHSUNDEEP
pythonlistdataframedictionary
Marlon Henrique Teixeira
Marlon Henrique Teixeira

Reputation: 93

How to generate a dataframe from a list of n dictionaries generated by n iterations of a loop?

I've generated a list dictionaries by iterating a function n times. Therefore, as a result for d, I have n dictionaries distincts. This is d:

 d = {'Insumos' : ["%s" % frame['DESCRIÇÃO'].unique()], 'Valor previsto' : ['%.2f' % y_pred_fut],
                      'MAPE' : [ '%.2f' % mean_absolute_percentage_error(y_mat_val, y_pred)], 
                      'MAE' : ['%.2f' %mean_absolute_error(y_mat_val, y_pred)], 'r2' : ['%.2f' %  r2_score(y_mat_val, y_pred)]}

And this is the result for n a specific iteration of d:

{'Insumos': ["['ABUTILOM (ABUTILON STRIATUM)']"], 'Valor previsto': ['30.56'], 'MAPE': ['5.59'], 'MAE': ['1.60'], 'r2': ['-16.70']}
{'Insumos': ["['ACALIFA (ACALYPHA WILKESIANA)']"], 'Valor previsto': ['31.22'], 'MAPE': ['3.24'], 'MAE': ['0.96'], 'r2': ['-2.24']}
{'Insumos': ['[\'ACIONADOR MANUAL TIPO "QUEBRE O VIDRO"\']'], 'Valor previsto': ['72.52'], 'MAPE': ['4.76'], 'MAE': ['3.21'], 'r2': ['-17.48']}
{'Insumos': ["['ADUBO QUÍMICO NPK, 10:10:10']"], 'Valor previsto': ['2.71'], 'MAPE': ['5.02'], 'MAE': ['0.12'], 'r2': ['0.41']}

If I apply pd.DataFrame.from_records(d), I get n distinct dataframes as below:

0  ['ABUTILOM (ABUTILON STRIATUM)']  1.60  5.59          30.56  -16.70
                             Insumos   MAE  MAPE Valor previsto     r2
0  ['ACALIFA (ACALYPHA WILKESIANA)']  0.96  3.24          31.22  -2.24
                                      Insumos   ...        r2
0  ['ACIONADOR MANUAL TIPO "QUEBRE O VIDRO"']   ...    -17.48

[1 rows x 5 columns]
                           Insumos   MAE  MAPE Valor previsto    r2
0  ['ADUBO QUÍMICO NPK, 10:10:10']  0.12  5.02           2.71  0.41
                               Insumos   MAE  MAPE Valor previsto     r2
0  ['ALAMANDA (ALLAMANDA NERIIFOLIA)']  2.13  7.03          32.93  -8.51
                                             Insumos  ...       r2
0  ['ALVENARIA DE EMBASAMENTO - TIJOLOS MACIÇOS C...  ...    -1.83

[1 rows x 5 columns]
.
.
.

I want to get all the n distinct dictionaries resulting from n iterations of d and to make a unique dataframe.

Thanks!

Upvotes: 1

Views: 56

Answers (2)

Leonid Mednikov
Leonid Mednikov

Reputation: 973

As you feed one d to pd.DataFrame it can only produce DataFrame with that one line. You need to combine d values. The simplest (but not the most efficient) way is to create a list and add each calculated d in it with append(d) like that

d_list = []
for some_data in some_data_source:
    d = get_d(some_data)
    d_list.append(d)

df = pd.DataFrame(d_list)

A list of dicts will produce DataFrame like you want.

P.S. And it is not clear, why you embrace one value in a dict like here

'MAPE' : [ '%.2f' % mean_absolute_percentage_error(y_mat_val, y_pred)]

It would make it difficult to manipulate later. Single value is better to be stored as is 

'MAPE' : '%.2f' % mean_absolute_percentage_error(y_mat_val, y_pred)

And if you want to make some calculations in DataFrame, you'd better not convert the value into string, but store the value. You can convert to string later

'MAPE' : mean_absolute_percentage_error(y_mat_val, y_pred)

Upvotes: 1

user3468054
user3468054

Reputation: 610

You need to use from_dict rather than from_records if you have a dictionary.

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.from_dict.html

If you have multiple input dictionaries, put your dictionaries into a list:

d = [
    {'Insumos': ["['ABUTILOM (ABUTILON STRIATUM)']"], 'Valor previsto': ['30.56'], 'MAPE': ['5.59'], 'MAE': ['1.60'], 'r2': ['-16.70']},
    {'Insumos': ["['ACALIFA (ACALYPHA WILKESIANA)']"], 'Valor previsto': ['31.22'], 'MAPE': ['3.24'], 'MAE': ['0.96'], 'r2': ['-2.24']},
    {'Insumos': ['[\'ACIONADOR MANUAL TIPO "QUEBRE O VIDRO"\']'], 'Valor previsto': ['72.52'], 'MAPE': ['4.76'], 'MAE': ['3.21'], 'r2': ['-17.48']},
    {'Insumos': ["['ADUBO QUÍMICO NPK, 10:10:10']"], 'Valor previsto': ['2.71'], 'MAPE': ['5.02'], 'MAE': ['0.12'], 'r2': ['0.41']},
]

Then I think it should work as you intend.

>>>>pd.DataFrame.from_records(d)
                                        Insumos     MAE    MAPE  \
 0            [['ABUTILOM (ABUTILON STRIATUM)']]  [1.60]  [5.59]
 1           [['ACALIFA (ACALYPHA WILKESIANA)']]  [0.96]  [3.24]
 2  [['ACIONADOR MANUAL TIPO "QUEBRE O VIDRO"']]  [3.21]  [4.76]
 3             [['ADUBO QU?MICO NPK, 10:10:10']]  [0.12]  [5.02]

  Valor previsto        r2
0        [30.56]  [-16.70]
1        [31.22]   [-2.24]
2        [72.52]  [-17.48]
3         [2.71]    [0.41]

Upvotes: 1

Related Questions