CODEWITHSUNDEEP
python-3.xpandasdataframe
alukard990
alukard990

Reputation: 841

Append pandas dataframe from other pandas dataframe that are values of a dictionary

I have a dictionary with N pairs (key,value), where N is unknown; each value is a pandas dataframe that contains a different set of columns. For example:

d = {'DF1': pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), columns=['a', 'b', 'c']),
'DF2': pd.DataFrame(np.array([[10, 11 ,12], [13, 14, 15]]),columns=['d', 'e'])}

I would to append all dataframes contained into the dictionary, in a third empty dataframe because I have to save all dataframes of dictionary into a parquet file. But if I use the following lines of code, there is no dataframe into df3:

df3 = pd.Dataframe()
for key in d: 
    df3.append(d[key], ignore_index=True)

How can I append all dataframes into df3?

UPDATE 1: all dataframes into the dictionary may have common columns

Upvotes: 1

Views: 53

Answers (2)

Georgina Skibinski
Georgina Skibinski

Reputation: 13387

Try:

v=list(d.values())
df3=v[0]
for el in v[1:]:
    df3=pd.concat([df3,el])
df3=df3.reset_index(drop=True)

Or simpler, per your comment:

df3 = pd.concat(d.values(), axis=0).reset_index(drop=True)

Upvotes: 1

gellerm
gellerm

Reputation: 47

I think a better approach is to use your for loop to concatenate the dataframes, i.e. pd.concat. Here's a link to the docs for how to use the function: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html. The catch with this is to make sure you append along the right axis (0 or 1)!

Upvotes: 1

Related Questions