CODEWITHSUNDEEP
pythonpandascsvdataframe
Zakariah Siyaji
Zakariah Siyaji

Reputation: 1119

How to create a csv file from a list of pandas DataFrames?

I would like to create a csv file from a list of pandas dataframes. My data maintains all of the same column names besides for 1 column which is swapped each time. The following is what my list looks like based on the index of the list:

index 0:

height weight age isFunny
  4      2    21     0

index 1:

height weight age isCool
  10     30   81    34

index 2:

height weight age isDumb
  4      2    34     10

index 3:

height weight age isTalker
  4      12   25     3

I thought that the best way to save this data was in a list of pandas dataframes but if there is a better alternative, then please suggest any.

Upvotes: 0

Views: 532

Answers (2)

CJR
CJR

Reputation: 3985

Assuming myDF is your list of DataFrames you can serialize it like this:

import pickle
pickle.dump(myDF, "my_file.pickle")

Then you can load it whenever with this:

import pickle
myDF = pickle.load("my_file.pickle")

What you want to do can technically be done with a CSV file but it would be unwise to have a CSV with multiple headers. You could do that, though:

my_file_name = "my_file.csv"
[df.to_csv(my_file_name, mode="a+", index=False) for df in myDF]

Don't do this. This is dumb.

Based on your additional context, I would suggest doing this (which is similar in concept to the other answer):

melted_df = pd.concat([df.melt(id_vars = ['height', 'weight', 'age']) for df in myDF])
melted_df.to_csv("my_file.csv")

You may wish to unroll the MultiIndex into columns again.

Upvotes: 1

hellpanderr
hellpanderr

Reputation: 5896

You can use reduce to create a single dataframe and save it to a csv.

df_1 = pd.DataFrame([[4,2,21,0]],columns='height weight age isFunny'.split())
df_2 = pd.DataFrame([[10,30,81,35]],columns='height weight age isCool'.split())
df_3 = pd.DataFrame([[4,2,34,10]],columns='height weight age isDumb'.split())
df_4 = pd.DataFrame([[4,12,25,3]],columns='height weight age isTalker'.split())
l = [df_1, df_2, df_3, df_4]

from functools import reduce
>>> reduce(lambda x, y: x.append(y), l)

   age  height  isCool  isDumb  isFunny  isTalker  weight
0   21       4     NaN     NaN      0.0       NaN       2
1   81      10    35.0     NaN      NaN       NaN      30
2   34       4     NaN    10.0      NaN       NaN       2
3   25       4     NaN     NaN      NaN       3.0      12

Upvotes: 3

Related Questions