CODEWITHSUNDEEP
pythonpandasdataframeconcatenation
SMar3552
SMar3552

Reputation: 113

Pandas - Concatenating Dataframes

I have a script with if statements that has 14 possible dataframes

['result_14', 'result_13', 'result_12', 'result_11', 'result_10', 'result_9', 'result_8', 'result_7', 'result_6', 'result_5', 'result_4', 'result_3', 'result_2', 'result_1']

Not all dataframes are created every time I run the script. It is dependent on a secondary input variable. I am now attempting to concatenate dataframes but run into issue with those that do not exist.

pd.concat(([result_14, result_13, result_12, result_11, result_10, result_9, result_8, result_7, result_6, result_5, result_4, result_3, result_2, result_1]), ignore_index=True)

NameError: name 'result_13' is not defined

I have tried finding all dfs that exist in my python memory and parsing the results but this creates a list rather than a list of dataframes

alldfs = [var for var in dir() if isinstance(eval(var), pd.core.frame.DataFrame)]
SelectDFs = [s for s in alldfs if "result" in s]
SelectDFs

['result_14', 'result_15', 'result_12', 'result_11', 'result_10', 'result_9', 'result_8', 'result_7', 'result_6', 'result_5', 'result_4', 'result_3', 'result_2', 'result_1']

pd.concat(([SelectDFs]), ignore_index=True)
TypeError: cannot concatenate object of type '<class 'list'>'; only Series and DataFrame objs are valid

Upvotes: 2

Views: 305

Answers (3)

BENY
BENY

Reputation: 323376

You can try

%who_ls  DataFrame
# %whos DataFrame

In your case

l = %who_ls  DataFrame
pd.concat([eval(dfn) for dfn in l if dfn.startswith('result')], ignore_index=True)

Upvotes: 2

hessam_kk
hessam_kk

Reputation: 319

Have you tried to convert them into DFs? I mean when you want to concat them, it raise an error which says your data need to be dfs rahter than lists, so have you tried to convert your lists into DFs?

this link may help you: Convert List to Pandas Dataframe Column

Upvotes: 0

GodWin1100
GodWin1100

Reputation: 1420

You are passing list of string and not Dataframe object.

And once you re able to get DF Object you can pass SelecteDFs without bracket.

pd.concat(SelectDFs, ignore_index=True)

Upvotes: 0

Related Questions