CODEWITHSUNDEEP
pythonpandas
ericmjl
ericmjl

Reputation: 14724

Appending to an empty DataFrame in Pandas?

Is it possible to append to an empty data frame that doesn't contain any indices or columns?

I have tried to do this, but keep getting an empty dataframe at the end.

e.g.

import pandas as pd

df = pd.DataFrame()
data = ['some kind of data here' --> I have checked the type already, and it is a dataframe]
df.append(data)

The result looks like this:

Empty DataFrame
Columns: []
Index: []

Upvotes: 331

Views: 645963

Answers (6)

cs95
cs95

Reputation: 403128

Why am I getting "AttributeError: 'DataFrame' object has no attribute 'append'?

Starting from pandas >= 2.0, append has been removed, use pd.concat instead.

The idiomatic way to append DataFrames (it to collect all your smaller DataFrames into a list, and then make one single call to pd.concat.

df_list = []
for df in some_function_that_yields_df():
    df_list.append(df)

big_df = pd.concat(df_list)

The rationale for its removal was to discourage iteratively growing DataFrames in a loop (which is what people typically use append for). This is because append makes a new copy at each stage, resulting in quadratic complexity in memory. See Creating an empty Pandas DataFrame, and then filling it (in particular, this answer) where answers discourage iteratively growing DataFrames.

More info:

Upvotes: 2

DSM
DSM

Reputation: 353499

This should work:

>>> df = pd.DataFrame()
>>> data = pd.DataFrame({"A": range(3)})
>>> df = df.append(data) 
>>> df

   A
0  0
1  1
2  2

Since the append doesn't happen in-place, so you'll have to store the output if you want it:

>>> df = pd.DataFrame()
>>> data = pd.DataFrame({"A": range(3)})
>>> df.append(data)  # without storing
>>> df
Empty DataFrame
Columns: []
Index: []
>>> df = df.append(data)
>>> df
   A
0  0
1  1
2  2

Upvotes: 546

Kloster Matias
Kloster Matias

Reputation: 553

The answers are very useful, but since pandas.DataFrame.append was deprecated (as already mentioned by various users), and the answers using pandas.concat are not "Runnable Code Snippets" I would like to add the following snippet:

import pandas as pd

df = pd.DataFrame(columns =['name','age'])
row_to_append = pd.DataFrame([{'name':"Alice", 'age':"25"},{'name':"Bob", 'age':"32"}])
df = pd.concat([df,row_to_append])

So df is now:

    name age
0  Alice  25
1    Bob  32

Upvotes: 26

Wtower
Wtower

Reputation: 19922

pandas.DataFrame.append Deprecated since version 1.4.0: Use concat() instead.

Therefore:

df = pd.DataFrame() # empty dataframe
df2 = pd..DataFrame(...) # some dataframe with data

df = pd.concat([df, df2])

Upvotes: 8

Deepish
Deepish

Reputation: 786

You can concat the data in this way:

InfoDF = pd.DataFrame()
tempDF = pd.DataFrame(rows,columns=['id','min_date'])

InfoDF = pd.concat([InfoDF,tempDF])

Upvotes: 50

newmathwhodis
newmathwhodis

Reputation: 3289

And if you want to add a row, you can use a dictionary:

df = pd.DataFrame()
df = df.append({'name': 'Zed', 'age': 9, 'height': 2}, ignore_index=True)

which gives you:

   age  height name
0    9       2  Zed

Upvotes: 135

Related Questions