CODEWITHSUNDEEP
pythonpandasdataframe
Luis
Luis

Reputation: 1475

How to add column to a data frame?

I have the following code:

db_fields = ("id", "email", "status", "source")
df = DataFrame(results)
for col in db_fields:
    if col not in df.columns:
          COLUMN IS MISSING - COMMAND TO ADD COLUMN

If for example status column is missing it needs to be added to the data frame with nothing as value so when I export the df to csv I will always have the same schema of fields.

I know that to remove column I should do:

df = df.drop(col, 1)

But I don't know what is the best way to add column with empty value.

Upvotes: 0

Views: 85

Answers (3)

jezrael
jezrael

Reputation: 863361

You can create array of non exist columns and create new one with assign and dictionary:

df = pd.DataFrame({'id': ['a1','a2', 'b1'],
                  'a': ['a1','a2', 'b1'],
                  'source': ['a1','a2', 'b1']})
print (df)
   id   a source
0  a1  a1     a1
1  a2  a2     a2
2  b1  b1     b1

db_fields = ("id", "email", "status", "source")

#get missing columns
diff = np.setdiff1d(np.array(db_fields), df.columns)
print (diff)
['email' 'status']

#get original columns not existed in db_fields
diff1 = np.setdiff1d(df.columns, np.array(db_fields)).tolist()
print (diff1)
['a']

#add missing columns with change order
d = dict.fromkeys(diff, np.nan)
df = df.assign(**d)[diff1 + list(db_fields)]
print (df)
    a  id  email  status source
0  a1  a1    NaN     NaN     a1
1  a2  a2    NaN     NaN     a2
2  b1  b1    NaN     NaN     b1

#if necessary first db_fields
df = df.assign(**d)[list(db_fields) + diff1]
print (df)
   id  email  status source   a
0  a1    NaN     NaN     a1  a1
1  a2    NaN     NaN     a2  a2
2  b1    NaN     NaN     b1  b1

Upvotes: 1

Manrique
Manrique

Reputation: 2231

Here you have it, plain and simple, in just one line:

import numpy as np
db_fields = ("id", "email", "status", "source")
df = DataFrame(results)
for col in db_fields:
    if col not in df.columns:
        # Add the column
        df[col] = np.nan

BTW: You can also drop a column using df.drop(inplace=True).

Upvotes: 1

Mohit Motwani
Mohit Motwani

Reputation: 4792

This method will added status column with Null values:

import numpy as np
df['status'] = np.nan

Alternatively:

df['status'] = None

So:

db_fields = ("id", "email", "status", "source")
for col in db_fields:
    if col not in df.columns:
        df[col] = None

Upvotes: 1

Related Questions