Reputation: 1475
I have the following code:
db_fields = ("id", "email", "status", "source")
df = DataFrame(results)
for col in db_fields:
if col not in df.columns:
COLUMN IS MISSING - COMMAND TO ADD COLUMN
If for example status
column is missing it needs to be added to the data frame with nothing as value so when I export the df
to csv
I will always have the same schema of fields.
I know that to remove column I should do:
df = df.drop(col, 1)
But I don't know what is the best way to add column with empty value.
Upvotes: 0
Views: 85
Reputation: 863361
You can create array of non exist columns and create new one with assign
and dictionary:
df = pd.DataFrame({'id': ['a1','a2', 'b1'],
'a': ['a1','a2', 'b1'],
'source': ['a1','a2', 'b1']})
print (df)
id a source
0 a1 a1 a1
1 a2 a2 a2
2 b1 b1 b1
db_fields = ("id", "email", "status", "source")
#get missing columns
diff = np.setdiff1d(np.array(db_fields), df.columns)
print (diff)
['email' 'status']
#get original columns not existed in db_fields
diff1 = np.setdiff1d(df.columns, np.array(db_fields)).tolist()
print (diff1)
['a']
#add missing columns with change order
d = dict.fromkeys(diff, np.nan)
df = df.assign(**d)[diff1 + list(db_fields)]
print (df)
a id email status source
0 a1 a1 NaN NaN a1
1 a2 a2 NaN NaN a2
2 b1 b1 NaN NaN b1
#if necessary first db_fields
df = df.assign(**d)[list(db_fields) + diff1]
print (df)
id email status source a
0 a1 NaN NaN a1 a1
1 a2 NaN NaN a2 a2
2 b1 NaN NaN b1 b1
Upvotes: 1
Reputation: 2231
Here you have it, plain and simple, in just one line:
import numpy as np
db_fields = ("id", "email", "status", "source")
df = DataFrame(results)
for col in db_fields:
if col not in df.columns:
# Add the column
df[col] = np.nan
BTW: You can also drop a column using df.drop(inplace=True)
.
Upvotes: 1
Reputation: 4792
This method will added status column with Null values:
import numpy as np
df['status'] = np.nan
Alternatively:
df['status'] = None
So:
db_fields = ("id", "email", "status", "source")
for col in db_fields:
if col not in df.columns:
df[col] = None
Upvotes: 1