CODEWITHSUNDEEP
pythonpandas
EMC
EMC

Reputation: 749

Pandas notnull is not working on column in data frame

A follow-up question from Combine Pandas data frame column values into new column

I have successfully combined a series of ID's into one field and now I need to filter out any rows that did not end up with a Combined ID value. Usually I would use notnull but on this column it is not working. Can anyone fill me in on the problem? Thanks!

df_merged['Combined_ID']  = df_merged[['ID1','ID2','ID3','ID4','ID5']].apply(lambda x : ''.join([e for e in x if isinstance(e, basestring)]), axis=1)

#Remove any rows that do not have an ID in the new field
#This is not removing the rows that do not have a combined ID value
df_merged = df_merged[pd.notnull(df_merged['Combined_ID'])]

Upvotes: 1

Views: 2788

Answers (1)

Andy Hayden
Andy Hayden

Reputation: 375675

This column is never going to be null. If every item in the row is not a basestring then the function returns ''.

Therefore the following should work:

df_merged = df_merged[df_merged['Combined_ID'] != '']

Upvotes: 2

Related Questions