CODEWITHSUNDEEP
pythonpandasdataframe
Abhijith Rao
Abhijith Rao

Reputation: 69

DataFrame.ne return false when the data it is comparing is None type

I have been trying to combine two Pandas dataframes and compare the elements column wise and if they are not equal, paint them some color so it is easily distinguishable. The problem is when I try to compare None values. When both the values are None, the Dataframe.ne method return False, but I want it to return True since both are None.

Here's my code so far:

def highlight_diff(data, color='yellow'):
    attr = 'background-color: {}'.format(color)
    other = data.xs('Original', axis='columns', level=-1)
    return pd.DataFrame(np.where(data.ne(other, level=0), attr, ''),
                    index=data.index, columns=data.columns)

Upvotes: 2

Views: 475

Answers (1)

jezrael
jezrael

Reputation: 863166

One trick is replace None/NaNs to same values only for comparing in both DataFrames:

Notice: Use values for replace which are not in both DataFrames, for avoid compare NaN in df1 and repalced value in df2 as True (False positive)

def highlight_diff(data, color='yellow'):
    attr = 'background-color: {}'.format(color)
    other = data.xs('Original', axis='columns', level=-1)
    return pd.DataFrame(np.where(data.fillna('miss').ne(other.fillna('miss'), level=0), attr, ''),
                    index=data.index, columns=data.columns)

Upvotes: 2

Related Questions