CODEWITHSUNDEEP
pythonpandas
financial_physician
financial_physician

Reputation: 1978

Select rows where two specific columns are both non-null

I essentially want to do this 

mrgd.query('security == security and comnam == comnam')

But this just false always.

Alternatively I was thinking maybe I could do

mrgd.eval('security == security') & mrgd.eval(comnam == comnam')

But this doesn't work.

mrgd['security'].notnull() == mrgd['comnam'].notnull()

I suppose I could do this

mrgd[['security','comnam']].notnull()

and then sum the two columns generated to get an integer (2 would mean it was successful) but what's the conventional way for doing this? I've found questions similar to this but not quite the same thing.

Upvotes: 5

Views: 7444

Answers (2)

Andy L.
Andy L.

Reputation: 25239

The recommended way is using isna, notna vs isnull, notnull. On checking both columns, just chain additional all on axis=1

Check both columns are Non-NaN

mrgd[['security','comnam']].notna().all(1)

Check either columns are Non-NaN

mrgd[['security','comnam']].notna().any(1)

Upvotes: 4

Andrei Berenda
Andrei Berenda

Reputation: 1986

I think you can do like this:

import pandas as pd

df = pd.DataFrame(
    [[None, None], 
     [3, None], 
     [None, 2], 
     [2, 3]], 
    columns=['security', 'comnam',],
)
df[df['security'].notnull() & df['comnam'].notnull()]

The result is: 

  security  comnam
3   2.0 3.0

I have tried:

df.query('security == security and comnam == comnam')

And the result is the same.

Upvotes: 5

Related Questions