CODEWITHSUNDEEP
pythonpandas
RDJ
RDJ

Reputation: 4122

Pandas: Conditional statement doesn't work as expected

Given the following dummy data my aim is to determine if an employee was employed at the end of 2014 and return a new boolean column to this effect.

name    hire_date    termination_date   grade_2014
James   1999-01-01   NaT                10.0
Amara   2015-12-07   NaT                NaN
Haj     2012-08-13   2016-04-04         9.0
Bill    1999-01-12   2014-02-04         7.0

I've written the following list comp to achieve this.

from itertools import izip
df['active_end_2014'] = ['true' if
                     (hire < pd.Timestamp(2014, 12, 31) and termination == pd.NaT) |
                     (termination > pd.Timestamp(2015, 1, 1) and grade_2014 != np.nan)
                     else 'false' for grade_2014, termination, hire in izip(df['grade_2014'],
                                                                           df['termination_date'],
                                                                           df['hire_date'])]

The correct boolean is returned for all employees but James who get 'false'.

df[df['name'] == 'James']

name    hire_date   termination_date    grade_2014  active_end_2014
James   1999-01-01  NaT                 10.0        false

Why isn't he assigned 'true', as surely he fulfills this condition:

hire < pd.Timestamp(2014, 12, 31) and termination == pd.NaT

Is this an issue with the parentheses or the selection of pd.Nat? Or perhaps how I'm constructing the list comp more broadly?

Upvotes: 1

Views: 247

Answers (2)

jf328
jf328

Reputation: 7351

You are comparing NaN's with ==, which will result in False. Use pd.isnull.

>>> pd.NaT == pd.NaT
False
>>> pd.isnull(pd.NaT)
True

Upvotes: 1

MaxU - stand with Ukraine
MaxU - stand with Ukraine

Reputation: 210842

you should use boolean indexing properly:

In [81]: df['active_end_2014'] = \
    ...:     ((df.hire_date < '2014-12-31') & df.termination_date.isnull()) | \
    ...:     ((df.termination_date > '2015-01-01') & df.grade_2014.notnull())

In [82]: df
Out[82]:
    name  hire_date termination_date  grade_2014 active_end_2014
0  James 1999-01-01              NaT        10.0            True
1  Amara 2015-12-07              NaT         NaN           False
2    Haj 2012-08-13       2016-04-04         9.0            True
3   Bill 1999-01-12       2014-02-04         7.0           False

Upvotes: 2

Related Questions