CODEWITHSUNDEEP
pythonpandasdataframecriteriaminimum
cwse
cwse

Reputation: 584

Pandas Check for mulitple minimum consequtive criterias

I have a scenario where I want to check for a minimum criterion (0.6) being met over consecutive pandas dataframe rows in a column (Col1), which also meets a criterion when the starting value is at least (0.7) i.e.:

Col1
0.3
0.5
0.55
0.8 = true
0.65 = true
0.9 = true
0.61 = true
0.3
0.6
0.67
0.74 = true
0.63 = true
0.61 = true

In other words, the check would be True if the value is at least 0.7, or if the value is at least 0.6 and the previous values are at least 0.6 with the first value in the consecutive series being at least 0.7.

It will be running off a very large data set so needs to be efficient. I am thinking something with shift() would work...but can't get it quite right.

Upvotes: 0

Views: 46

Answers (1)

akuiper
akuiper

Reputation: 214957

You can use Series.where() to construct the logical Series.

Steps:

  • initialize the Series with nan values;
  • assign True for all values larger than 0.7
  • assign False for all values smaller than 0.6
  • forward fill values between 0.6~0.7 as it depends on previous values
  • fill possible missing values at the beginning of the Series
  • convert the dtype to boolean (optional)

so:

import pandas as pd
import numpy as np
df['check'] = np.nan
df['check'] = (df['check'].where(df.Col1 < 0.7, True)
                          .where(df.Col1 > 0.6, False)
                          .ffill().fillna(False)
                          .astype(bool))

enter image description here

Upvotes: 3

Related Questions