CODEWITHSUNDEEP
pythonpandasdataframe
user1896796
user1896796

Reputation: 749

Validating dataframe column data

I have a below pseudocode which I need to write using pandas. 

if group_min_size && group_max_size
      if group_min_size == 0 && group_max_size > 0
        if group_max_size >= 2
          errors.add(:group_min_size, "must be greater than or equal to 2 and less than or equal to group_max_size (#{group_max_size})")
        end

        if group_max_size < 2
          errors.add(:group_min_size, "must be greater than 2")
          errors.add(:group_max_size, "must be greater than 2")
        end
      end

      if group_min_size > 0 && group_max_size == 0
        if group_min_size >= 2
          errors.add(:group_max_size, "must be greater than or equal to #{group_min_size}")
        end

        if group_min_size < 2
          errors.add(:group_min_size, "must be greater than 2")
          errors.add(:group_max_size, "must be greater than 2")
        end
      end
    end

I tried to break in smaller parts and write something like below-

m8 = ((~df['group_min_size'].notna() & ~df['group_min_size'].notna()) | ((~df['group_min_size'] == 0) & (~df['group_max_size'] > 2)) | (df['group_max_size'] >= 2))

This is for 

if group_min_size == 0 && group_max_size > 0
        if group_max_size >= 2
          errors.add(:group_min_size, "must be greater than or equal to 2 and less than or equal to group_max_size (#{group_max_size})")
        end

But is not quite working as expected.

Below is my test data - 

   group_min_size  group_max_size
0             0.0             1.0
1            10.0            20.0
2             0.0             3.0
3             3.0             0.0
4             NaN             NaN
5             2.0             2.0
6             2.0             2.0
7             2.0             2.0
8             2.0             2.0

Based on the psudo code logic, the output should be:

False
True 
False
False
True
True
True
True
True

How do I write this logic in pandas?

Upvotes: 0

Views: 111

Answers (1)

user3471881
user3471881

Reputation: 2724

Just answer your questions step by step. Begin by creating your booleans:

min_equal_0 = df['group_min_size'] == 0
min_above_0 = df['group_min_size'] > 0
min_above_equal_2 = df['group_min_size'] >= 2
min_below_2 = df['group_min_size'] < 2

max_equal_0 = df['group_max_size'] == 0
max_above_0 = df['group_max_size'] > 0
max_above_equal_2 = df['group_max_size'] >= 2
max_below_2 = df['group_max_size'] < 2

Now we can look at creating our masks according to the pseudo-code:

first_mask = ~(min_equal_0 & max_above_0 & (max_below_2 | max_above_equal_2))
second_mask = ~(max_equal_0 & min_above_0 & (min_below_2 | min_above_equal_2))

If we combine the two:

>> first_mask & second_mask

0    False
1     True
2    False
3    False
4     True
5     True
6     True
7     True
8     True
dtype: bool

If you want to treat NaN as False, just add them:

min_is_not_null = df['group_min_size'].notnull()
max_is_not_null = df['group_max_size'].notnull()
>> min_is_not_null & max_is_not_null & first_mask & second_mask
0    False
1     True
2    False
3    False
4    False
5     True
6     True
7     True
8     True
dtype: bool

Upvotes: 1

Related Questions