CODEWITHSUNDEEP
pythonpandas
CandleWax
CandleWax

Reputation: 2219

How to identify certain rows within a specified range of columns?

I have a df where I need to identify any rows that have a value found in a list, within columns which are also in another list.
For this example I need to identify any rows that have a value of J,Q,R within any column that starts with 'Month'.
The final df will have a column that shows true or false if either of the letters exist within the list of columns.

df = pd.DataFrame({'KEY': ['1312', '1345', '5555', '5555','5555'], 
              'Month1': [1, 1, 1,1,1],
              'Month2': [1, 1, 'J',1,1],
              'Month3': [1, 1, 1,1,1],
              'Month4': [1, 'J', 1,1,1],
              'Month5': [1, 1, 1,0,0],
              'Month6': [1, 1, 1,0,0],
              'Date1': [20120304, 20120102, 20120203,20120402,4],
              'Date2': [20120405,20120104,20120502,20120501,4],
              'StartMonth': [3,1,1,4,3],
              'EndMonth': [4,1,3,5,5],
              'ID': [1,2,3,3,4]})

df[['KEY','ID','Date1','Date2','StartMonth','EndMonth','Month1', 'Month2','Month3','Month4','Month5','Month6']]

Expected outcome:

    Date1       Date2       EndMonth    ID  KEY     Month1  Month2  Month3  Month4  Month5  Month6  StartMonth  HemoFacB
0   20120304    20120405    4           1   1312    1       1       1       1       1       1       3           False
1   20120102    20120104    1           2   1345    1       1       1       J       1       1       1           True
2   20120203    20120502    3           3   5555    1       J       1       1       1       1       1           True
3   20120402    20120501    5           3   5555    1       1       1       1       0       0       4           False
4   4           4           5           4   5555    1       1       1       1       0       0       3           False

My initial attempt resulted in the error below:

codes = ['J','Q','R']
cols = [col for col in df if col.startswith(('Month'))]
df['HemoFacB'] = np.where(df[cols].isin(codes),1,0)

ValueError: Wrong number of items passed 6, placement implies 1

Upvotes: 3

Views: 458

Answers (2)

CandleWax
CandleWax

Reputation: 2219

I forgot to add .any().

The code below works.

df['HemoFacB'] = np.where(df[cols].isin(codes),1,0).any(1)

The error suggests that I was trying to compare too many (6 cols) items into 1 result. By using .any(), this function returns 'True' if any of the iterables (cols) = 'True', and false if the iterable returned all 'False', ultimately reducing the number of items to just 1. So by adding .any(1) to the end, the script consolidates the 6 items passed to just 1 item.

Upvotes: 6

wolfbagel
wolfbagel

Reputation: 478

Here's a solution that doesn't use numpy. I didn't use all of the fields but I'm sure you'll understand it. Also, I used a DataFrame last after manipulating my dictionary. I find it much easier to do that. 

import pandas as pd

mydict = {'KEY': ['1312', '1345', '5555', '5555','5555'], 'Month1': [1, 'J', 3,4,'J']}

#print(df)

truth_list = []
for val in zip(*mydict.values()):
    #print(val)
    #print("This is key: {} and value: {}".format(key, val))
    if 'J' in val:
        #print("True")
       truth_list.append('True')
    else:
        #print("False")
        truth_list.append('False')
    #print("Row {}".format(row = row + 1))

mydict.update({'HemoFacB': truth_list})

df = pd.DataFrame(mydict)
print(df)

Upvotes: 1

Related Questions