CODEWITHSUNDEEP
python-3.xpandasjoinlambdaapply
SFC
SFC

Reputation: 793

Use lambda, apply, and join function on a pandas dataframe

Goal

Apply deid_notes function to df

Background

I have a df that resembles this sample df

import pandas as pd
df = pd.DataFrame({'Text' : ['there are many different types of crayons', 
                                   'i like a lot of sports cares', 
                                   'the middle east has many camels '], 

                      'P_ID': [1,2,3], 
                      'Word' : ['crayons', 'cars', 'camels'],
                      'P_Name' : ['John', 'Mary', 'Jacob'],
                      'N_ID' : ['A1', 'A2', 'A3']

                     })

#rearrange columns
df = df[['Text','N_ID', 'P_ID', 'P_Name', 'Word']]
df

    Text                  N_ID P_ID P_Name  Word
0   many types of crayons   A1  1    John   crayons
1   i like sports cars      A2  2    Mary   cars
2   has many camels         A3  3    Jacob  camels

I use the following function to deidentify certain words within the Text column using NeuroNER http://neuroner.com/

def deid_notes(text):

    #use predict function from neuorNER to tag words to be deidentified 
    ner_list = n1.predict(text)      

    #n1.predict wont work in this toy example because neuroNER package needs to be installed (and installation is difficult) 
    #but the output resembles this: [{'start': 1, 'end:' 11, 'id': 1, 'tagged word': crayon}]

    #use start and end position of tagged words to deidentify and replace with **BLOCK**
    if len(ner_list) > 0:
        parts_to_take = [(0, ner_list[0]['start'])] + [(first["end"]+1, second["start"]) for first, second in zip(ner_list, ner_list[1:])] + [(ner_list[-1]['end'], len(text)-1)] 
        parts = [text[start:end] for start, end in parts_to_take] 
        deid = '**BLOCK**'.join(parts)

    #if n1.predict does not identify any words to be deidentified, place NaN 
    else:
        deid='NaN'

    return pd.Series(deid, index='Deid')

Problem

I apply the deid_notes function to my df using the following code

fx = lambda x: deid_notes(x.Text,axis=1)
df.join(df.apply(fx))

But I get the following error 

AttributeError: ("'Series' object has no attribute 'Text'", 'occurred at index Text')

Question

How do I get the deid_notes function to work on my df?

Upvotes: 2

Views: 3424

Answers (1)

Mankind_2000
Mankind_2000

Reputation: 2218

Assuming you are returning a pandas series as output from deid_notes function taking text as the only input argument. Pass the axis = 1 argument to the apply instead of died_notes. For eg. 

# Dummy function
def deid_notes(text):
    deid = 'prediction to: ' + text
    return pd.Series(deid, index = ['Deid'])

fx = lambda x: deid_notes(x.Text)
df.join(df.apply(fx, axis =1))

Upvotes: 1

Related Questions