CODEWITHSUNDEEP
pythoncsvpandas
fightstarr20
fightstarr20

Reputation: 12608

Python Pandas - Return default value when match not found

I am comparing 2 CSV files with pandas python and all is working correctly. It is matching on the employee_id column and outputting the results to a csv file

    df1 = pd.read_csv('input1.csv', sep=',\s+', delimiter=',', encoding="utf-8")
    df2 = pd.read_csv('input2.csv', sep=',\s,', delimiter=',', encoding="utf-8")
    df3 = pd.merge(df1,df2, on='employee_id', how='right')
    df3.to_csv('output.csv', encoding='utf-8', index=False)

When it does not find a match it returns a blank result, I would like it to instead return not_found

Is this possible with Pandas or should I do some processing afterwards?

Upvotes: 3

Views: 1344

Answers (1)

jezrael
jezrael

Reputation: 863031

I think you can use fillna if no NaN in df1:

df1 = pd.DataFrame({'employee_id':[1,2,3],
                   'B':[4,5,6],
                   'C':[7,8,9]})

print (df1)
   B  C  employee_id
0  4  7            1
1  5  8            2
2  6  9            3

df2 = pd.DataFrame({'employee_id':[1,4,6],
                   'D':[4,5,6],
                   'E':[7,8,9]})

print (df2)
   D  E  employee_id
0  4  7            1
1  5  8            4
2  6  9            6

df3 = pd.merge(df1,df2, on='employee_id', how='right')
df3[df1.columns] = df3[df1.columns].fillna('not_found')
print (df3)
           B          C  employee_id  D  E
0          4          7            1  4  7
1  not_found  not_found            4  5  8
2  not_found  not_found            6  6  9

But if NaN in df1 is necessary create mask for identify missing values from right join - with parameter indicator=True in merge or with isin and negate mask by ~:

df1 = pd.DataFrame({'employee_id':[1,2,3],
                   'B':[np.nan,5,6],
                   'C':[7,8,9]})

print (df1)
     B  C  employee_id
0  NaN  7            1
1  5.0  8            2
2  6.0  9            3

df3 = pd.merge(df1,df2, on='employee_id', how='right', indicator=True)
mask = df3['_merge'] == 'right_only'
df3.loc[mask, df1.columns.difference(['employee_id'])] = 
df3.loc[mask,df1.columns.difference(['employee_id'])].fillna('not_found')

df3 = df3.drop('_merge', axis=1)
print (df3)
           B          C  employee_id  D  E
0        NaN          7            1  4  7
1  not_found  not_found            4  5  8
2  not_found  not_found            6  6  9

df3 = pd.merge(df1,df2, on='employee_id', how='right')
mask = ~df2['employee_id'].isin(df1['employee_id'])

df3.loc[mask, df1.columns.difference(['employee_id'])] = \
df3.loc[mask,df1.columns.difference(['employee_id'])].fillna('not_found')

print (df3)
           B          C  employee_id  D  E
0        NaN          7            1  4  7
1  not_found  not_found            4  5  8
2  not_found  not_found            6  6  9

Upvotes: 2

Related Questions