CODEWITHSUNDEEP
pythonpandasargumentsapplypython-applymap
Neeraj Hanumante
Neeraj Hanumante

Reputation: 1684

Pandas: How to use applymap/apply function with arguements to a dataframe without looping

Background

Link 1 shows that apply can be applied to a Series. I want to use the apply function on a subset of a DataFrame without looping through the columns.

Example code

Creating a sample DataFrame of size 7, 7

def f_test_df(n_rows, n_cols):
    df1 = pd.DataFrame(np.random.rand(n_rows, n_cols))
    df = df1.applymap(lambda x: round(x*10))
    return df 

np.random.seed(seed=1)
df1 = f_test_df(7, 7)

df1

Desired function is should return the same value if the number is within a predefined range, else based on whether it's on the lower or upper side of the limit, corresponding values should be returned. The function to be applied is as follows:

def f_bounds(x, lower, upper):
    if x < lower:
        return 'lower'
    elif x > upper:
        return 'upper'
    else:
        return x

The selected portion of the DataFrame where a function needs to be applied

df1.loc[2:5, 2:5]

Selected portion of df1

Applying the function:

lower = 2
upper = 5
df1.loc[2:5, 2:5].apply(f_bounds, args=(lower, upper))

I encountered the following error:

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Hence, I changed the approach and used looping across the columns, as shown below (which works well):

for j in range(2, 5):
    print(df1.loc[2:5, j].apply(f_bounds, args=(lower, upper)))

Other approaches that were not tested

Link 2 Referring to answer 2 here, advised against using applymap with arguments. So, I did not use applymap because the function requires 2 additional arguments. Readers please note, applymap has been used in the answer.

Desired outcome

I want to implement this function requiring arguements without looping over the columns to a dataframe.

Upvotes: 2

Views: 1784

Answers (2)

Akshay Sehgal
Akshay Sehgal

Reputation: 19332

You can directly use using applymap with a lambda function that takes in the parameters on the window of the DataFrame. Then you can update the view directly to update the original DataFrame -

df1.loc[2:5, 2:5] = df1.loc[2:5, 2:5].applymap(lambda x: f_bounds(x, lower, upper))
print(df1)

    0  1      2      3      4      5  6
0   4  7      0      3      1      1  2
1   3  4      5      4      7      2  9
2   0  7      4  upper  lower      2  8
3  10  3  upper  upper  upper  lower  0
4   2  9  lower      4  upper      5  7
5   3  7  upper  lower  upper  upper  7
6   3  8      1      4      9      3  3

EDIT:

Here is another way to do what you are trying to do without using apply or applymap

cond1 = df1[(df1.loc[2:5, 2:5]<lower)].notna()
cond2 = df1[(df1.loc[2:5, 2:5]>upper)].notna() 
df1_new = df1.where(~cond1, 'lower').where(~cond2, 'upper')
print(df1_new)

    0  1      2      3      4      5  6
0   4  7      0      3      1      1  2
1   3  4      5      4      7      2  9
2   0  7      4  upper  lower      2  8
3  10  3  upper  upper  upper  lower  0
4   2  9  lower      4  upper      5  7
5   3  7  upper  lower  upper  upper  7
6   3  8      1      4      9      3  3

Upvotes: 2

jezrael
jezrael

Reputation: 863176

You can use stack with unstack:

lower = 2
upper = 5
a = df1.loc[2:5, 2:5].stack().apply(f_bounds, args=(lower, upper)).unstack()
print (a)
       2      3      4      5
2      4  upper  lower      2
3  upper  upper  upper  lower
4  lower      4  upper      5
5  upper  lower  upper  upper

Or pass DataFrame to function:

def f_bounds(x, lower, upper):
    m1 = x < lower
    m2 = x > upper
    
    return np.select([m1, m2], ['lower', 'upper'], default=x)

    
lower = 2
upper = 5
a = pd.DataFrame(f_bounds(df1.loc[2:5, 2:5], 2 ,5),
                 index=df1.index[2:6],
                 columns = df1.columns[2:6])

Upvotes: 2

Related Questions