CODEWITHSUNDEEP
pythonrelu
i.meant.to.be
i.meant.to.be

Reputation: 31

ValueError with ReLU function in python

I declared ReLU function like this:

def relu(x):
    return (x if x > 0 else 0)

and an ValueError has occured and its traceback message is

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

But if I change ReLU function with numpy, it works:

def relu_np(x):
    return np.maximum(0, x)

Why this function(relu(x)) doesn't work? I cannot understand it...

================================

Used code:

>>> x = np.arange(-5.0, 5.0, 0.1)
>>> y = relu(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "filename", line, in relu
    return (x if x > 0 else 0)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Upvotes: 2

Views: 360

Answers (3)

Akshay Sehgal
Akshay Sehgal

Reputation: 19322

TLDR; Your first function is not using vectorized methods which means it expects a single float/int value as input, while your second function takes advantage of Numpy's vectorization.

Vectorization in NumPy

Your second function uses numpy functions which are vectorized and run on each individual element of the array.

import numpy as np

arr = np.arange(-5.0, 5.0, 0.5)

def relu_np(x):
    return np.maximum(0, x)

relu_np(arr)

# array([0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.5, 1. ,
#        1.5, 2. , 2.5, 3. , 3.5, 4. , 4.5])

Your second function however uses a ternary operator (x if x > 0 else 0) which expects a single value input and outputs a single value. This is why when you pass a single element, it would work, but on passing an array it fails to run the function on each element independently.

def relu(x):
    return (x if x > 0 else 0)

relu(-8)

## 0

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Note: The reason for this error is due to the ternary operator you are using (x if x > 0 else 0). The condition x>0 can only take the value True or False for a given integer/float value. However, when you pass an array, you will need to use something like any() or all() to aggregate that list of boolean values to a single one, before you can apply your if, else clause.

Solutions -

There are a few ways you can make this work -

1. Using np.vectorize (not recommended, lower performance than pure numpy approach)

import numpy as np

arr = np.arange(-5.0, 5.0, 0.5)

def relu(x):
    return (x if x > 0.0 else 0.0)

relu_vec = np.vectorize(relu)
relu_vec(arr)

# array([0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.5, 1. ,
#        1.5, 2. , 2.5, 3. , 3.5, 4. , 4.5])

2. Iteration over the array with list comprehension

import numpy as np

arr = np.arange(-5.0, 5.0, 0.5)

def relu(x):
    return (x if x > 0 else 0)

arr = np.array(arr)

np.array([relu(i) for i in arr])

# array([0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.5, 1. ,
#        1.5, 2. , 2.5, 3. , 3.5, 4. , 4.5])

Upvotes: 2

Colin Bernet
Colin Bernet

Reputation: 1394

Keep in mind that x > 0 is an array of booleans, a mask if you like:

array([False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True,  True,  True,  True,  True,  True,  True,  True,  True,
        True])

So it does not make sense to do if x>0, since x contains several elements, which can be True or False. This is the source of your error.

Your second implementation of numpy is good ! Another implementation (maybe more clear?) might be:

def relu(x):
  return x * (x > 0)

In this implementation, we do an elementwise multiplication of x, which is a range of values along the x axis, by 0 if the element of x is below 0, and 1 if the element is above.

Upvotes: 1

Tobi208
Tobi208

Reputation: 1376

Disclaimer: please someone correct me if I'm wrong, I'm not 100% sure about how numpy does things.

Your function relu expects a single numerical value and compares it to 0 and returns whatever is larger. x if x > 0 else 0 would be equal to max(x, 0) where max is a builtin Python function.

relu_np on the other hand uses the numpy function maximum which accepts 2 numbers or arrays or iterables. That means you can pass your numpy array x and it applies the maximum function to every single item automatically. I believe this is called 'vectorized'.

To make the relu function you have work the way it is, you need to call it differently. You'd have to manually apply your function to every element. You could do something like y = np.array(list(map(relu, x))).

Upvotes: 1

Related Questions