CODEWITHSUNDEEP
pythontensorflowneural-networkkeras
AR_
AR_

Reputation: 478

Error using 'selu' activation function with Keras

I'm using Keras with Tensorflow backend. When I'm trying to use the 'selu' activation function using: 

model.add(Dense(32, input_shape=(input_length - 1,)))
model.add(Activation('selu'))

The error I get is: 

ValueError: Unknown activation function:selu

Is there any solution to this?

Upvotes: 1

Views: 2887

Answers (1)

Wilmar van Ommeren
Wilmar van Ommeren

Reputation: 7689

Selu is not in your activations.py of keras (most likely because it was added Jun 14, 2017, only 22 days ago). You can just add the missing code in the activations.py file or create your own selu activation in the script.

Example code

from keras.activations import elu

def selu(x):
    """Scaled Exponential Linear Unit. (Klambauer et al., 2017)
    # Arguments
        x: A tensor or variable to compute the activation function for.
    # References
        - [Self-Normalizing Neural Networks](https://arxiv.org/abs/1706.02515)
    """
    alpha = 1.6732632423543772848170429916717
    scale = 1.0507009873554804934193349852946
    return scale * elu(x, alpha)

model.add(Dense(32, input_shape=(input_length - 1,)), activation=selu)

NOTE:

With tensorflow 2.0 keras is included. You can get the selu activation with:

from tensorflow.keras.activations import selu

Upvotes: 9

Related Questions