CODEWITHSUNDEEP
pythontensorflowkerastensorflow2.0python-3.8
revy
revy

Reputation: 667

Error when creating a model with a custom activation fuction

I'm trying to implement a custom activation function (pentanh) based on the Tanh activation. However, when I add this fuction to my model, it raises a ValueError.

The custom activation fuction:

import tensorflow.keras.backend as K
from tensorflow.keras.layers import Layer


class Pentanh(Layer):

    def __init__(self, **kwargs):
        super(Pentanh, self).__init__(**kwargs)
        self.supports_masking = True
        self.__name__ = 'pentanh'

    def call(self, inputs):
        return K.switch(K.greater(inputs, 0), K.tanh(inputs), 0.25 * K.tanh(inputs))

    def get_config(self):
        return super(Pentanh, self).get_config()

    def compute_output_shape(self, input_shape):
        return input_shape

I use the custom activation function when adding a LSTM layer to my model:

layer_lstm = Bidirectional(LSTM(256, activation="pentanh", return_sequences=True))(layer_embeddings)

And, before creating the model, I update Keras' custom objects:

from tensorflow import keras
keras.utils.get_custom_objects().update({'pentanh': Pentanh()})

The error:

File "C:\Users\PycharmProjects\code\util.py", line 106, in create_model
  layer_lstm = Bidirectional(LSTM(256, activation="pentanh", return_sequences=True))(layer_embeddings)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\wrappers.py", line 418, in __init__
  self.forward_layer = self._recreate_layer_from_config(layer)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\wrappers.py", line 494, in _recreate_layer_from_config
  return layer.__class__.from_config(config)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 2882, in from_config
  return cls(**config)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1057, in __init__
  super(LSTM, self).__init__(
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 1103, in __init__
  super(DropoutRNNCellMixin, self).__init__(*args, **kwargs)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 2729, in __init__
  cell = LSTMCell(
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 2324, in __init__
  self.activation = activations.get(activation)
File "C:\Users\env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
  return target(*args, **kwargs)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\activations.py", line 531, in get
  return deserialize(identifier)
File "C:\Users\env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
  return target(*args, **kwargs)
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\activations.py", line 488, in deserialize
  return deserialize_keras_object(
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\utils\generic_utils.py", line 346, in deserialize_keras_object
  (cls, cls_config) = class_and_config_for_serialized_keras_object(
File "C:\Users\env\lib\site-packages\tensorflow\python\keras\utils\generic_utils.py", line 296, in class_and_config_for_serialized_keras_object
  raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)

ValueError: Unknown activation function: Pentanh

Upvotes: 0

Views: 318

Answers (1)

Kaveh
Kaveh

Reputation: 4980

I don't know really why your function does not work only for LSTM layers. It works for example for a dense layer. However in order to resolve your problem I defined this as a function and it works also for a LSTM layer.

Here is the code:

def my_pentanh(inputs):
    return K.switch(K.greater(inputs, 0), K.tanh(inputs), 0.25 * K.tanh(inputs))

keras.utils.get_custom_objects().update({'my_pentanh': my_pentanh})

and the LSTM layer:

tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True, activation='my_pentanh')),

Upvotes: 1

Related Questions