How to customize Keras layer names and also have it automatically increment layer.name

Question

I am currently trying to have multiple layers with a customized activation with the name cust_sig. But when I try to compile the model, I get a ValueError raised as multiple layers have the same name cust_sig. I am aware that I can manually change the name for every layer but wanted to know if there is something that can be done to automatically have _1, _2, ... added to the name as it does for in-built layers. The model definition can be found below.

# Creating a model
from tensorflow.python.keras import keras
from tensorflow.python.keras.models import Model
from tensorflow.python.keras.layers import Dense

# Custom activation function
from tensorflow.python.keras.layers import Activation
from tensorflow.python.keras import backend as K
from keras.utils.generic_utils import get_custom_objects

def custom_activation(x):
    return (K.sigmoid(x) * 5) - 1

get_custom_objects().update({'custom_activation': Activation(custom_activation)})

data_format = 'channels_first'

spec_input = keras.layers.Input(shape=(1, 3, 256), name='spec')
x = keras.layers.Flatten(data_format)(spec_input)

for layer in range(3):
  x = Dense(512)(x)
  x = Activation('custom_activation', name='cust_sig')(x)

out = Dense(256, activation="sigmoid", name='out')(x)
model = Model(inputs=spec_input, outputs=out)

The error message is shown below

Traceback (most recent call last):
  File "/home/xyz/anaconda3/envs/ctf/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/home/xyz/anaconda3/envs/ctf/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 315, in _init_graph_network
    self.inputs, self.outputs)
  File "/home/xyz/anaconda3/envs/ctf/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1861, in _map_graph_network
    str(all_names.count(name)) + ' times in the model. '
ValueError: The name "cust_sig" is used 3 times in the model. All layer names should be unique.

Answer 1

The Tensorflow library does not provide auto incrementation for custom names. You have to create a whole new Layer Class, which is impractical.

The following trick is the simplest method to create an Activation layer and name it. Feel free to copy-paste, it requires no changes:

def custom_activation_layer(activation, name=None):
    class_name = tf.keras.layers.Activation.__name__

    # Set the desired name here. We default to the function name.
    if name:
        tf.keras.layers.Activation.__name__ = name
    else:
        tf.keras.layers.Activation.__name__ = f'Activation_{activation.__name__}'
    
    # The layer name is set in the constructor.
    activation_layer = tf.keras.layers.Activation(activation)

    tf.keras.layers.Activation.__name__ = class_name
    return activation_layer

Simply replace your activation layers with this function:

    # Replace the activation layer...
    layer = tf.keras.layers.Activation(my_custom_activation)

    # ... With the function.
    layer = custom_activation_layer(my_custom_activation)
    

When plotting your model's graph, the name is auto-incremented, avoiding graph Errors. It displays as such:

model = create_model()
tf.keras.utils.plot_model(model, show_shapes=True)

Answer 2

If you want to use a specific_name multiple times with number suffix, use this:

tf.get_default_graph().unique_name("specific_name")

or

tf.compat.v1.get_default_graph().unique_name("specific_name")

In your case:

...
for layer in range(3):
  x = Dense(512)(x)
  x = Activation('custom_activation', name=tf.get_default_graph().unique_name("cust_sig"))(x)
...

Answer 3

Below should do:

def custom_activation(x):
    return (K.sigmoid(x) * 5) - 1

class CustSig(Layer):
    def __init__(self, my_activation, **kwargs):
        super(CustSig, self).__init__(**kwargs)
        self.supports_masking = True
        self.activation = my_activation

    def call(self, inputs):
        return self.activation(inputs)

    def get_config(self):
        config = {'activation': activations.serialize(self.activation)}
        base_config = super(Activation, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

    def compute_output_shape(self, input_shape):
        return input_shape

---

Explanation:

From source code, automatic naming works as follows:

if not name:
  self._name = backend.unique_object_name(
      generic_utils.to_snake_case(self.__class__.__name__),
      zero_based=zero_based)
else:
  self._name = name

The Keras graph is checked for existing objects with the same name as the object you're defining - if any exist, continues to increment by 1 until none match. The catch is, you cannot specify name=, as that eliminates automatic naming per above conditional.

The only workaround is likely defining your own custom activation layer using desired name as class name as above, which manifests itself as follows:

ipt = Input(shape=(1, 3, 256), name='spec')
x   = Flatten('channels_last')(ipt)
for _ in range(3):
    x   = Dense(512)(x)
    x   = CustSig(custom_activation)(x)
out = Dense(256, activation='sigmoid', name='out')(x)

model = Model(ipt, out)

print(model.layers[3].name)
print(model.layers[5].name)
print(model.layers[7].name)

cust_sig
cust_sig_1
cust_sig_2

Answer 4

If you check the source code of Layer class, you can find these lines that decide the name of layer.

if not name:
    prefix = self.__class__.__name__
    name = _to_snake_case(prefix) + '_' + str(K.get_uid(prefix))
self.name = name

K.get_uid(prefix) will get the unique id from the graph, that's why you see activation_1, activation_2.

If you want to have the same effect on your customized activation function, a better way is to define your own class which inherits from Layer.

class MyAct(Layer):
    def __init__(self):
        super().__init__()

    def call(self, inputs):
        return (K.sigmoid(inputs) * 5) - 1 

spec_input = Input(shape=(10,10))
x = Flatten()(spec_input)
for layer in range(3):
    x = Dense(512)(x)
    x = MyAct()(x)

model = Model(spec_input, x)
model.summary()

Output

# Layer (type)                 Output Shape              Param #   
# =================================================================
# input_1 (InputLayer)         (None, 10, 10)            0         
# _________________________________________________________________
# flatten_1 (Flatten)          (None, 100)               0         
# _________________________________________________________________
# dense_1 (Dense)              (None, 512)               51712     
# _________________________________________________________________
# my_act_1 (MyAct)             (None, 512)               0         
# _________________________________________________________________
# dense_2 (Dense)              (None, 512)               262656    
# _________________________________________________________________
# my_act_2 (MyAct)             (None, 512)               0         
# _________________________________________________________________
# dense_3 (Dense)              (None, 512)               262656    
# _________________________________________________________________
# my_act_3 (MyAct)             (None, 512)               0         
# =================================================================
# Total params: 577,024
# Trainable params: 577,024
# Non-trainable params: 0