CODEWITHSUNDEEP
tensorflowkerasactivation-function
Reza kardoost
Reza kardoost

Reputation: 1

How to create PSS and SS activation functions in Keras

I'm using keras with tensorflow backend. My goal is to add my custom activation functions (PSS, SS) but I don't know how can i implement them.

Positive Smooth Staircase (PSS) activation function

Where n is number of output labels, w is a constant.

Smooth Staircase (SS) activation function

Where n is number of output labels (output neurons), c is a constant.

Upvotes: 0

Views: 48

Answers (1)

TF_Chinmay
TF_Chinmay

Reputation: 151

To add a custom activation function in Keras, you can define a Python function that takes a NumPy array as input and returns a NumPy array as output.

Your function should also support automatic differentiation, which means that Keras should be able to automatically compute the derivative of your function in order to backpropagate gradients during training.

Once you have defined your custom activation function, you can add it to Keras as a custom layer. To do this, you can create a subclass of the keras.layers.Layer class. In your subclass, you should implement the build() and call() methods.

The build() method is responsible for initializing any weights or biases that your layer may need. The call() method is responsible for performing the forward pass of your layer.

In the call() method, you should apply your custom activation function to the input data.

You can refer this code and insert your formula in there :

import keras

model = keras.Sequential([
    keras.layers.Dense(128, activation=my_custom_activation_function, input_shape=(10,)),
    keras.layers.Dense(64, activation='relu'),
    keras.layers.Dense(10, activation='softmax')
])

# Compile the model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

# Train the model
model.fit(x_train, y_train, epochs=10)

# Evaluate the model on the test set
loss, accuracy = model.evaluate(x_test, y_test)

# Print the accuracy
print('Accuracy:', accuracy)

Upvotes: 0

Related Questions