CODEWITHSUNDEEP
tensorflowkeras
Oved D
Oved D

Reputation: 7452

what is the keras equivalent of this tensorflow softmax loss + l2 regularization

I came across this code I want to convert to keras:

l2 = lambda_loss_amount * sum(
   tf.nn.l2_loss(tf_var) for tf_var in tf.trainable_variables()
) # L2 loss prevents this overkill neural network to overfit the data
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y, logits=pred)) + l2 # Softmax loss

How would this be written as a Keras loss function?

Upvotes: 2

Views: 556

Answers (2)

edkeveked
edkeveked

Reputation: 18401

You can use the activation and kernel_regularizer on keras layer as the following:

Dense(..., activation='softmax', kernel_regularizer=regularizers.l2(0))

Upvotes: 1

Simon Schrodi
Simon Schrodi

Reputation: 814

See here for a description of regularizers in keras. Here a toy example:

from keras import regularizers
model.add(Dense(64, input_dim=64,
            kernel_regularizer=regularizers.l2(lambda_loss_amount),
            bias_regularizer=regularizers.l2(lambda_loss_amount)))

Upvotes: 1

Related Questions