CODEWITHSUNDEEP
tensorflowcomputer-visionconv-neural-networkregularized
Olivia Brown
Olivia Brown

Reputation: 642

How to implement L2 regularized cost function for Convolutional Neural Network

I have implemented a CNN model for digit classification. My model is overfitting a lot, In order to overcome overfitting I'm trying to use L2 Regularization in my cost function. I have a small confusion how can I select <weights> to put in the cost equation (last line of the code).

...

x = tf.placeholder(tf.float32, shape=[None, img_size, img_size, num_channels], name='x') # Input
y_true = tf.placeholder(tf.float32, shape=[None, num_classes], name='y_true') # Labels

<Convolution Layer 1>

<Convolution Layer 2>

<Convolution Layer 3>

<Fully Coonected 1>

<Fully Coonected 2> O/P = layer_fc2 

# Loss Function
lambda = 0.01
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=layer_fc2, labels=y_true)
# cost        = tf.reduce_mean(cross_entropy) # Nornmal Loss
cost          = tf.reduce_mean(cross_entropy + lambda * tf.nn.l2_loss(<weights>)) # Regularized Loss

...

Upvotes: 1

Views: 867

Answers (1)

rkellerm
rkellerm

Reputation: 5512

You should define the L2 loss with respect to the weights - use trainable_variables for that:

C = tf.nn.softmax_cross_entropy_with_logits(logits=layer_fc2, labels=y_true)
l2_loss = tf.add_n([tf.nn.l2_loss(v) for v in tf.trainable_variables()])
C = C + lambda * l2_loss

Upvotes: 3

Related Questions