CODEWITHSUNDEEP
machine-learningtensorflowneural-networkdeep-learning
random40154443
random40154443

Reputation: 1140

Dropout in Tensorflow CNN network while using triplet based training

I have implemented distance learning using triplets of data using a CNN in Tensorflow. For this I have a shared set of weights for each layer. I want to employ dropout technique to compensate for over-fitting. I was thinking of doing this in the following way:

h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
h_fc2_drop = tf.nn.dropout(h_fc2, keep_prob)
h_fc3_drop = tf.nn.dropout(h_fc3, keep_prob)

Here, h_fci are the outputs for the three data samples from a previous layer. This poses a potential problem in the sense that dropout is probabilistic hence even if I am providing the same dropout probability, it is possible that different set of neurons are dropped for each of the three outputs whereas ideally I would want the same set of neurons to be used for each. What would be the proper way to implement dropout in this context then ?

Upvotes: 0

Views: 457

Answers (1)

Alexandre Passos
Alexandre Passos

Reputation: 5206

Assuming everything has the same shape, setting the same seed should make them all identical.

In practice though data samples are minibatched, and tf.nn.dropout will drop out the same units in the entire minibatch, so switching to stacking your examples is probably a good idea here.

Upvotes: 1

Related Questions