CODEWITHSUNDEEP
machine-learningkerastf.kerasdropout
mbohde
mbohde

Reputation: 139

Are dropout layers applied to validation data in Keras?

So I know for making predictions based on a model, dropout layers will not be included because it makes the model wrong on purpose. However would these layers still be applied in the validation data tests for each epoch or not?

Upvotes: 2

Views: 1612

Answers (1)

Akshay Sehgal
Akshay Sehgal

Reputation: 19312

The goal of dropout is to ensure that the model does not end up having too much dependency on a set of nodes while ignoring other nodes almost compeletely (which leads to overfitting) and instead forces the model to depend on all the nodes in the network. This is done by applying a bitmask (0s and 1s of a specified dropout proportion) to the input of the nodes during the training process for that layer. This is analogous to turning off a percentage of neurons while training so the model is now forced to minimize the loss while depending on the nodes which are currently on. This bitmask is refreshed for each batch.

Validation is done on out-of-sample data and is an inference task instead of a training task. At the end of each epoch, once the gradients are updated after traversing over the batches from the complete dataset, an inference is made on the validation data. This involves the forward computation of the COMPLETE GRAPH and ignores the dropouts, since they only used to regularize the model during training.

TLDR;

  • Q: Are dropout layers applied to validation data in Keras?
  • A: No.

Reference on Keras documentation -

Note that the Dropout layer only applies when training is set to True such that no values are dropped during inference.

Upvotes: 3

Related Questions