Reputation: 2155
I am currently trying to implement a model using Batch Normalization in Keras. I have succesfully implemented it for the training phase.
However, for testing, Batch Normalization calculates the statistics (mean and variance) of the entire population before doing a forward pass through the network (the BN mean and variance are pre-calculated, and then kept static; this is in contrast to the training phase, where the mean and variance are determined by the batch).
My question regarding Keras is:
Assume (X, y) is the entire population. Assume (X_batch, y_batch) is a batch (a subset of that entire population)
If I use the
model.test_on_batch(X_batch, y_batch)
how can I pass on to the batch-normalization layer the mean and variance of the entire population for X and y? Is there any way I can let keras handle this automatically?
Upvotes: 12
Views: 6978
Reputation: 12617
how can I pass on to the batch-normalization layer the mean and variance of the entire population for X and y? Is there any way I can let keras handle this automatically?
Keras should just do it (in sufficiently recent versions):
https://github.com/fchollet/keras/issues/81
To double-check, you may want to try batch_size=1
at test/prediction time, and if Keras fails to use the global statistics, you'll probably see very bad results.
Upvotes: 9