CODEWITHSUNDEEP
pytorch
mcExchange
mcExchange

Reputation: 6496

Pytorch: Intermediate testing during training

How can I test my pytorch model on validation data during training? I know that there is the function myNet.eval() which apparantly switches of any dropout layers, but is it also preventing the gradients from being accumulated?
Also how would I undo the myNet.eval() command in order to continue with the training?

If anyone has some code snippet / toy example I would be grateful!

Upvotes: 5

Views: 5301

Answers (2)

nemo
nemo

Reputation: 57599

How can I test my pytorch model on validation data during training?

There are plenty examples where there are train and test steps for every epoch during training. An easy one would be the official MNIST example. Since pytorch does not offer any high-level training, validation or scoring framework you have to write it yourself. Commonly this consists of

  • a data loader (commonly based on torch.utils.dataloader.Dataloader)
  • a main loop over the total number of epochs
  • a train() function that uses training data to optimize the model
  • a test() or valid() function to measure the effectiveness of the model given validation data and a metric

This is also what you will find in the linked example.

Alternatively you can use a framework that provides basic looping and validation facilities so you don't have to implement everything by yourself all the time.

  • tnt is torchnet for pytorch, supplying you with different metrics (such as accuracy) and abstraction of the train loop. See this MNIST example.
  • inferno and torchsample attempt to model things very similar to Keras and provide some tools for validation
  • skorch is a scikit-learn wrapper for pytorch that lets you use all the tools and metrics from sklearn

Also how would I undo the myNet.eval() command in order to continue with the training?

myNet.train() or, alternatively, supply a boolean to switch between eval and training: myNet.train(True) for train mode.

Upvotes: 4

Siyuan Ren
Siyuan Ren

Reputation: 7844

I know that there is the function myNet.eval() which apparantly switches of any dropout layers, but is it also preventing the gradients from being accumulated?

It doesn't prevent gradients from accumulating.

But I think during testing, you do want to ignore gradients. In that case, you should mark the variable input to the network as volatile=True, and it will save some time and space used in forward calculation.

Also how would I undo the myNet.eval() command in order to continue with the training?

myNet.train()

Upvotes: 1

Related Questions