CODEWITHSUNDEEP
machine-learningneural-networkconv-neural-network
Hello Lili
Hello Lili

Reputation: 1587

Online training in convolutional neural network

In this article the neural network is trained online - the input is selected during training:

This can be done by selecting the hard positive/negative exemplars from within a mini-batch.

Can someone explain online training in detail?

Upvotes: 1

Views: 1236

Answers (1)

Imran
Imran

Reputation: 13458

Online learning describes training a model one input data point at a time, or one mini-batch at a time. Typically, online learning is used because the data is coming in a stream, or because the data is too large to keep in memory all at once.

Naively training one example at a time will typically take more steps to reach the same accuracy as training offline, because you cannot optimize your loss function against your entire data set in each step.

However, the paper you linked uses online learning to intelligently select training examples a few a time in order to speed up convergence of the model. Their method attempts to select the examples that are currently hardest for the model, so that the model may make the most improvement at each training step.

Upvotes: 2

Related Questions