CODEWITHSUNDEEP
deep-learningpytorchmeta-learningfew-shot-learninglearn2learn
The Exile
The Exile

Reputation: 704

Too much fluctuation in F1 Score curve during meta training with MAML

I am training VGG11 on a custom image dataset for 3-way 5-shot image classification using MAML from learn2learn. I am encapsulating the whole VGG11 model with MAML, i.e., not just the classification head. My hyperparameters are as follows:

During the training, I noticed that after taking the first outer-loop optimization step, i.e., AdamW.step(), loss skyrockets to very large values, like ten thousands. Is this normal? Also, I am measuring the micro F1 score as accuracy metric of which curve for meta training/validation is as follows: Meta Train/Val F1 Score curve

It is fluctuating too much in my opinion, is this normal? What could be the reason of this? Thanks

Upvotes: 0

Views: 134

Answers (1)

The Exile
The Exile

Reputation: 704

I figured it out. I was using VGG11 with vanilla BatchNorm layers from PyTorch which was not working properly in meta training setup. I removed BatchNorm layers and now it works as expected.

Upvotes: 0

Related Questions