CODEWITHSUNDEEP
pythonpytorchhuggingface-transformers
Moritz Blum
Moritz Blum

Reputation: 103

Why does the evaluation loss increases when training a huggingface transformers NER model?

Training a huggingface transformers NER model according to the documentation, the evaluation loss increases after a few epochs, but the other scores (accuracy, precision, recall, f1) continuously getting better. The behaviour seems unexpected, is there a simple explanation for this effect? Can this depend on the given data?

enter image description here

model = TokenClassificationModel.from_pretrained('roberta-base', num_labels=len(tag_values))


model.train()
model.zero_grad()

   for epoch in range(epochs):
      for batch in range(batches):
         -- train --
         ...

      train_loss = model.evaluate(train_data)
      validation_loss = model.evaluate(validation_data)

Upvotes: 1

Views: 2207

Answers (1)

Ohad Bruker
Ohad Bruker

Reputation: 524

Accuracy and loss are not necessarily exactly (inversely) correlated.
The loss function is often an approximation of the accuracy function - unlike accuracy, the loss function must be differentiable.
A good explanation can be found here.

Upvotes: 1

Related Questions