CODEWITHSUNDEEP
python-3.xtensorflowkerasnltkpos-tagger
Celso França
Celso França

Reputation: 734

How to calculate pos tagger accuracy by each tag

I've made a keras model pos tagger using the NLP4Hackers' article as baseline. Currently I can computate the accuracy directly over the keras model.evaluate method. Acctualy, I would like to calculate the accuracy by tag as show below:

'JJ': 98.56 accuracy,

'NNS': 99.01 accuracy

'NN': 96.43 accuracy

...

Any suggestion will be appreciated.

Thank you.

Upvotes: 1

Views: 1320

Answers (1)

Gabriel M
Gabriel M

Reputation: 1514

All the evaluation metrics that you can imagine are in Scikit-learn

You have two possibilities, either you compute the confusion matrix and you look the diagonal values.

y_true = ["cat", "ant", "cat", "cat", "ant", "bird"]
y_pred = ["ant", "ant", "cat", "cat", "ant", "cat"]
confusion_matrix(y_true, y_pred, labels=["ant", "bird", "cat"])
array([[2, 0, 0],
       [0, 0, 1],
       [1, 0, 2]])

Or you compute the F1 score label by label

from sklearn.metrics import f1_score
y_true = [0, 1, 2, 0, 1, 2]
y_pred = [0, 2, 1, 0, 0, 1]
f1_score(y_true, y_pred, average=None)

Upvotes: 3

Related Questions