Error "Warning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter"

Question

How can I solve this error?

Warning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use zero_division parameter to control this behavior. _warn_prf(average, modifier, msg_start, len(result))

The error appears when I add Adam's tuning parameter.

# Tuning parameter from keras.optimizers import Adam
optimize = Adam(learning_rate=0.00001, beta_1=0.9, beta_2=0.99)
model.compile(optimizer=optimize, loss='categorical_crossentropy', metrics=['accuracy'])

What is the error of this code?

from sklearn.metrics import confusion_matrix, classification_report

prediksi = model.predict(test_data_generator)
y_pred = np.argmax(prediksi, axis=1)
print(confusion_matrix(test_data_generator.classes, y_pred))
print(classification_report(test_data_generator.classes, y_pred))

I've also tried using labels=np.unique(y_pred), but the results do not show the value of the accuracy of.

Answer 1

You should know:

• When true positive + false positive == 0, precision is undefined.
• When true positive + false negative == 0, recall is undefined.

In such cases, by default the metric will be set to 0, as will f-score, and UndefinedMetricWarning will be raised. This behavior can be modified with zero_division.

• Detail on https://github.com/scikit-learn/scikit-learn/issues/14876

And now you can change the behaviour by zero_division:

zero_division : string or int, default="warn"
    Sets the behavior when there is a zero division. If set to
    ("warn"|0)/1, returns 0/1 when both precision and recall are zero

Answer 2

This warning occurs because y_true contains labels that are not present in your predictions (y_pred) like in the example below:

import numpy as np
from sklearn.metrics import confusion_matrix, classification_report

y_pred = np.ones(10,)
y_true = np.ones(10,)
y_true[0]=0
print(confusion_matrix(y_true,y_pred))
print(classification_report(y_true,y_pred))

You can remove this warning by setting classification_report argumentzero_division=1.

But it is not wise as it shows you that there is a problem with your classifier.