CODEWITHSUNDEEP
pythontensorflowkerascross-entropy
Anakin Skywalker
Anakin Skywalker

Reputation: 2520

Selecting validation metric for `categorical_crossentropy` in Keras

I am looking at these two questions and documentation:

Whats the output for Keras categorical_accuracy metrics?

Categorical crossentropy need to use categorical_accuracy or accuracy as the metrics in keras?

https://keras.io/api/metrics/probabilistic_metrics/#categoricalcrossentropy-class

For classification of X-Rays images I (15 classes) I do:

# Compile a model
model1.compile(optimizer = 'adam', loss = 'categorical_crossentropy', 
metrics = ['accuracy']) 

# Fit the model
history1 = model1.fit_generator(train_generator, epochs = 10, 
steps_per_epoch = 10, verbose = 1, validation_data = valid_generator)

My model works and I have an output:

But I am not sure how to add validation accuracy here to compare results and avoid over/underfitting.

Upvotes: 1

Views: 2664

Answers (1)

Leo
Leo

Reputation: 111

I hope the following can help you:

The use of "categorical_crossentropy" tells me that your labels are a one hot encoding over different classes.

Let's say you have 15 classes, the correct prediction would be a vector with 14 zeros, and a one at the corresponding index. In this context "accuracy" will be very high as your model will be correctly predicting mostly zero everywhere, so the accuracy should easily be at least 13/15 = 0.86.

A more suitable metric would be "categorical_accuracy" which will give you 1 if the model predicts the correct index, and else 0.

If you have a validation "categorical_accuracy" better than 1/15 = 0.067 (assuming your class are correctly balanced), your model is better than random.

You can find a list of metrics at keras metrics.

Upvotes: 3

Related Questions