CODEWITHSUNDEEP
pythonkeras
ScalaBoy
ScalaBoy

Reputation: 3392

How to get the history callback metrics in Keras?

How can I retrieve the history of callback metrics? I have a class Metrics and I use it in the fit function of Keras model as follows callbacks=[model_metrics].

This is the complete code of the class Metrics and fit function.

class Metrics(Callback):

    def on_train_begin(self, logs={}):
        self.val_f1s = []
        self.val_bal_accs = []

    def on_epoch_end(self, epoch, logs={}):
        val_predict = np.argmax((np.asarray(self.model.predict(self.validation_data[0]))).round(), axis=1)
        val_targ = np.argmax(self.validation_data[1], axis=1)
        _val_f1 = metrics.f1_score(val_targ, val_predict, average='weighted')
        _val_bal_acc = metrics.balanced_accuracy_score(val_targ, val_predict)    
        self.val_f1s.append(_val_f1)
        self.val_bal_accs.append(_val_bal_acc)
        print(" — val_f1: {:f} — val_bal_acc: {:f}".format(_val_f1, _val_bal_acc))
        return

model_metrics = Metrics()

history = model.fit(np.array(X_train), y_train, 
                    validation_data=(np.array(X_test), y_test),
                    epochs=5,
                    batch_size=2,
                    callbacks=[model_metrics],
                    shuffle=False,
                    verbose=1)

How can I get the history of val_f1 and val_bal_acc? Now I can only access loss, val_loss, acc, val_acc:

print(history.history.keys())

Upvotes: 3

Views: 3064

Answers (1)

Primusa
Primusa

Reputation: 13498

To interact with keras history API you need to pass in arguments for metrics and not callbacks.

In it's current state your val_f1 and val_bal_acc aren't going to be stored in the history object but will rather be stored in your model_metrics object.

You can access them like so:

model_metrics.val_f1s

It's the same as accessing an attribute for any object.

Finally if you do want to create a custom metric and want access it from history you need to define a custom metric (as a function) and then pass it into the metrics kwarg in model.compile. This is done like so:

def my_metric(y_true y_pred):
    return y_true # just a dummy return value

# assume that the model is defined somewhere
model.compile(loss=..., optimizer=..., metrics = [my_metric]

And then you will be able to find val_my_metric in the history object that you get out of the fit.

Upvotes: 4

Related Questions