CODEWITHSUNDEEP
pythontensorflowmachine-learningkerasloss-function
Henryk Borzymowski
Henryk Borzymowski

Reputation: 1078

How can I use a weighted MSE as loss function in a Keras model?

I am trying to use a custom loss function for calculating a weighted MSE in a regression taks (values in the task:-1,-0.5, 0, 0.5 , 1, 1.5, 3 etc.). Here is my implementation of custom loss function:

import tensorflow
import tensorflow.keras.backend as kb

def weighted_mse(y, yhat):
    ind_losses = tensorflow.keras.losses.mean_squared_error(y, yhat)
    weights_ind = kb.map_fn(lambda yi: weight_dict[kb.get_value(yi)], y, dtype='float32')
    # average loss over weighted sum of the batch
    return tensorflow.math.divide(tensorflow.math.reduce_sum(tensorflow.math.multiply(ind_losses, weights_ind)), len(y))

I am running an example which is working:

weight_dict = {-1.0: 70.78125, 0.0: 1.7224334600760458, 0.5: 4.58502024291498, 1.0: 7.524916943521595, 1.5: 32.357142857142854, 2.0: 50.33333333333333, 2.5: 566.25, 3.0: 566.25}
y_true = tensorflow.convert_to_tensor([[0.5],[3]])
y_pred = tensorflow.convert_to_tensor([[0.5],[0]])

weighted_mse(y_true, y_pred)

But when inputted into my model, it throws the following error:

AttributeError: 'Tensor' object has no attribute '_numpy'

Here is how I use the custom loss function:

    model.compile(
    optimizer=opt,
    loss={
        "predicted_class": weighted_mse
    })

EDIT:

when changing weight_dict[kb.get_value(yi)] to weight_dict[float(yi)] I get the following error:

TypeError: float() argument must be a string or a number, not 'builtin_function_or_method'

Upvotes: 1

Views: 2835

Answers (2)

today
today

Reputation: 33470

What you want is basically the idea of sample weight. When using training API of Keras, alongside your data you can pass another array containing the weight for each sample which is used to determine the contribution of each sample in the loss function.

To use it, you can use sample_weight argument of fit method:

model.fit(X, y, sample_weight=X_weight, ...)

Note that X_weight should be an array of the same length as X (i.e. one weight value for each training sample). Further, if X is a tf.data.Dataset instance or a generator, this argument does not work and instead you need to pass the sample weight as the third element of the tuple returned by X.

Upvotes: 1

Sahith Kurapati
Sahith Kurapati

Reputation: 1715

This usually happens in an old version of tensorflow. There are 2 things you can try:

  1. Add this line to the jupyter notebook when you are importing tensorflow like so:
import tensorflow as tf
tf.enable_eager_execution()
  1. Upgrade tensorflow with the following command in prompt:
pip install tensorflow --upgrade

This is most probably because of eager execution. See the docs here for more info.

Upvotes: 0

Related Questions