CODEWITHSUNDEEP
keras
Ryan
Ryan

Reputation: 10179

How to pass weights to mean squared error in keras

I am trying to approach a regression problem, which is multi label with 8 labels for which i am using mean squared error loss, but the data set is imbalanced and i want to pass weights to the loss function.Currently i am compiling the model this way.

model.compile(loss='mse', optimizer=Adam(lr=0.0001), metrics=['mse', 'acc'])

Could someone please suggest if it is possible to add weights to mean squared error,if so, how could i do it?

Thanks in advance

The labels look like so

enter image description here

# 
model = Sequential()    
model.add(effnet)
model.add(GlobalAveragePooling2D())
model.add(Dropout(0.5))
model.add(Dense(8,name = 'nelu', activation=elu))
model.compile(loss=custom_mse(class_weights), 
optimizer=Adam(lr=0.0001), metrics=['mse', 'acc'])

Upvotes: 3

Views: 2440

Answers (2)

Mohamad Zeina
Mohamad Zeina

Reputation: 443

First create a dictionary of how much you want to weight each class, for example:

class_weights = {0: 1,
                 1: 1,
                 2: 1, 
                 3: 9, 
                 4: 1...} # Do this for all eight classes

Then pass them into model.fit:

model.fit(X, y, class_weight=class_weights)

Upvotes: 2

Rishab P
Rishab P

Reputation: 1633

import keras
from keras.models import Sequential
from keras.layers import Conv2D, Flatten, Dense, Conv1D, LSTM, TimeDistributed
import keras.backend as K


# custom loss function
def custom_mse(class_weights):
    def loss_fixed(y_true, y_pred):
        """
        :param y_true: A tensor of the same shape as `y_pred`
        :param y_pred:  A tensor resulting from a sigmoid
        :return: Output tensor.
        """
        # print('y_pred:', K.int_shape(y_pred))
        # print('y_true:', K.int_shape(y_true))
        y_pred = K.reshape(y_pred, (8, 1))
        y_pred = K.dot(class_weights, y_pred)
        # calculating mean squared error
        mse = K.mean(K.square(y_pred - y_true), axis=-1)
        # print('mse:', K.int_shape(mse))
        return mse

model = Sequential()
model.add(Conv1D(8, (1), input_shape=(28, 28)))
model.add(Flatten())
model.add(Dense(8))

# custom class weights
class_weights = K.variable([[0.25, 1., 2., 3., 2., 0.6, 0.5, 0.15]])
# print('class_weights:', K.int_shape(class_weights))

model.compile(optimizer='adam', loss=custom_mse(class_weights), metrics=['accuracy'])

Here is a small implementation of a custom loss function based on your problem statement

  • You find more information about keras loss function from losses.py and also check out its official documentation from here

  • Keras does not handle low-level operations such as tensor products, convolutions and so on itself. Instead, it relies on a specialized, well-optimized tensor manipulation library to do so, serving as the "backend engine" of Keras. More information about keras backend can be found here and also check out its official documentation from here

  • Use K.int_shape(tensor_name) to find the dimensions of a tensor.

Upvotes: 5

Related Questions