CODEWITHSUNDEEP
tensorflowkerasdeep-learningconv-neural-networkloss-function
user1953366
user1953366

Reputation: 1611

Combining two loss function in Keras in Sequential model with ndarray output

I am training a CNN model in Keras (object detection in image and LiDAR (Kaggle Lyft Competition)). As an output I have a 34 channel gird. So output dimension is: LENGTH x WIDTH X 34. First 10 channels are for different categories of objects (ideally as one hot vector) and rest of 24 channels are coordinates of bounding box in 3D.

For first 10 channels I want to use: keras.losses.categorical_crossentropy, and for rest of 24: keras.losses.mean_squared_error

Also since numbers of objects differ drastically, to avoid bias, I am weighing each class. Since output is ndarray, I had to write custom loss function (for class weighing). Code is mostly from: Custom loss function for U-net in keras using class weights: `class_weight` not supported for 3+ dimensional targets

def weightedLoss(weightsList):

def lossFunc(true, pred):

    axis = -1 #if channels last 
    #axis=  1 #if channels first


    #argmax returns the index of the element with the greatest value
    #done in the class axis, it returns the class index    
    classSelectors = K.argmax(true, axis=axis) 

    #considering weights are ordered by class, for each class
    #true(1) if the class index is equal to the weight index   
    one64 = np.ones(1, dtype=np.int64)  #Needed to avod int32 and int64 error
    classSelectors = [K.equal(one64[0]*i, classSelectors) for i in range(len(weightsList))]

    #casting boolean to float for calculations  
    #each tensor in the list contains 1 where ground true class is equal to its index 
    #if you sum all these, you will get a tensor full of ones. 
    classSelectors = [K.cast(x, K.floatx()) for x in classSelectors]

    #for each of the selections above, multiply their respective weight
    weights = [sel * w for sel,w in zip(classSelectors, weightsList)] 

    #sums all the selections
    #result is a tensor with the respective weight for each element in predictions
    weightMultiplier = weights[0]
    for i in range(1, len(weights)):
        weightMultiplier = weightMultiplier + weights[i]

    op_chan_loss = keras.losses.categorical_crossentropy
    op_box_loss = keras.losses.mean_squared_error
    #make sure your originalLossFunc only collapses the class axis
    #you need the other axes intact to multiply the weights tensor
    print(type(true), type(pred))
    loss = op_chan_loss(true, pred) 
    loss = loss * weightMultiplier

    return loss
return lossFunc

However I am not sure how to combine two loss functions together in this custom loss function. Please help.

Upvotes: 1

Views: 1210

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86630

Use two outputs. Your model must be a Functional API model:

#basic example of the initial part of your model
inputs = Input(input_shape)
intermediate_output = Conv2D(...)(inputs)
intermediate_output = Conv2D(...)(intermediate_output)

At some point in your model, you will separate two branches. Maybe the last layer or a little earlier (since the nature of the two outputs is different, maybe (needs test, of course) each output needs one or two layers to adapt better. 

output1 = SomeLayer(...)(intermediate_output)
....
output1 = Dense(10, activation='softmax', name='cat_out')(output1) 


output2 = SomeLayer(...)(intermediate_output)
....
output2 = SomeLayer(24, name='bound_out')(output2) #maybe choose an activation

Create a model with two outputs:

model = Model(inputs, [output1, output2])

Now this allows you to have a different loss function for each output, and different metrics:

model.compile(loss = [weightedLoss(weights), 'mse'], 
              metrics = {'cat_out':[metrics1], 'bound_out':[metrics2], 
              optimizer = ...)

Train with two outputs:

model.fit(x_train, [y_train[:,:,:,:10], y_train[:,:,:,-24:]], ...)

I didn't check your loss function, but I saw it has loops (usually a bad thing for tensor operations).

I suggest that your weight list be a tensor:

weightsTensor = K.constant(listOfWeights)

And when you're selecting your weights, try to use tf.gather or tf.gather_nd. I didn't spend time thinking about what is the correct function and how to use it, but once you do it, you simply multiply your results by the weights:

#this may need change and might be a little troublesome to understand
selectedWeights = tf.gather_nd(weightsTensor, classSelectors, ....) 

loss = keras.losses.categorical_crossentropy(true, pred)
loss = selectedWeights * loss

Upvotes: 3

Related Questions