CODEWITHSUNDEEP
pythontheanokeras
Nejla
Nejla

Reputation: 133

customised loss function in keras using theano function

I'd like to use my own binary_crossentropy instead of using the one that comes with Keras library. Here is my custom function:

    import theano
    from keras import backend as K

    def elementwise_multiply(a, b): # a and b are tensors
       c = a * b
       return theano.function([a, b], c)

    def custom_objective(y_true, y_pred):  
       first_log = K.log(y_pred)
       first_log = elementwise_multiply(first_log, y_true)
       second_log = K.log(1 - y_pred)
       second_log = elementwise_multiply(second_log, (1 - y_true))
       result = second_log + first_log
       return K.mean(result, axis=-1)

note: This is for practice. I'm aware of T.nnet.binary_crossentropy(y_pred, y_true)

But, when I compile the model:

sgd = SGD(lr=0.001)
model.compile(loss = custom_objective, optimizer = sgd)

I get this error:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () 36 37 sgd = SGD(lr=0.001) ---> 38 model.compile(loss = custom_objective, optimizer = sgd) 39 # ==============================================

C:\Program Files (x86)\Anaconda3\lib\site-packages\keras\models.py in compile(self, optimizer, loss, class_mode) 418 else: 419 mask = None --> 420 train_loss = weighted_loss(self.y, self.y_train, self.weights, mask) 421 test_loss = weighted_loss(self.y, self.y_test, self.weights, mask) 422

C:\Program Files (x86)\Anaconda3\lib\site-packages\keras\models.py in weighted(y_true, y_pred, weights, mask) 80 ''' 81 # score_array has ndim >= 2 ---> 82 score_array = fn(y_true, y_pred) 83 if mask is not None: 84 # mask should have the same shape as score_array

in custom_objective(y_true, y_pred) 11 second_log = K.log(1 - K.clip(y_true, K.epsilon(), np.inf)) 12 second_log = elementwise_multiply(second_log, (1-y_true)) ---> 13 result = second_log + first_log 14 #result = np.multiply(result, y_pred) 15 return K.mean(result, axis=-1)

TypeError: unsupported operand type(s) for +: 'Function' and 'Function'

when I replace elementwise_multiply with inline function:

def custom_objective(y_true, y_pred):  
    first_log = K.log(y_pred)    
    first_log = first_log * y_true
    second_log = K.log(1 - y_pred)
    second_log = second_log * (1-y_true)
    result = second_log + first_log
    return K.mean(result, axis=-1)

the model compiles but the loss value is nan:

Epoch 1/1 945/945 [==============================] - 62s - loss: nan - acc: 0.0011 - val_loss: nan - val_acc: 0.0000e+00

Could someone help me with this please?!

Thanks

Upvotes: 3

Views: 1273

Answers (1)

Nejla
Nejla

Reputation: 133

I found the problem. I had to multiply the return value by "-1" as I'm using stochastic gradient decedent (sgd) as optimiser and not stochastic gradient ascent!

Here is the code and it works like a charm:

import theano
from keras import backend as K

def custom_objective(y_true, y_pred):  
    first_log = K.log(y_pred)    
    first_log = first_log * y_true
    second_log = K.log(1 - y_pred)
    second_log = second_log * (1 - y_true)
    result = second_log + first_log
    return (-1 * K.mean(result))

Upvotes: 4

Related Questions