CODEWITHSUNDEEP
pythonkerastheano
Moshe Wainner
Moshe Wainner

Reputation: 21

Custom loss function implementation

I'm trying to implement a new loss function of my own. When I tried to debug it (or print in it) I've noticed it is called only once at the model creating section of the code.

How can I know what y_pred and y_true contains (shapes, data etc..) if I cannot run my code into this function while fitting the model?

I wrote this loss function:

def my_loss(y_true, y_pred):
    # run over the sequence, jump by 3
    # calc the label
    # if the label incorrect punish

    y_pred = K.reshape(y_pred, (1, 88, 3))

    y_pred = K.argmax(y_pred, axis=1)

    zero_count = K.sum(K.clip(y_pred, 0, 0))
    one_count = K.sum(K.clip(y_pred, 1, 1))
    two_count = K.sum(K.clip(y_pred, 2, 2))

    zero_punish = 1 - zero_count / K.count_params(y_true)
    one_punish = 1- one_count/ K.count_params(y_true)
    two_punish = 1- two_count/ K.count_params(y_true)

    false_arr = K.not_equal(y_true, y_pred)

    mask0 = K.equal(y_true, K.zeros_like(y_pred))
    mask0_miss = K.dot(false_arr, mask0) * zero_punish

    mask1 = K.equal(y_true, K.ones_like(y_pred))
    mask1_miss = K.dot(false_arr, mask1) * one_punish

    mask2 = K.equal(y_true, K.zeros_like(y_pred)+2)
    mask2_miss = K.dot(false_arr, mask2) * two_punish

    return K.sum(mask0_miss) + K.sum(mask1_miss) + K.sum(mask2_miss)

It fails on:

theano.gof.fg.MissingInputError: A variable that is an input to the graph was 
neither provided as an input to the function nor given a value. A chain of 
variables leading from this input to an output is [/dense_1_target, Shape.0]. 
This chain may not be unique
Backtrace when the variable is created:

How can I fix it?

Upvotes: 1

Views: 1663

Answers (1)

Autonomous
Autonomous

Reputation: 9075

You have to understand that Theano is a symbolic language. For example, when we define the following loss function in Keras:

def myLossFn(y_true, y_pred):
    return K.mean(K.abs(y_pred - y_true), axis=-1)

Theano is just making a symbolic rule in a computational graph, which would be executed when it gets values i.e. when you train the model with some mini-batches.

As far as your question on how to debug your model goes, you can use theano.function for that. Now, you want to know if your loss calculation is correct. You do the following.

You can implement the python/numpy version of your loss function. Pass two random vectors to your numpy-loss-function and get a number. To verify if theano gives nearly identical result, define something as follows.

import theano
from theano import tensor as T
from keras import backend as K

Y_true = T.frow('Y_true')
Y_pred = T.fcol('Y_pred')
out = K.mean(K.abs(Y_pred - Y_true), axis=-1)

f = theano.function([Y_true, Y_pred], out)

# creating some values
y_true = np.random.random((10,))
y_pred = np.random.random((10,))

numpy_loss_result = np.mean(np.abs(y_true-y_pred))
theano_loss_result = f(y_true, y_pred)

# check if both are close enough
print numpy_loss_result-theano_loss_result # should be less than 1e-5

Basically, theano.function is a way to put values and evaluate those symbolic expressions. I hope this helps.

Upvotes: 2

Related Questions