CODEWITHSUNDEEP
pythonpython-3.xtensorflowneural-network
Arsenal591
Arsenal591

Reputation: 1626

How to minimize this loss function in tensorflow?

Let's say the output of one neural network is an m × n matrix(or tensor) X. What I want to minimize is the "difference" between each pair of column vectors. That is, if we write the matrix X as [x(1), x(2), ... , x(n)], then the loss function will be like: [x(1) - x(2)]^2 + [x(1) - x(3)]^2 + ... + [x(1) - x(n)]^2 + [x(2) - x(3)] + ... + [x(2) - x(n)]^2 + ... + [x(n-1) - x(n)]^2.

My first thought is like this, which does not work:

def get_loss(tensor):
    res = tf.Variable(0.0)
    for i in range(n - 1):
        for j in range(i + 1, n):
            res = res + tf.reduce_sum(tf.square(tensor[:,i] - tensor[:, j]))
    return res

I guess the function get_loss(tensor)does not really bulid the connection between tensor and res, because after I run this, I got a very strange output:

a = tf.placeholder(tf.float32, [3, 2])
b = tf.Variable([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
loss = get_loss(a + b)
train = tf.train.GradientDescentOptimizer(0.01).minimize(loss)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

for i in range(200):
    sess.run(train,feed_dict = {a : [[1.0, 2.0],[3.0, 4.0], [5.0, 6.0]]})

print('loss = ', sess.run(loss ,feed_dict = {a : [[1.0, 2.0],[3.0, 4.0], [5.0, 6.0]]}))

The output is -12.0, which is meaningless because the value of the loss function cannot be negative.

So, what I want to figure out is: 1) Why the output is negative? 2) How do I implement this loss function correctly in Tensorflow?

Upvotes: 0

Views: 2436

Answers (1)

ma3oun
ma3oun

Reputation: 3790

You don't want the 'res' variable to be part of your optimization. Try using the following function instead:

def get_loss(tensor):
    n=tensor.shape[-1]
    res = tf.constant(0.0,shape=tensor.shape)
    for i in range(n - 1):
        for j in range(i + 1, n):
             res = res + tf.reduce_sum(tf.square(tensor[:,i] - tensor[:,j]))
    return res

Upvotes: 1

Related Questions