CODEWITHSUNDEEP
pythontensorflowmachine-learning
ztyh
ztyh

Reputation: 421

tensorflow custom loss that are not in the form of sum of single sample errors

I was working through here. I am currently modifying the loss. Looking at,

    deltas=tf.square(y_est-y)
    loss=tf.reduce_sum(deltas)

I am understanding this to be calculating the squared difference between the output and the true label. Then the loss is the sum of these squares. So, writing the squared single sample error as S_i for sample i, things are simple for the case when the batch loss is just \sum_{i} f(S_i), where summation goes through the all samples. But what if you cannot write the loss in this form? That is, the batch loss for the whole data is f({S_i}) for some general f with i going over all the samples. That is, the loss for the whole data cannot be calculated as some simple linear combination of the losses for the constituent samples. How do you code this in tensorflow? Thank you.

Below is more details on f. The outputs of the neural network are u_i where i goes from 1 to n. n is the number of samples we have. My error is something like

sum_{i from 1 to n} C_i log{sum_{k from 1 to n} I_{ik} exp{-d(u_i,u_k)} }

C_i is number of nodes connected to node i, which I already have and is a constant. I_{ik} is 1 if node i and node k is not connected.

Thanks for the code. maybe my question has not been worded correctly. I am not really looking for the code for the loss. This I can do myself. If you look at,

deltas=tf.square(y_est-y)
loss=tf.reduce_sum(deltas)

deltas, are they (1,3)? A bit above it reads

# Placeholders for input and output data
X = tf.placeholder(shape=(120, 4), dtype=tf.float64, name='X')
y = tf.placeholder(shape=(120, 3), dtype=tf.float64, name='y')

# Variables for two group of weights between the three layers of the network
W1 = tf.Variable(np.random.rand(4, hidden_nodes), dtype=tf.float64)
W2 = tf.Variable(np.random.rand(hidden_nodes, 3), dtype=tf.float64)

# Create the neural net graph
A1 = tf.sigmoid(tf.matmul(X, W1))
y_est = tf.sigmoid(tf.matmul(A1, W2))

I think it is (1,3). They use (1,3) y_est and y. I want to know the specific tensorflow syntax for working with (m,3) y_est and y for any given m.

Upvotes: 0

Views: 397

Answers (1)

Umang Gupta
Umang Gupta

Reputation: 16450

I might be wrong with syntaxes ... but this should give you a general idea. Also, you can optimize it further by vectorization. I have simply put your loss function as it is.
N is the batch size. 

def f(C, I, U, N):
    loss = 0
    for i in range(N):
         sum_ = 0
         for k in range(N):
              sum_ += I[i,k] * tf.exp(d(u[i]-u[k])
         loss += C[i]*tf.log(sum)
    return loss    
loss = f(C,I,U, batch_size)

Upvotes: 2

Related Questions