CODEWITHSUNDEEP
pythontensorflow
jsfa11
jsfa11

Reputation: 513

Tensorflow Estimator logits and labels must have the same shape

Just fyi... This is not the bug #4715 that has been reported and fixed.

print('Tensorflow version {} is loaded.'.format(tf.__version__))
#Tensorflow version 1.4.0 is loaded.

I have put together a custom Estimator with only 2 features and a binary classification. The following code works, but does really converge on something useful. 

input_layer = tf.feature_column.input_layer(features, feature_columns)
h1 = tf.layers.Dense(h1_size, activation=tf.nn.relu)(input_layer)
h2 = tf.layers.Dense(h2_size, activation=tf.nn.relu)(h1)
logits = tf.layers.Dense(NUM_CLASSES)(h2)
labels = tf.squeeze(labels, 1)
loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)

To improve the model, I would like to change the way the loss is calculated. Specifically, I would prefer not to use softmax and instead use sigmoid_cross_entropy. Thus, I change the loss to;

loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=labels, logits=logits)

and get these errors;

ValueError: Shapes (?,) and (?, 2) must have the same rank
....
During handling of the above exception, another exception occurred:
....
ValueError: Shapes (?,) and (?, 2) are not compatible
....
During handling of the above exception, another exception occurred:
....
ValueError: logits and labels must have the same shape ((?, 2) vs (?,))

Since I do not really understand why the squeeze() is necessary, I removed it and I get this;

ValueError: Dimensions 1 and 2 are not compatible
....
During handling of the above exception, another exception occurred:
....
ValueError: logits and labels must have the same shape ((?, 2) vs (?, 1))

This made me think that I could pass a one_hot to the loss calculation to solve the shape problem. I tried this, along with the squeeze() just before the one_hot(). Though, the error becomes much more complicated, indicating that I might be headed down the wrong path. (For the record, the one_hot adds a dimension to the matrix, so without the squeeze, the tensor becomes (?, 1, 2)).

loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=tf.one_hot(labels, depth=2), logits=logits)
InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 288 values, but the requested shape has 1

Could someone help me to understand why my labels and logits are fine for the sparse_softmax_cross_entropy. but not for the sigmoid_cross_entropy_with_logits? Is there anything that I can do to reshape the tensors to allow the sigmoid loss function?

Upvotes: 2

Views: 4803

Answers (1)

jsfa11
jsfa11

Reputation: 513

The answer was to create a one_hot first and then to squeeze() the resulting tensor.

This worked for the loss calculation.

loss = tf.losses.sigmoid_cross_entropy(
        multi_class_labels=tf.squeeze(tf.one_hot(labels, depth=2), axis=1),
        logits=logits)

The sigmoid_cross_entropy goes on to make the sigmoid_cross_entropy_with_logits. During the search I ended up switching to what is posted merely by laziness.

Upvotes: 2

Related Questions