Reputation: 513
Just fyi... This is not the bug #4715 that has been reported and fixed.
print('Tensorflow version {} is loaded.'.format(tf.__version__))
#Tensorflow version 1.4.0 is loaded.
I have put together a custom Estimator with only 2 features and a binary classification. The following code works, but does really converge on something useful.
input_layer = tf.feature_column.input_layer(features, feature_columns)
h1 = tf.layers.Dense(h1_size, activation=tf.nn.relu)(input_layer)
h2 = tf.layers.Dense(h2_size, activation=tf.nn.relu)(h1)
logits = tf.layers.Dense(NUM_CLASSES)(h2)
labels = tf.squeeze(labels, 1)
loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
To improve the model, I would like to change the way the loss is calculated. Specifically, I would prefer not to use softmax and instead use sigmoid_cross_entropy. Thus, I change the loss to;
loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=labels, logits=logits)
and get these errors;
ValueError: Shapes (?,) and (?, 2) must have the same rank
....
During handling of the above exception, another exception occurred:
....
ValueError: Shapes (?,) and (?, 2) are not compatible
....
During handling of the above exception, another exception occurred:
....
ValueError: logits and labels must have the same shape ((?, 2) vs (?,))
Since I do not really understand why the squeeze() is necessary, I removed it and I get this;
ValueError: Dimensions 1 and 2 are not compatible
....
During handling of the above exception, another exception occurred:
....
ValueError: logits and labels must have the same shape ((?, 2) vs (?, 1))
This made me think that I could pass a one_hot to the loss calculation to solve the shape problem. I tried this, along with the squeeze() just before the one_hot(). Though, the error becomes much more complicated, indicating that I might be headed down the wrong path. (For the record, the one_hot adds a dimension to the matrix, so without the squeeze, the tensor becomes (?, 1, 2)).
loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=tf.one_hot(labels, depth=2), logits=logits)
InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 288 values, but the requested shape has 1
Could someone help me to understand why my labels and logits are fine for the sparse_softmax_cross_entropy. but not for the sigmoid_cross_entropy_with_logits? Is there anything that I can do to reshape the tensors to allow the sigmoid loss function?
Upvotes: 2
Views: 4803
Reputation: 513
The answer was to create a one_hot first and then to squeeze() the resulting tensor.
This worked for the loss calculation.
loss = tf.losses.sigmoid_cross_entropy(
multi_class_labels=tf.squeeze(tf.one_hot(labels, depth=2), axis=1),
logits=logits)
The sigmoid_cross_entropy goes on to make the sigmoid_cross_entropy_with_logits. During the search I ended up switching to what is posted merely by laziness.
Upvotes: 2