CODEWITHSUNDEEP
pythontensorflow
Sai Kumar
Sai Kumar

Reputation: 715

How do I compute one hot encoding using tf.one_hot?

I'm trying to build a one hot encoding of y_train of mnist data-set using tensorflow. I couldn't understand how to do it?

# unique values 0 - 9
y_train = array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)

In keras we'll do something like

# this converts it into one hot encoding
one hot_encoding = tf.keras.utils.to_categorical(y_train)

Where as in tf.one_hot what should be my input to indices & depth parameters? After doing one hot encoding how can I convert it back to numpy array from 2d-tensor?

Upvotes: 6

Views: 9556

Answers (2)

kelkka
kelkka

Reputation: 1004

I'd like to counter what @Andrew Fan has said. First, the above y label list does not start at index 0, which is what is required. Just look at the first column (i.e. index 0) in all of those examples: they're all empty. This will create a redundant class in the learning and may cause problems. One hot creates a simple list with 1 for that index position and zeros elsewhere. Therefore, your depth has to be the same as the number of classes, but you also have to start at index 0.

Upvotes: 2

Andrew Fan
Andrew Fan

Reputation: 1322

I'm not familiar with Tensorflow but after some tests, this is what I've found:

tf.one_hot() takes an indices and a depth. The indices are the values to actually convert to a one-hot encoding. depth refers to the maximum value to utilize.

For example, take the following code:

y = [1, 2, 3, 2, 1]
tf.keras.utils.to_categorical(y)
sess = tf.Session();
with sess.as_default():
    print(tf.one_hot(y, 2).eval())
    print(tf.one_hot(y, 4).eval())
    print(tf.one_hot(y, 6).eval())

tf.keras.utils.to_categorical(y) Returns the following:

array([[0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 0., 1.],
       [0., 0., 1., 0.],
       [0., 1., 0., 0.]], dtype=float32)

In contrast, the tf.one_hot() options (2, 4, and 6) do the following:

[[0. 1.]
 [0. 0.]
 [0. 0.]
 [0. 0.]
 [0. 1.]]
[[0. 1. 0. 0.]
 [0. 0. 1. 0.]
 [0. 0. 0. 1.]
 [0. 0. 1. 0.]
 [0. 1. 0. 0.]]
[[0. 1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0.]
 [0. 0. 1. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0.]]

As can be seen here, to mimic tf.keras.utils.to_categorical() using tf.one_hot(), the depth parameter should be equivalent to the maximum value present in the array, +1 for 0. In this case, the maximum value is 3, so there are four possible values in the encoding - 0, 1, 2, and 3. As such, a depth of 4 is required to represent all of these values in the one-hot encoding.

As for conversion to numpy, as shown above, using a Tensorflow session, running eval() on a tensor converts it to a numpy array. For methods on doing this, refer to How can I convert a tensor into a numpy array in TensorFlow?.

I'm not familiar with Tensorflow but I hope this helps.

Note: for the purposes of MNIST, a depth of 10 should be sufficient.

Upvotes: 7

Related Questions