CODEWITHSUNDEEP
tensorflow
altier2856
altier2856

Reputation: 33

Give me a code example usage for tensorflow's tf.metrics.sparse_average_precision_at_k

Can you give me a code example that uses tf.metrics.sparse_average_precision_at_k? I cannot find anything on the Internet... :(

If I have a multi-labeled dataset like this one (each example may have more than one target label):

(total number of classes = 5)
y1 = [class_0, class_1]
y2 = [class_1, class_2]
y3 = [class_0]

and my system predicts:

prediction for y1 -> [0.1, 0.3, 0.2, 0.0, 0.0]
prediction for y2 -> [0.0, 0.3, 0.7, 0.4, 0.4]
prediction for y3 -> [0.1, 0.3, 0.2, 0.3, 0.5]

How can I compute for k=3, for example?

P.S.: Feel free to suggest your own example, if you can't comprehend this one.

EDIT: My code so far: I really don't get it... Pls advise for a single prediction (y1 only) as well as for several predictions at once (with different number of true classes in each).

import numpy as np
import tensorflow as tf
sess = tf.InteractiveSession()
tf.local_variables_initializer().run()
y1 = tf.constant( np.array([0, 1])  )
y2 = tf.constant( np.array([1, 2])  )
y3 = tf.constant( np.array([0]) )
p1 = tf.constant( np.array([0.1, 0.3, 0.2, 0.0, 0.0]) )
p2 = tf.constant( np.array([0.0, 0.3, 0.7, 0.4, 0.4]) )
p3 = tf.constant( np.array([0.1, 0.3, 0.2, 0.3, 0.5]) )
metric, update = tf.metrics.sparse_average_precision_at_k(tf.cast(y1, tf.int64), p1, 3)
print(sess.run(update))

Upvotes: 1

Views: 2641

Answers (2)

Hong Lan
Hong Lan

Reputation: 41

The tf.metrics.sparse_average_precision_at_k will be replaced by tf.metrics.average_precision_at_k. And by browsing the code in tensorflow, you will find that when your inputs are y_true and y_pred, this function will actually transform the y_pred to y_pred_idx, by using top_k function.

y_true is a tensor of shape (batch_size, num_labels), and y_pred is of shape (batch_size, num_classes)

You can also see some discussion in this issue, and this example comes from this issue.

import tensorflow as tf
import numpy as np

y_true = np.array([[2], [1], [0], [3], [0]]).astype(np.int64)
y_true = tf.identity(y_true)

y_pred = np.array([[0.1, 0.2, 0.6, 0.1],
                   [0.8, 0.05, 0.1, 0.05],
                   [0.3, 0.4, 0.1, 0.2],
                   [0.6, 0.25, 0.1, 0.05],
                   [0.1, 0.2, 0.6, 0.1]
                   ]).astype(np.float32)
y_pred = tf.identity(y_pred)

_, m_ap = tf.metrics.sparse_average_precision_at_k(y_true, y_pred, 3)

sess = tf.Session()
sess.run(tf.local_variables_initializer())

stream_vars = [i for i in tf.local_variables()]

tf_map = sess.run(m_ap)
print(tf_map)

print((sess.run(stream_vars)))

tmp_rank = tf.nn.top_k(y_pred,3)

print(sess.run(tmp_rank))

This line stream_vars = [i for i in tf.local_variables()] helps you see the two local_variables which is created in this tf.metrics.sparse_average_precision_at_k function.

This line tmp_rank = tf.nn.top_k(y_pred,3) in order to helps you understand by changing the value of k ,the prediction index which is used in tf.metrics.sparse_average_precision_at_k .

You can change the value of k to see the different result, and the tmp_rank represents the index which is used in calculating the average precision.

For example: when k=1, only the first batch match the label, so the average precision at 1 result will be 1/6 = 0.16666666. When k=2, the third batch will also match the label, so the average precision at 2 result will be (1+(1/2))/6=0.25.

Upvotes: 1

Alexandre Passos
Alexandre Passos

Reputation: 5206

metric, update = tf.metrics.sparse_average_precision_at_k(tf.stack(y1, y2, y3), tf.stack(p1, p2, p3), 3)
print session.run(update)

Upvotes: 0

Related Questions