CODEWITHSUNDEEP
pythontensorflowkerascustomizationmetrics
Zhongzheng_11
Zhongzheng_11

Reputation: 181

How to build a custom accuracy metric with tolerance in TF2?

I want to build a custom accuracy metric with tolerance. Instead of counting elements exactly equal in y_true and y_pred, this accuracy regards the two elements are consistent if their difference within a given tolerance value. For example, if the differences between predicted degrees and true degrees are smaller than 5 degree, we can think the results are correct and calculate the accuracy based on this rule. I want to use this metric in model.compile so it should be a callable function.

I wrote a function as follows.

def accuracy_with_tolerence(y_true,y_pred):
    """
    y_true/y_pred: batch of samples; (BatchSize, 1)
    """
    threshold = 5
    differnece = tf.abs(tf.subtract(y_true,y_pred)) - threshold
    boolean_results = [True if i < 0 else False for i in differnece]
    return K.mean(math_ops.cast(boolean_results, K.floatx()))

It can return the correct accuracy value.

x = tf.constant([1, 2, 3], dtype=tf.float32)
y = tf.constant([5, 8, 10], dtype=tf.float32)
acc = accuracy_with_tolerence(x,y)
print(acc)

tf.Tensor(0.33333334, shape=(), dtype=float32)

But when I want to use it in compile, there is an error:

# Initialize ResNet50
model = resnet50()
model.compile(optimizer='adam',loss='mse',metrics=[accuracy_with_tolerence])
model.load_weights(checkpoint_filepath_0)
model.evaluate(x_test,y_test)

OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

It seems I cannot iterate the Tensor. So how can I get element-wise boolean comparison results in the metric function? How can I realize this accuracy function?

Thank you in advance.

Upvotes: 1

Views: 707

Answers (1)

runDOSrun
runDOSrun

Reputation: 10995

You can't make a list comprehension with a tensor. The operation you're looking for is tf.where and you can use it as follows:

def accuracy_with_tolerence(y_true, y_pred):
    threshold = 5
    differnece = tf.abs(tf.subtract(y_true, y_pred)) - threshold
    boolean_results = tf.where(differnece>0, True, False)
    return K.mean(math_ops.cast(boolean_results, K.floatx()))

Note that you can simplify the code further:

    ...
    boolean_results = tf.where(tf.abs(tf.subtract(y_true, y_pred)) - threshold>0, 1., 0.)
    return K.mean(boolean_results)

Upvotes: 1

Related Questions