CODEWITHSUNDEEP
pythontensorflowmachine-learningregression
Aneesh Bhat
Aneesh Bhat

Reputation: 15

Custom accuracy metric over a range, for Regression

I need to write an accuracy metric (during training) that calculates accuracy % over a range. So it would show my custom accuracy every epoch during training.

For eg. If y_pred is 343 and y_target is 340, and the range is 10, it should be counted as correct. Meaning, any predicted value in the range [340-5, 340+5] would be counted as correct for that instance.

But the existing accuracy metric would consider the above to be wrong, and accuracy is very low for regression problems.

Upvotes: 0

Views: 462

Answers (2)

gboffi
gboffi

Reputation: 25053

If you use Numpy

import numpy as np

...

good = np.abs(pred-target) <= (range/2)

would be a vector of booleans (False and True) telling you if the prediction is good enough.

Booleans in Python can be used as integers (with values respectively equal to 0 and 1), and this implies that you can sum the good array to know how many predictions were on target

n_good = np.sum(good)

Small Benchmark

In [1]: import numpy as np 
   ...:  
   ...: target = np.ones(2**18, dtype=int)*10 
   ...: predicted = np.random.randint(0, 21, 2**18)                                       

In [2]: %timeit [1 for i,j in zip(predicted, target) if i in range(j-5,j+6)] 
   ...: %timeit sum(1 for i,j in zip(predicted, target) if i in range(j-5,j+6)) 
   ...: %timeit np.abs(predicted-target)<=5 
   ...: %timeit np.sum(np.abs(predicted-target)<=5)                                       
983 ms ± 6.22 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
964 ms ± 11.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
1.63 ms ± 44.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
2.18 ms ± 10.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [3]:

Upvotes: 1

Sociopath
Sociopath

Reputation: 13426

You need:

y_pred = [125, 332, 268, 349]
y_target = [129, 342, 265, 370]

def accuracy(y_pred, y_target):
    a = [1 for i,j in zip(y_pred, y_target) if i in range(j-5,j+6)]

    return sum(a)/len(y_pred)

print(accuracy(y_pred, y_target))

Output:

0.5

Upvotes: 1

Related Questions