CODEWITHSUNDEEP
iosshadermetal
Chris
Chris

Reputation: 2907

Comparison of floats in the Metal Shading Language

Is there an easier way of comparing two floats in the metal shading language?

I'm currently multiplying by 1000 then converting to an int to get an integer comparison.

int r = (int)(color.r * 1000);
int checkColorR = (int)(50.0/255.0 * 1000);
if (r == checkColorR) ...

I'm after something like: Mathf.Approximately in Unity

Upvotes: 1

Views: 796

Answers (1)

Ken Thomases
Ken Thomases

Reputation: 90521

A common technique is to compare the absolute difference to some desired precision:

if (fabs(color.r - 50.0/255.0) < 1.0/1000.0) ...

You can, of course, define a function for this rather than repeating it all over the place.

Your current technique is bad because it does integer truncation. The two values could be very close, but on either side of an integer, and they would compare as not equal because one would be truncated to the integer below. For example, 4.9999 and 5.0001.

Upvotes: 2

Related Questions