Reputation: 11933
I'm having issues in my code due to the multiple representations that a same float number can have. For instance, these number are considered to be the same:
I don't care much for a big precision, I need to calculate CRC of those numbers and they should be the same, so my approach was to use this code:
private static float Rounding(float v, float p)
{
return Mathf.Round (v * p) / p;
}
Where p is my precision.
That seems to be working well, but in that case, if I used p = 1e7, the first number multiplication would result in 299999.4 and rounded to 299999, becoming 0.0299999, while 0.03 would remain 0.03. That's my problem.
Am I doing the right thing? Maybe the answer is that I can't use a p this big, but I don't know how floats work, so I don't know the size of p to work in all situations.
However, I know that my float values resides in a range that is not as big as the float range (±1.5e−45 to ±3.4e38). What I mean is, if I know that my floats range is [-100; +100], I can assume that it will have enough decimal digits for p to be 3. Is that right?
Is there another way to do this?
Upvotes: 2
Views: 321
Reputation: 3902
I don't know if I'm getting you right, but if you need those two numbers to be considered as equal, you could use:
Math.Round(Convert.ToDecimal((float)0.0299999400),2); //result is 0.03
Math.Round(Convert.ToDecimal((float)0.0300000000),2); //result is 0.03
And there you have it. If you want to round your numbers to a precision of 1e7 then off course those two numbers will be different.
Upvotes: 0