Stewart Macdonald
Stewart Macdonald

Reputation: 2132

Simple issue with floats and ints

A simple question best asked with two lines of code:

CGFloat answer = (abs(-27.460757f) - 9.0f) * (800.0f / (46.0f - 9.0f));
    NSLog(@"According to my calculator, answer should be 399.15, but instead it is: %f", answer);

When I run this in Xcode (specifically, in the iPhone simulator), I get:

According to my calculator, answer should be 399.15, but instead it is: 389.189209

Is this just due to my lack of understanding of how floats are rounded?

Thanks!

Stewart

Upvotes: 1

Views: 109

Answers (1)

user557219
user557219

Reputation:

The abs() function operates on integers, so abs(-27.460757f) returns 27. Since you’re using floats, use fabsf() instead.

Upvotes: 5

Related Questions