legends2k
legends2k

Reputation: 32884

Different results for the same float calculations

In the ray tracing assignment I'm doing, I've to calculate the X-offset of the ray shooting from the camera; the offset calculation goes like this

FovY is given as an input; I've remebered to convert it to radians the moment I read the variable.

OffsetX = tan (FovX / 2) * ((col - (width / 2)) / (width / 2))

FovX = tan(FovY / 2) * aspect = tan(FovY / 2) * (width / height)

Substituting in the original equation and writing the code:

float OffsetX = tan(FovY / 2.0f) * (width / height) * ((col - (width / 2.0f)) / (width / 2.0f));

gave me an incorrect stretched image, and it took me hours to get it right, which was after finding out that the same equation works after simplifying it.

Final rearranged equation was:

float OffsetX = tan(FovY / 2.0f) * (2.0f / height) * (col - (width / 2.0f));

I tried debugging and indeed the results were different for both equations.

Would there be any round off error of some sort? Can someone please explain this quirkiness to me?

#include <cmath>
#include <iostream>
#include <cstdint>

using namespace std;

int main()
{
    const float PI = 3.1415f;
    const uint32_t width = 160, height = 120;
    const auto fovy = (30.0f * (PI / 180.0f));
    size_t j = 0;
    auto alpha = (tan(fovy / 2.0f) * (width / height)) * (((j + 0.5f) - (width / 2.0f)) / (width / 2.0f));
    cout << alpha << endl;
    alpha = tan(fovy / 2.0f) * (2.0f / height) * ((j + 0.5f) - (width / 2.0f));
    cout << alpha << endl;
}

Upvotes: 1

Views: 248

Answers (1)

Stephen Canon
Stephen Canon

Reputation: 106117

Let me guess: width and height are integers.

When you do:

(width / height)

you're getting integer division, which discards the fractional part of the result. If you instead do:

((double)width / height)

then the two results will bet nearly identical.


as an aside, you can simplify the expression further:

tan(FovY / 2.0f) * (2.0f*col - width) / height

Upvotes: 6

Related Questions