FPS camera implementation error (OpenGL/GLUT/C++)

Question

I try to implement a camera class. One of its modes should be the basic FPS camera behaviour, when you can change direction with the mouse, strafe left-right and move forward-backward compared to the forward/upward directions. I have the following code, which seems to work fine in the beginning, but then the direction vector get corrupted:

My idea is that I do not store angles for the current direction, but rotate the forward vector (I already have) by 1-2 degrees per pixel on glutPassiveMotionFunc():

void Camera::Control(int x, int y)
{
    switch (mode) {
        case AVATAR:
            if (x < prevX) direction.Rotate(1. * (prevX - x), up.Inverted());
            if (x > prevX) direction.Rotate(1. * (x - prevX), up);
            if (y < prevY) direction.Rotate(1. * (prevY - y), direction % up);
            if (y > prevY) direction.Rotate(1. * (y - prevY), up % direction);
            direction.Normalize();
            //up = (direction % up) % direction; //UP in vitual space or for the camera?
            prevX = x;
            prevY = y;
            //Debug vector values:
            //printf("\nwinX: %d\twinY: %d\n", x, y);
            //printf("radX: %0.3f\tradY: %0.3f\n", x, y);
            //printf("dirX: %0.3f\ndirY: %0.3f\ndirZ: %0.3f\n", direction.X, direction.Y, direction.Z);
            //printf("posX: %0.3f\nposY: %0.3f\nposZ: %0.3f\n", position.X, position.Y, position.Z);
            break;
        case ORBIT: /* ... */ break;
        default: break;
    }
}

Strafe works well:

void Camera::Control(int key)
{
    switch (key) {
        case 'w': position += direction.Normal(); break;
        case 's': position -= direction.Normal(); break;
        case 'a': position += (up % direction).Normal(); break;
        case 'd': position += (direction % up).Normal(); break;
        default: break;
    }
}

How the camera refreshes its view:

void Camera::Draw()
{
    gluLookAt(position.X, position.Y, position.Z, position.X + direction.X, position.Y + direction.Y, position.Z + direction.Z, up.X, up.Y, up.Z);
}

The rotation of vectors:

void Vector::Rotate(float angle, float x, float y, float z)
{
    float rad = angle * PI / 180.;
    float s = sin(rad), c = cos(rad);
    float matrix[9] = {
        x*x*(1-c)+c,    y*x*(1-c)-s*z,  z*x*(1-c)+s*y,
        x*y*(1-c)+s*z,  y*y*(1-c)+c,    z*y*(1-c)+s*x,
        x*z*(1-c)-s*y,  y*z*(1-c)+s*x,  z*z*(1-c)+c
    };
    *this = Vector(X * matrix[0] + Y * matrix[3] + Z * matrix[6], X * matrix[1] + Y * matrix[4] + Z * matrix[7], X * matrix[2] + Y * matrix[5] + Z * matrix[8]);
}

void Vector::Rotate(float angle, Vector& axis)
{
    Rotate(angle, axis.X, axis.Y, axis.Z);
}

When I tested the Rotate function and the whole concept rotating only aound the X axis, it worked fine. I can look around the UP(0, 1, 0) vector (Y axis). Rotating only around (direction % up) and (up % direction) also works. (Vector::operator%(Vector& other) defines cross product.) The problem occurs when I calculate rotations for both X and Y position. Any ideas?

Answer 1

Compute direction % up and normalize the result before you apply the Y rotation.