Quaternion-based First Person View Camera

Question

I have been learning OpenGL by following the tutorial, located at https://paroj.github.io/gltut/.

Passing the basics, I got a bit stuck at understanding quaternions and their relation to spatial orientation and transformations, especially from world- to camera-space and vice versa. In the chapter Camera-Relative Orientation, the author makes a camera, which rotates a model in world space relative to the camera orientation. Quoting:

We want to apply an orientation offset (R), which takes points in camera-space. If we wanted to apply this to the camera matrix, it would simply be multiplied by the camera matrix: R * C * O * p. That's nice and all, but we want to apply a transform to O, not to C.

My uneducated guess would be that if we applied the offset to camera space, we would get the first-person camera. Is this correct? Instead, the offset is applied to the model in world space, making the spaceship spin relative to that space, and not to camera space. We just observe it spin from camera space.

Inspired by at least some understanding of quaternions (or so I thought), I tried to implement the first person camera. It has two properties:

struct Camera{ 
  glm::vec3 position; // Position in world space.
  glm::quat orientation; // Orientation in world space.
}

Position is modified in reaction to keyboard actions, while the orientation changes due to mouse movement on screen.

Note: GLM overloads * operator for glm::quat * glm::vec3 with the relation for rotating a vector by a quaternion (more compact form of v' = qvq^-1)

For example, moving forward and moving right:

glm::vec3 worldOffset;
float scaleFactor = 0.5f;
if (glfwGetKey(window, GLFW_KEY_W) == GLFW_PRESS) {
    worldOffset = orientation * (axis_vectors[AxisVector::AXIS_Z_NEG]); // AXIS_Z_NEG = glm::vec3(0, 0, -1)
    position += worldOffset * scaleFactor;
}
if (glfwGetKey(window, GLFW_KEY_D) == GLFW_PRESS) {
    worldOffset = orientation  * (axis_vectors[AxisVector::AXIS_X_NEG]); // AXIS_Z_NEG = glm::vec3(-1, 0, 0)
    position += worldOffset * scaleFactor;
}

Orientation and position information is passed to glm::lookAt matrix for constructing the world-to-camera transformation, like so:

auto camPosition = position;
auto camForward = orientation * glm::vec3(0.0, 0.0, -1.0);
viewMatrix = glm::lookAt(camPosition, camPosition + camForward, glm::vec3(0.0, 1.0, 0.0));

Combining model, view and projection matrices and passing the result to vertex shader displays everything okay - the way one would expect to see things from the first-person POV. However, things get messy when I add mouse movements, tracking the amount of movement in x and y directions. I want to rotate around the world y-axis and local x-axis:

auto xOffset = glm::angleAxis(xAmount, axis_vectors[AxisVector::AXIS_Y_POS]); // mouse movement in x-direction
auto yOffset = glm::angleAxis(yAmount, axis_vectors[AxisVector::AXIS_X_POS]); // mouse movement in y-direction
orientation = orientation * xOffset; // Works OK, can look left/right
orientation = yOffset * orientation; // When adding this line, things get ugly

What would the problem be here? I admit, I don't have enough knowledge to debug the mouse movement code properly, I mainly followed the lines, saying "right multiply to apply the offset in world space, left multiply to do it in camera space."

I feel like I know things half-way, drawing conclusions from a plethora of e-resources on the subject, while getting more educated and more confused at the same time. Thanks for any answers.

Answer 1

The problem lied with the usage of glm::lookAt for constructing the view matrix. Instead, I am now constructing the view matrix like so:

auto rotate = glm::mat4_cast(entity->orientation);
auto translate = glm::mat4(1.0f);
translate = glm::translate(translate, -entity->position);
viewMatrix = rotate * translate;

For translation, I'm left multiplying with an inverse of orientation instead of orientation now.

glm::quat invOrient = glm::conjugate(orientation);
if (glfwGetKey(window, GLFW_KEY_W) == GLFW_PRESS) {
  worldOffset = invOrient * (axis_vectors[AxisVector::AXIS_Z_NEG]);
  position += worldOffset * scaleFactor;
}
...

Everything else is the same, apart from some further offset quaternion normalizations in the mouse movement code.

The camera now behaves and feels like a first-person camera.

I still don't properly understand the difference between view matrix and lookAt matrix, if there is any. But that's the topic for another question.

Answer 2

To rotate a glm quaternion representing orientation:

//Precomputation: 
//pitch (rot around x in radians), 
//yaw (rot around y in radians), 
//roll (rot around z in radians)
//are computed/incremented by mouse/keyboard events

To compute view matrix:

void CameraFPSQuaternion::UpdateView()
{
  //FPS camera:  RotationX(pitch) * RotationY(yaw)
  glm::quat qPitch = glm::angleAxis(pitch, glm::vec3(1, 0, 0));
  glm::quat qYaw = glm::angleAxis(yaw, glm::vec3(0, 1, 0));
  glm::quat qRoll = glm::angleAxis(roll,glm::vec3(0,0,1));  

  //For a FPS camera we can omit roll
  glm::quat orientation = qPitch * qYaw;
  orientation = glm::normalize(orientation);
  glm::mat4 rotate = glm::mat4_cast(orientation);

  glm::mat4 translate = glm::mat4(1.0f);
  translate = glm::translate(translate, -eye);

  viewMatrix = rotate * translate;
}

If you want to store the quaternion, then you recompute it whenever yaw, pitch, or roll changes:

void CameraFPSQuaternion::RotatePitch(float rads) // rotate around cams local X axis
{
  glm::quat qPitch = glm::angleAxis(rads, glm::vec3(1, 0, 0));

  m_orientation = glm::normalize(qPitch) * m_orientation;
  glm::mat4 rotate = glm::mat4_cast(m_orientation);

  glm::mat4 translate = glm::mat4(1.0f);
  translate = glm::translate(translate, -eye);

  m_viewMatrix = rotate * translate;
}

If you want to give a rotation speed around a given axis, you use slerp:

void CameraFPSQuaternion::Update(float deltaTimeSeconds)
{
  //FPS camera:  RotationX(pitch) * RotationY(yaw)
  glm::quat qPitch = glm::angleAxis(m_d_pitch, glm::vec3(1, 0, 0));
  glm::quat qYaw = glm::angleAxis(m_d_yaw, glm::vec3(0, 1, 0));
  glm::quat qRoll = glm::angleAxis(m_d_roll,glm::vec3(0,0,1));  

  //For a FPS camera we can omit roll
  glm::quat m_d_orientation = qPitch * qYaw;
  glm::quat delta = glm::mix(glm::quat(0,0,0,0),m_d_orientation,deltaTimeSeconds);
  m_orientation = glm::normalize(delta) * m_orientation;
  glm::mat4 rotate = glm::mat4_cast(orientation);

  glm::mat4 translate = glm::mat4(1.0f);
  translate = glm::translate(translate, -eye);

  viewMatrix = rotate * translate;
}