Determine what object a mouse is pointed over?

Question

Can someone show me the simplest way you can think of to determine what object is being clicked so I can move that object independently using the glut functions?

Answer 1

The questionnaire asked for a 2D-solution, but it wasn't obvious at first glance. For you all out there who wants to do this in 3D, but don't want to put together some kind of pixel_id mapping buffer, I put together the following solution.

It makes a ray from the camera which can be used to find a collision when searching through all objects. It will select the collision closest to the camera.

Note! This is slow if you have many objects, as it needs to loop through them all.

//Must put somewhere in your code to initialize mouse handler
glutMouseFunc(HandleMouse);

Main principle:

#include <limits>
#include <Eigen/Core>
#include <GL/glew.h>
#include <GL/glut.h>
using namespace Eigen;

//You glut mouse handling function
void HandleMouse(int button, int state, int x, int y) {

    //Assuming you only want to process left mouse button clicks
    if (state != GLUT_DOWN || button != GLUT_LEFT_BUTTON) {
        return;
    }

    
    Vector3d up; up << 0, 0, 1; // if you changed your "up"-vector in your world, then this line needs to be fixed

    //Assuming you keep track of the camera position and where it is pointing
    //Do some math and make a ray come out of the camera
    Matrix4d custom_modelview = GetModelViewFromCamPos(camera_pos, camera_look_at, up);
    Matrix4d inverse_projection = GetInverseProjectionMatrix();
    Vector3d ray_from_camera = Get3dRayInWorldFromModelViewAndInverseProjection(x, y, custom_modelview, inverse_projection);

    //Search the object which intersects with ray
    SearchForClosestIntersectingObject(ray_from_camera, camera_pos);

}


//Your intersection search function
void SearchForClosestIntersectingObject(Vector3d ray_from_camera, Vector3d camera_pos) {
    double smallest_dist_to_camera = std::numeric_limits<double>::max();
    Foo* leading_cand = NULL;

    //Search
    for (Foo* obj : my_objects) {

        //You will need to implement your own collision detector. This function should take in a camera position and a 3d vector ray which goes out
        //from the camera. It should return the distance from a collision to camera if a collision is detected, or else return -1.
        //The math is different depending on if you are intersecting with a plane, a ball, a square or triangle etc.
        double intersection_dist_to_camera = obj->RayCollidesWithMe(ray_from_camera, camera_pos);

        if (intersection_dist_to_camera != -1 && intersection_dist_to_camera < smallest_dist_to_camera) {
            smallest_dist_to_camera = intersection_dist_to_camera;
            leading_cand = obj;
        }

    }
    if (leading_cand != NULL) {
        //Do whatever with your clicked object
        DoWhatever(leading_cand);
    }


}

Helping functions:

//Make a modelview matrix from a specific camera position, target and direction up;
Matrix4d GetModelViewFromCamPos(Vector3d eye, Vector3d target, Vector3d upDir)
{
    // compute the forward vector from target to eye
    Vector3d forward = eye - target;
    forward.normalize();                 // make unit length

    // compute the left vector
    Vector3d left = upDir.cross(forward); // cross product
    left.normalize();

    // recompute the orthonormal up vector
    Vector3d up = forward.cross(left);    // cross product

    // init 4x4 matrix
    Matrix4d matrix; matrix << 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1;

    // set rotation part, inverse rotation matrix: M^-1 = M^T for Euclidean transform
    matrix << left(0), up(0), forward(0), 0, left(1), up(1), forward(1), 0, left(2), up(2), forward(2), 0, -left(0) * eye(0) - left(1) * eye(1) - left(2) * eye(2), -up(0) * eye(0) - up(1) * eye(1) - up(2) * eye(2), -forward(0) * eye(0) - forward(1) * eye(1) - forward(2) * eye(2), 1;


    return matrix;
}

//A function which grabs a copy of the projection matrix from GL and makes some changes to it
Matrix4d GetInverseProjectionMatrix() {
    double op[16]; // original projection matrix
    glGetDoublev(GL_PROJECTION_MATRIX, op);

    double a = op[0];
    double b = op[5];
    double c = op[10];
    double d = op[14];
    double e = op[11];

    double inverse_proj_arr[16] = { 0.0 };
    inverse_proj_arr[0] = 1.0 / a;
    inverse_proj_arr[5] = 1.0 / b;
    inverse_proj_arr[11] = 1.0 / d;
    inverse_proj_arr[14] = 1.0 / e;
    inverse_proj_arr[15] = -c / (d * e);

    Matrix4d inverse_proj;
    inverse_proj = Map<Matrix4d>(inverse_proj_arr);
    return inverse_proj;
}

//Make a ray
Vector3d Get3dRayInWorldFromModelViewAndInverseProjection(int mouse_x, int mouse_y, Matrix4d mv, Matrix4d ip) {
    float space_x = (2.0f * mouse_x) / glutGet(GLUT_WINDOW_WIDTH) - 1.0f;
    float space_y = 1.0f - (2.0f * mouse_y) / glutGet(GLUT_WINDOW_HEIGHT);
    float space_z = 1.0f;
    Vector3d ray_nds;
    ray_nds << space_x, space_y, space_z;
    Vector4d ray_clip = Vector4d(ray_nds(0), ray_nds(1), -1.0, 1.0);
    Vector4d ray_eye = ip * ray_clip;
    ray_eye << ray_eye(0), ray_eye(1), -1, 0;
    Vector4d ray_world_4d = mv.inverse().transpose() * (ray_eye);
    Vector3d ray_world_3d; ray_world_3d << ray_world_4d(0), ray_world_4d(1), ray_world_4d(2);
    ray_world_3d.normalize();
    return ray_world_3d;
}

Answer 2

Regardless of whether you're working with 2D or 3D by far the simplest solution is to assign each pickable object a unique colour, which isn't normally used at all. Then for every click event you render (to the back buffer) the same scene with the unique colour applied to each object. By disabling lighting etc. the problem then becomes one of looking at the pixel color under the mouse and using a lookup table to see which object was clicked.

Even in 16 bit colour depth that still gets you 2^16 unique pickable objects and in reality that's rare in modern application to have less than 2^24.

Answer 3

As we are working with 2D objects, an object is pointed at if the position of the mouse is inside the object. This notion of being inside differs for different geometric shapes.

For a rectangle width upper left corner c and width, height the the function could look like:

bool isInsideRectangle(double x, double y) {
    // The mouse is inside if the coordinates are 'inside' all walls
    return (x > c.x &&
            y > c.y &&
            x < c.x + width &&
            y < c.y + height);
}

For a circle with center c and radius r, it could look like this:

bool isInsideCircle(double x, double y) {
    // The mouse is inside if the distance to the center of the circle
    // is less than the radius
    return std::sqrt((x - c.x)*(x - c.x) + (y - c.y)*(y - c.y)) < r;
}

For another shape you would have to figure out another function for how to calculate if a mouse position is inside or not, however in many cases you can simplify it to a bounding rectangle or sphere.