ray intersection misses the target

Question

I'm trying to pick a 3d point. I read various sites but my code doesn't work.

on right mouse click:

glGetFloatv(GL_MODELVIEW_MATRIX,mv_mat)
glGetFloatv(GL_PROJECTION_MATRIX,p_mat)

ip_mat = np.linalg.inv(mat4(p_mat))

# clip = array[
# (2*x)/window_width-1
# 1-(y*2)/window.height
# -1
# 1
camera_coord = np.dot(ip_mat,clip)
camera_coord = np.array([camera_coord[0,0],camera_coord[0,1],-1,0])

imv_mat = np.linalg.inv(mat4(mv_mat))

ray_world = np.dot(imv_mat,camera_coord)
ray_world = np.array([ray_world[0],ray_world[1],ray_world[2]])
ray_world = ray_world/np.linalg.norm(ray_world)

Intersect_sphere function:

v = np.array([model.rx,model.ry,model.rz]) - np.array([-0.5, 0.5, 0])
b = 2 * np.dot(v, ray_world)
c = np.dot(v, v) - 1 * 1
delta = b * b - 4 * c

if (delta > 0):
    print('select')
    return True

return False

edit: I found a typo. Even after changing the code still does not work.

Answer 1

If you want to pick a point in the window, then you have to transform form window coordinates to world coordinates or object coordinates.

To map window coordinates to object coordinates, gluUnProject can be used.
The parameters to gluUnProject are of type GLdouble.

Create an array for the projection matrix and the view matrix of type GLdouble and an array of type GLint for the viewport rectangle:

self.mv_mat = (GLdouble * 16)()
self.p_mat  = (GLdouble * 16)()
self.v_rect = (GLint * 4)()

Get the current projection matrix, model view matrix and viewport rectangle:

glGetDoublev(GL_MODELVIEW_MATRIX, self.mv_mat)
glGetDoublev(GL_PROJECTION_MATRIX, self.p_mat)
glGetIntegerv(GL_VIEWPORT, self.v_rect)

On the viewport is drawn the 2 dimensional (perspective) projection of a 3 dimensional scene. The scene is looked at from one point, the camera position. To find the object which is "picked" in the window, you have to find a the viewing ray where the object is on. A ray is defined by 2 points. Find a point near to the camera and a point far in the depth of the scene, which are on the "picked" window position to define the ray. The picked object is that object, which is closest to the camera. In normalized device space, all points which have the same x and y coordinate are on the same ray, as seen from the camera position.
The 1st and 2nd coordinates of a point in window space are the x and y coordinate in pixel, the 3rd coordinate is the depth in range [0, 1].
So a ray trough coordinate (x,y) from near the camera to the far depth is defined by the 2 points p0 and p1, where:

p0 = (x, y, 0)
p1 = (x, y, 1)

This to points have to be transformed 2 world space by gluUnProject:

ray_near = [GLdouble() for _ in range(3)]
ray_far  = [GLdouble() for _ in range(3)]
gluUnProject(x, y, 0, mv_mat, p_mat, v_rect, *ray_near)
gluUnProject(x, y, 1, mv_mat, p_mat, v_rect, *ray_far)

A ray intersects a sphere, if distance from the center point of the sphere, to the nearest point on the ray, is less or equal the radius of the sphere.

Calculate the normalized direction of the ray:

p0 = [v.value for v in ray_near]
p1 = [v.value for v in ray_far]

r_dir = np.subtract(p0, p1)
r_dir = r_dir / np.linalg.norm(r_dir)

Calculate the nearest point on the ray to the center point of the sphere:

p0_cpt = np.subtract(p0, cpt)
near_pt = np.subtract(p0, r_dir * np.dot(p0_cpt, r_dir))

Calculate the distance of the point on the ray to the center point:

dist = np.linalg.norm(np.subtract(near_pt, cpt))

If the distance is less or equal the radius of the sphere, then the ray hits the sphere:

isIntersecting = dist <= radius

See the short PyGlet example:

from pyglet.gl import *
from pyglet.window import key
import numpy as np

class Window(pyglet.window.Window):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.sphere = gluNewQuadric() 
        self.vp_valid = False
        self.mouse_pos = (0, 0)
        self.mv_mat = (GLdouble * 16)()
        self.p_mat  = (GLdouble * 16)()
        self.v_rect = (GLint * 4)() 

    def on_resize(self, width, height):
        self.vp_valid = False

    def isectSphere(self, p0, p1, cpt, radius):

        # normalized ray direction
        r_dir = np.subtract(p0, p1)
        r_dir = r_dir / np.linalg.norm(r_dir)

        # nearest point on the ray to the sphere
        p0_cpt = np.subtract(p0, cpt)
        near_pt = np.subtract(p0, r_dir * np.dot(p0_cpt, r_dir))

        # distance to center point
        dist = np.linalg.norm(np.subtract(near_pt, cpt))

        # intersect if dist less or equal the radius of the sphere 
        return dist <= radius

    def on_draw(self):

        if not self.vp_valid:
            self.vp_valid = True
            glViewport(0, 0, self.width, self.height)
            glMatrixMode(GL_PROJECTION)
            glLoadIdentity()
            gluPerspective(45, self.width/self.height, 0.1, 50.0)
            glMatrixMode(GL_MODELVIEW)
            glLoadIdentity()
            gluLookAt(0, -8, 0, 0, 0, 0, 0, 0, 1)

            glGetDoublev(GL_MODELVIEW_MATRIX, self.mv_mat)
            glGetDoublev(GL_PROJECTION_MATRIX, self.p_mat)
            glGetIntegerv(GL_VIEWPORT, self.v_rect)

        temp_val = [GLdouble() for _ in range(3)]
        gluUnProject(*self.mouse_pos, 0, self.mv_mat, self.p_mat, self.v_rect, *temp_val)
        self.mouse_near = [v.value for v in temp_val]
        gluUnProject(*self.mouse_pos, 1, self.mv_mat, self.p_mat, self.v_rect, *temp_val)
        self.mouse_far = [v.value for v in temp_val]

        isect_a = self.isectSphere(self.mouse_near, self.mouse_far, [-1.5, 0, 0], 1)
        isect_b = self.isectSphere(self.mouse_near, self.mouse_far, [1.5, 0, 0], 1)

        glEnable(GL_DEPTH_TEST)
        glEnable(GL_LIGHTING)
        glShadeModel(GL_SMOOTH)
        glEnable(GL_COLOR_MATERIAL)
        glColorMaterial(GL_FRONT_AND_BACK, GL_AMBIENT_AND_DIFFUSE)

        glEnable(GL_LIGHT0)
        glLightfv(GL_LIGHT0, GL_AMBIENT, (GLfloat *4)(0.1, 0.1, 0.1, 1))
        glLightfv(GL_LIGHT0, GL_DIFFUSE, (GLfloat *4)(1.0, 1.0, 1.0, 1))
        glLightfv(GL_LIGHT0, GL_POSITION, (GLfloat *4)(0, -1, 0, 0))

        glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)

        glPushMatrix()

        glTranslatef(-1.5, 0, 0)
        if isect_a:
            glColor4f(1.0, 0.5, 0.5, 1)
        else:
            glColor4f(0.5, 0.2, 0.2, 1)
        gluSphere(self.sphere, 1.0, 32, 16)

        glTranslatef(3, 0, 0)
        if isect_b:
            glColor4f(0.5, 0.5, 1.0, 1)
        else:
            glColor4f(0.2, 0.2, 0.5, 1)
        gluSphere(self.sphere, 1.0, 32, 16)

        glPopMatrix()

    def on_mouse_motion(self,x,y,dx,dy):
        self.mouse_pos = (x, y)

if __name__ == "__main__":
    window = Window(width=800, height=600, resizable=True)
    pyglet.app.run()