Where did I go wrong trying to implement picking with raycasting in OpenGL ES 2.0 for iOS?

Question

I'm trying to implement picking with raycasting in OpenGL ES 2.0 for an iPad App. Specifically, I want to know which cell of a minecraft-like map made up of square cells with varying heights the user has tapped on. However, my code does not find an intersection with any cell.

In my gesture recognizer method, I first get the point and viewport dimensions

UITapGestureRecognizer* tapRecognizer = (UITapGestureRecognizer*) recognizer;
CGPoint tapLoc = [tapRecognizer locationInView: self.view];

GLint viewport[4];
glGetIntegerv(GL_VIEWPORT, viewport);

Then I flip the x axis , because my app is in landscape mode and openGL coords start at a different corner than window coords

tapLoc.x = viewport[2] -  tapLoc.x;

And perform the unprojection with GLKitMathUnproject for z = 0 and z = 1 values, then pass the resulting vectors into the function which will return the coordinates of the cell the user tapped as a struct BPPoint (just two integers, x and y). The map currently has no transformations on it, so I assume that I should use the identity matrix for its transform, and the projection matrix was made with glkmatrix4makeperspective.

GLKVector3 nearPt = GLKMathUnproject(GLKVector3Make(tapLoc.x, tapLoc.y, 0.0), GLKMatrix4Identity, [BPGlobalEnv globalVars].cameraMatrix, &viewport[0] , &testResult);

GLKVector3 farPt = GLKMathUnproject(GLKVector3Make(tapLoc.x, tapLoc.y, 1.0), GLKMatrix4Identity, [BPGlobalEnv globalVars].cameraMatrix, &viewport[0] , &testResult);

BPPoint destination = [gameEngine.currMap gridForPoint: farPt fromPoint: nearPt];

This is the body of that method. I basically just loop through and re-generate the geometry for the top of each square of my map (inefficient, I know) by the same method I used to get the geometry to pass into a VBO to draw it in the first place.

-(BPPoint) gridForPoint: (GLKVector3) ray fromPoint: (GLKVector3) cameraCoords

{
size_t i = 0;
size_t j;

GLfloat* buffer = malloc(sizeof(GLfloat) * squareSize);

for (NSArray* row in self.mapCells) {

    j = 0;

    for (BPGridCell* cell in row) {

        GLfloat a[3];
        a[0] = i * tileSize;
        a[1] = cell.height * step;
        a[2] = j * tileSize;
        GLfloat b[3];
        b[0] = (i + 1) * tileSize;
        b[1] = cell.height * step;
        b[2] = (j + 1) * tileSize;

        [self squareForPoint:a point:b pointer: buffer];

        GLKVector3 v1 = GLKVector3Make(buffer[0], buffer[1], buffer[2]);
        GLKVector3 v2 = GLKVector3Make(buffer[3], buffer[4], buffer[5]);
        GLKVector3 v3 = GLKVector3Make(buffer[6], buffer[7], buffer[8]);
        GLKVector3 v21 = GLKVector3Make(buffer[9], buffer[10], buffer[11]);
        GLKVector3 v22 = GLKVector3Make(buffer[12], buffer[13], buffer[14]);
        GLKVector3 v23 = GLKVector3Make(buffer[15], buffer[16], buffer[17]);

        if ([self ray: ray fromCamera: cameraCoords intersectsTriangleWithV1: v1 V2: v2 V3: v3] ||
            [self ray: ray fromCamera: cameraCoords intersectsTriangleWithV1: v21 V2: v22 V3: v23]) {

            NSLog(@"found intersection");
            free(buffer);
            BPPoint toReturn;
            toReturn.x = i;
            toReturn.y = j;
            NSLog(@"%ld, %ld", i, j);
            return toReturn;
        }

        j += 1;
    }

    i += 1;
}

free(buffer);
BPPoint toReturn;
toReturn.x = i;
toReturn.y = j;
NSLog(@"%ld, %ld", i, j);
return toReturn;
}

And the code for testing a triangle for intersection, which I converted to objc from here (http://www.cs.virginia.edu/~gfx/Courses/2003/ImageSynthesis/papers/Acceleration/Fast%20MinimumStorage%20RayTriangle%20Intersection.pdf) follows:

-(BOOL) ray: (GLKVector3) ray fromCamera: (GLKVector3) camera intersectsTriangleWithV1: (GLKVector3) v1 V2: (GLKVector3) v2 V3: (GLKVector3) v3
{

GLKVector3 e1 = GLKVector3Subtract(v2, v1);
GLKVector3 e2 = GLKVector3Subtract(v3, v1);

GLKVector3 h = GLKVector3CrossProduct(ray, e2);

float det = GLKVector3DotProduct(e1, h);

if (det > -0.00001 && det < 0.00001) {

    return NO;
}

float invDet = 1.0/det;
GLKVector3 s = GLKVector3Subtract(camera, v1);

float u =  GLKVector3DotProduct(s, h) * invDet;

if (u < 0.0 || u > 1.0) {

    return NO;
}

GLKVector3 q = GLKVector3CrossProduct(s, e1);

float v = invDet * GLKVector3DotProduct(ray, q);

if (v < 0.0 || u + v > 1.0) {

    return NO;
}

    return YES;
}

When I run this with a 10x10 map, it prints 10,10 for the coordinates without finding an intersection typically. Where did I screw up conceptually or in implementing this or both?

Answer 1

I have found the solution: The glgetintegerv call returned flipped values, presumably because the device was in landscape mode. I also had to subtract the near point from the far point to get a correct direction vector for the ray.