opengl iOS glkit color picking

Question

Its about picking a 3d object using color-picking. I draw my mesh (update: Which resides in a static header file and is displayed fine to the screen in the same project) to a new offscreen framebuffer and then I use glReadPixels to identify if the user touched it.

Following the code from the project opengles_ch10_1 from the book "Learning OpenGLES for iOS" by Erik M.Buck, I wrote the following code which always prints to nslog the color 0,0,0 for the picked (by tapping) pixel.

My code is:

- (IBAction) tapGesture:(id)sender
{
   if ([(UITapGestureRecognizer *)sender state] == UIGestureRecognizerStateEnded) {

       CGPoint tapLocation = [(UITapGestureRecognizer *)sender locationInView:self.view];

        int tt = [self findMeshByPoint:tapLocation];

        //NSLog( @"tap value: %i", tt );
    }
}

- (NSUInteger)findMeshByPoint:(CGPoint)point
{
    //In openGL the y axis starts from the bottom of the screen

    point.y = self.view.bounds.size.height - point.y;

    GLKView *glView = (GLKView *)self.view;

    NSAssert([glView isKindOfClass:[GLKView class]],
             @"View controller's view is not a GLKView");

    // Make the view's context current

    [EAGLContext setCurrentContext:glView.context];

    glBindVertexArrayOES(0);

//        self.effect.constantColor = GLKVector4Make( 1.0f,  //This should be meshId/255.0f

    if(0 == _glVertexAttributeBufferID)
    {
        GLuint  glName;

        glGenBuffers(1,                // STEP 1

                     &glName);

        glBindBuffer(GL_ARRAY_BUFFER,  // STEP 2
                     glName);

        glBufferData(                  // STEP 3

                     GL_ARRAY_BUFFER,  // Initialize buffer contents

                    sizeof(parparit51OBJVertices), parparit51OBJVertices,

                     GL_STATIC_DRAW);            // Hint: cache in GPU memory

        _glVertexAttributeBufferID = glName;

    }
    else
    {
        glBindBuffer(GL_ARRAY_BUFFER,

                     _glVertexAttributeBufferID);
    }

    //glEnableVertexAttribArray(TETerrainPositionAttrib);

    glEnableVertexAttribArray( GLKVertexAttribPosition );

     if(0 == _program)
     {
            [self loadShadersWithName:@"UtilityPickTerrainShader"];

            [self buildFBO];

            NSAssert(0 != _program,

                     @"prepareOpenGL failed to load shaders");
        }

        glUseProgram(_program);

        // Pre-calculate the mvpMatrix

        GLKMatrix4 modelViewProjectionMatrix2 =

        GLKMatrix4Multiply(

                           self.effect.transform.projectionMatrix,

                           self.effect.transform.modelviewMatrix);

        // Standard matrices

        glUniformMatrix4fv(uniforms[UtilityPickTerrainMVPMatrix], 1, 0,

                           modelViewProjectionMatrix2.m);

//        glUniform2fv(uniforms[UtilityPickTerrainDimensionFactors], 1,

//                     &_temp1 ); //self.factors.v);  //I removed this from the shaders

        glUniform1f(uniforms[UtilityPickTerrainModelIndex],

                    1.0f ); // self.modelIndex / 255.0f);

    glBindFramebuffer(GL_FRAMEBUFFER, _pickFBO);

    glViewport(0, 0, 512, 512);

    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    //glEnableVertexAttribArray(GLKVertexAttribPosition);

    glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 12 * sizeof(GLfloat), 0);

    NSLog( @"******* 7" );

    [self.effect prepareToDraw];

    glDrawArrays(GL_TRIANGLES, 0, sizeof(parparit51OBJVertices) / sizeof(Vertex));

    NSLog( @"******* 7.5" );

    const GLfloat width = [glView drawableWidth];

    const GLfloat height = [glView drawableHeight];

    NSAssert(0 < width && 0 < height, @"Invalid drawble size");

    // Get info for picked location

    const GLKVector2 scaledProjectionPosition = {

        point.x / width,

        point.y / height

    };

    NSLog( @"******* 8" );

    GLubyte pixelColor[4];  // Red, Green, Blue, Alpha color

    GLint readLocationX = MIN((512 - 1),

                              (512 - 1) * scaledProjectionPosition.x);

    GLint readLocationY = MIN((512 - 1),

                              (512 - 1) * scaledProjectionPosition.y);

    glReadPixels(readLocationX,

                 readLocationY,

                 1,

                 1,

                 GL_RGBA,

                 GL_UNSIGNED_BYTE,

                 pixelColor);


    NSLog(@"pixelColor[0]=%i, pixelColor[1]=%i, pixelColor[2]=%i", pixelColor[0], pixelColor[1], pixelColor[2] );


    // Restore OpenGL state that pickTerrainEffect changed

    glBindFramebuffer(GL_FRAMEBUFFER, 0); // default frame buffer

    glViewport(0, 0, width, height); // full area of glView

    return 0;

}

-(void) buildFBO
{
    if ( 0 == _pickFBO )
    {

        GLuint colorTexture;

        // Create a texture object to apply to model

        glGenTextures(1, &colorTexture);

        glBindTexture(GL_TEXTURE_2D, colorTexture);

        // Set up filter and wrap modes for this texture object

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S,
                        GL_CLAMP_TO_EDGE);

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T,
                        GL_CLAMP_TO_EDGE);

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,
                        GL_LINEAR);

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
                        GL_LINEAR_MIPMAP_LINEAR);

        // Allocate a texture image we can render into

        // Pass NULL for the data parameter since we don't need to

        // load image data. We will be generating the image by

        // rendering to this texture.

        glTexImage2D(GL_TEXTURE_2D,

                     0,

                     GL_RGBA,

                     512,

                     512,

                     0,

                     GL_RGBA,

                     GL_UNSIGNED_BYTE,

                     NULL);

        GLuint depthRenderbuffer;

        glGenRenderbuffers(1, &depthRenderbuffer);

        glBindRenderbuffer(GL_RENDERBUFFER, depthRenderbuffer);

        glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16,

                              512, 512);

        glGenFramebuffers(1, &_pickFBO);

        glBindFramebuffer(GL_FRAMEBUFFER, _pickFBO);

        glFramebufferTexture2D(GL_FRAMEBUFFER,

                               GL_COLOR_ATTACHMENT0,

                               GL_TEXTURE_2D, colorTexture, 0);

        glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthRenderbuffer);

        if(glCheckFramebufferStatus(GL_FRAMEBUFFER) !=

           GL_FRAMEBUFFER_COMPLETE)
        {
            NSLog(@"failed to make complete framebuffer object %x", glCheckFramebufferStatus(GL_FRAMEBUFFER));

            //+++tbd+++UtilityPickTerrainEffectDestroyFBO(fboName);

            return;
        }

        //#ifdef DEBUG
        //    {  // Report any errors
        GLenum error = glGetError();

        if(GL_NO_ERROR != error)
        {

            NSLog(@"GL Error: 0x%x", error);

        }
        //    }
        //#endif
    }
}

Answer 1

I had a BAD_ACCESS error. I solved this by removing the code which handles the indices:

if(0 == _indexBufferID)
{
  // Indices haven't been sent to GPU yet
  // Create an element array buffer for mesh indices
    glGenBuffers(1, &_indexBufferID);
    NSAssert(0 != _indexBufferID,
             @"Failed to generate element array buffer");

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, _indexBufferID);
    glBufferData(GL_ELEMENT_ARRAY_BUFFER,
                 sizeof( parparit51OBJIndices ),
                 parparit51OBJIndices,
                 GL_STATIC_DRAW);
}
else
{
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, _indexBufferID);
}

Now I still need to understand why the code updated above always print 0,0,0 for any picked pixel.

Answer 2

Your error is that you are trying to access a Null Pointer -- you are almost certainly passing an array that you never actually initialized to your GLKEffect or giving it to OpenGL with a bufferData call, given that the error is occurring inside of prepareToDraw or glDrawArrays. There are several possible explanations for this that I can see, though I can't confirm any of them, since the relevant information is how you allocate the data that you are using either in perpareToDraw or in glDrawArrays.

The first is that "parparit51OBJVertices" might be allocated on the heap dynamically (are you calling malloc or the like? How do you specify the size of the array), in which case your calls to sizeof will be returning incorrect values (0 I think) which might lead to the EXEC_BAD_ACESS.

The other part of this that seems suspect is that you are calling glDrawArrays and passing in the number of vertices, but before that you bind a VBO for indices -- at best this is wasted work, since glDrawArrays will ignore the indices. Worse, if you are trying to draw an OBJ, as your variable name implies, then likely there is a considerable amount of geometry you aren't drawing at all, so even if you fix your EXEC_BAD_ACCESS problem you still will be getting bad results.