iOS OpenGL ES 2.0 Billboard Object On Sphere And Rotate With Sphere

Question

I have a sphere (earth) in OpenGL ES 2.0 for iOS. I also have markers that I want to place at lat/lons on the earth - but I want the markers to always face the user ( billboards ) but also move with the Earth when it rotates by touch. So I've tried to research billboarding and put together the following code. This code is from the billboarding function that is called after I create the earth ( which is translated backwards 6 units on the Z axis). I can't seem to get the planes of the billboards to always face the camera but also move with the Earth as it rotates. How can I do that?

    // Get the current modelview matrix
    GLKMatrix4 originalMat = self.effect.transform.modelviewMatrix;
    GLKMatrix4 currMat = self.effect.transform.modelviewMatrix;

    // Define the buffer designators
    GLuint billboardVertexArray;
    GLuint billboardVertexBuffer;
    GLuint billboardIndexBuffer;

    glGenVertexArraysOES(1, &billboardVertexArray);
    glBindVertexArrayOES(billboardVertexArray);

    // Now draw the billboard
    glGenBuffers(1, &billboardVertexBuffer);
    glBindBuffer(GL_ARRAY_BUFFER, billboardVertexBuffer);
    glBufferData(GL_ARRAY_BUFFER, sizeof(Billboard_vertex), Billboard_vertex, GL_STATIC_DRAW);

    glGenBuffers(1, &billboardIndexBuffer);
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, billboardIndexBuffer);
    glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(Billboard_index), Billboard_index, GL_STATIC_DRAW);

    // u0,v0,normalx0,normaly0,normalz0,x0,y0,z0
    glEnableVertexAttribArray(GLKVertexAttribPosition);
    glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 8*sizeof(float), BUFFER_OFFSET(5));
    glEnableVertexAttribArray(GLKVertexAttribTexCoord0);
    glVertexAttribPointer(GLKVertexAttribTexCoord0, 2, GL_FLOAT, GL_FALSE, 8*sizeof(float), BUFFER_OFFSET(0));
    glEnableVertexAttribArray(GLKVertexAttribNormal);
    glVertexAttribPointer(GLKVertexAttribNormal, 3, GL_FLOAT, GL_FALSE, 8*sizeof(float), BUFFER_OFFSET(2));

    // Enable the Earth texture
    self.effect.texture2d0.name = _billBoardTextureInfo.name;
    self.effect.texture2d0.target = _billBoardTextureInfo.target;

    // Bind the earth vertex array
    glBindVertexArrayOES(billboardVertexArray);

    // Now put a billboard at a specific Lat Lon - so first
    // calculate XYZ from lat lon
    XYZ xyz;
    xyz.x = 0; xyz.y = 0; xyz.z = 0;
    [self LLAtoXYZwithLat:0 andLon:0 andXYZ:&xyz];
    //NSLog(@"XYZ after: %f %f %f",xyz.x,xyz.y,xyz.z);

    // Move the billboard back so we can see it
    GLKMatrix4 modelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -6.0f);

    // In update, we convert the quaternion into a rotation matrix, and apply it to the model view matrix as usual.
    GLKMatrix4 rotation = GLKMatrix4MakeWithQuaternion(_quat);
    modelViewMatrix = GLKMatrix4Multiply(modelViewMatrix, rotation);

    // First create the variation translations to anchor the billboard
    GLKMatrix4 translationXYZ = GLKMatrix4MakeTranslation(xyz.x, xyz.y, xyz.z);
    GLKMatrix4 translationForLatLonWithTranslation = GLKMatrix4Multiply(modelViewMatrix,translationXYZ);

    // Scale this object as well
    GLKMatrix4 scaledWithTranslationAndRotation = GLKMatrix4Scale(translationForLatLonWithTranslation, scale, scale, scale);

    // Remove the Translation portion of the matrix
    // | xx xy xz xw |
    // | yx yy yz yw |
    // | zx zy zz zw |
    // | wx wy wz ww |
    //
    // | R       T |
    // | (0,0,0) 1 |
    //
    // d = sqrt( xx² + yx² + zx² )
    //
    // | d 0 0 T.x |
    // | 0 d 0 T.y |
    // | 0 0 d T.z |
    // | 0 0 0   1 |
    //
    // union _GLKMatrix4
    // {
    //     struct
    //     {
    //         float m00, m01, m02, m03;
    //         float m10, m11, m12, m13;
    //         float m20, m21, m22, m23;
    //         float m30, m31, m32, m33;
    //     };
    //     float m[16];
    // }
    // typedef union _GLKMatrix4 GLKMatrix4;

    // Construct the rows in the new matrix
    float d = sqrt( pow(currMat.m00,2) + pow(currMat.m10,2) + pow(currMat.m20,2) );
    GLKVector4 columnToInsert0 = GLKVector4Make(d, 0, 0, currMat.m03+xyz.x);
    GLKVector4 columnToInsert1 = GLKVector4Make(0, d, 0, currMat.m13+xyz.y);
    //GLKVector4 columnToInsert2 = GLKVector4Make(0, 0, d, currMat.m23-6+xyz.z);
    GLKVector4 columnToInsert3 = GLKVector4Make(0, 0, 0, 1);

    // Build the new Matrix
    GLKMatrix4 noTranslationInfo = GLKMatrix4SetRow(currMat, 0, columnToInsert0);
    noTranslationInfo = GLKMatrix4SetRow(noTranslationInfo, 1, columnToInsert1);
    //noTranslationInfo = GLKMatrix4SetRow(noTranslationInfo, 2, columnToInsert2);
    noTranslationInfo = GLKMatrix4SetRow(noTranslationInfo, 3, columnToInsert3);

    [self printMatrix:noTranslationInfo];

    // Assign the new matrix to draw with - no translation
    self.effect.transform.modelviewMatrix = noTranslationInfo;

    // Render the object with GLKit
    [self.effect prepareToDraw];

    // Draw elements from the index array and uv / vertex / normal info
    glDrawElements(GL_TRIANGLES,Billboard_polygoncount*3,GL_UNSIGNED_SHORT,0);

    // Restore the original matrix
    self.effect.transform.modelviewMatrix = originalMat;

Answer 1

This seems to work pretty good:

// Get the current modelview matrix
GLKMatrix4 originalMat = self.effect.transform.modelviewMatrix;
GLKMatrix4 currMat = self.effect.transform.modelviewMatrix;

// Print the original matrix for comparison
//NSLog(@"Original Matrix:");
//[self printMatrix:currMat];

// Define the buffer designators
GLuint billboardVertexArray;
GLuint billboardVertexBuffer;
GLuint billboardIndexBuffer;

glGenVertexArraysOES(1, &billboardVertexArray);
glBindVertexArrayOES(billboardVertexArray);

// Now draw the billboard
glGenBuffers(1, &billboardVertexBuffer);
glBindBuffer(GL_ARRAY_BUFFER, billboardVertexBuffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(Billboard_vertex), Billboard_vertex, GL_STATIC_DRAW);

glGenBuffers(1, &billboardIndexBuffer);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, billboardIndexBuffer);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(Billboard_index), Billboard_index, GL_STATIC_DRAW);

// u0,v0,normalx0,normaly0,normalz0,x0,y0,z0
glEnableVertexAttribArray(GLKVertexAttribPosition);
glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 8*sizeof(float), BUFFER_OFFSET(5));
glEnableVertexAttribArray(GLKVertexAttribTexCoord0);
glVertexAttribPointer(GLKVertexAttribTexCoord0, 2, GL_FLOAT, GL_FALSE, 8*sizeof(float), BUFFER_OFFSET(0));
glEnableVertexAttribArray(GLKVertexAttribNormal);
glVertexAttribPointer(GLKVertexAttribNormal, 3, GL_FLOAT, GL_FALSE, 8*sizeof(float), BUFFER_OFFSET(2));

// Enable the Earth texture
self.effect.texture2d0.name = _billBoardTextureInfo.name;
self.effect.texture2d0.target = _billBoardTextureInfo.target;

// Bind the earth vertex array
glBindVertexArrayOES(billboardVertexArray);

// Now put a billboard at a specific Lat Lon - so first
// calculate XYZ from lat lon
XYZ xyz;
xyz.x = 0; xyz.y = 0; xyz.z = 0;
[self LLAtoXYZwithLat:35 andLon:-97 andXYZ:&xyz];
//NSLog(@"XYZ after: %f %f %f",xyz.x,xyz.y,xyz.z);

// Scale this object as well
//GLKMatrix4 scaledWithTranslationAndRotation = GLKMatrix4Scale(translationForLatLonWithTranslation, scale, scale, scale);

// Remove the Translation portion of the matrix
// | xx xy xz xw |
// | yx yy yz yw |
// | zx zy zz zw |
// | wx wy wz ww |
//
// | R       T |
// | (0,0,0) 1 |
//
// d = sqrt( xx² + yx² + zx² )
//
// | d 0 0 T.x |
// | 0 d 0 T.y |
// | 0 0 d T.z |
// | 0 0 0   1 |
//
// union _GLKMatrix4
// {
//     struct
//     {
//         float m00, m01, m02, m03;
//         float m10, m11, m12, m13;
//         float m20, m21, m22, m23;
//         float m30, m31, m32, m33;
//     };
//     float m[16];
// }
// typedef union _GLKMatrix4 GLKMatrix4;

// Construct the rows in the new matrix
float d = sqrt( pow(currMat.m00,2) + pow(currMat.m10,2) + pow(currMat.m20,2) );
GLKVector4 columnToInsert0 = GLKVector4Make(d, 0, 0, xyz.x);
GLKVector4 columnToInsert1 = GLKVector4Make(0, d, 0, xyz.y);
GLKVector4 columnToInsert2 = GLKVector4Make(0, 0, d, xyz.z);
GLKVector4 columnToInsert3 = GLKVector4Make(0, 0, 0, 1);

// Build the new Matrix
GLKMatrix4 noTranslationInfo = GLKMatrix4SetRow(currMat, 0, columnToInsert0);
noTranslationInfo = GLKMatrix4SetRow(noTranslationInfo, 1, columnToInsert1);
noTranslationInfo = GLKMatrix4SetRow(noTranslationInfo, 2, columnToInsert2);
noTranslationInfo = GLKMatrix4SetRow(noTranslationInfo, 3, columnToInsert3);

// Print out the 'no translation' matrix
//NSLog(@"No Translation Matrix:");
//[self printMatrix:noTranslationInfo];

// Get a rotation matrix from the quaternion we last rotated the globe with
GLKMatrix4 rotationMatrixFromQuaternion = GLKMatrix4MakeWithQuaternion( _quat );
//NSLog(@"Rotation Matrix from Quaternion: ");
//[self printMatrix:rotationMatrixFromQuaternion];

// Now use the matrix produced from our Quaternion to rotate the global coordinates
// of the billboard object
GLKMatrix4 rotatedNoTranslationInfo = GLKMatrix4Multiply(rotationMatrixFromQuaternion, noTranslationInfo);

//NSLog(@"rotatedNoTranslationInfo:");
//[self printMatrix:rotatedNoTranslationInfo];

// Throw the world translation coordinates in the matrix
noTranslationInfo.m30 = ( rotatedNoTranslationInfo.m30 );
noTranslationInfo.m31 = ( rotatedNoTranslationInfo.m31 );
noTranslationInfo.m32 = ( rotatedNoTranslationInfo.m32 + GLOBAL_EARTH_Z_LOCATION );

//NSLog(@"Final Matrix:");
//[self printMatrix:noTranslationInfo];

// Assign the new matrix to draw with - no translation
self.effect.transform.modelviewMatrix = noTranslationInfo;

// Render the object with GLKit
[self.effect prepareToDraw];

// Draw elements from the index array and uv / vertex / normal info
glDrawElements(GL_TRIANGLES,Billboard_polygoncount*3,GL_UNSIGNED_SHORT,0);

// Restore the original matrix
self.effect.transform.modelviewMatrix = originalMat;

Answer 2

When I've done this, I sent four vertices for the billboard, all from the center of the rectangle, to the vertex shader (really two sets of three, for the two triangles) and had the vertex shader displace them to always face the viewer. I describe this process in my answer here, but the relevant vertex shader code is as follows:

attribute vec4 position;
attribute vec4 inputImpostorSpaceCoordinate;

varying mediump vec2 impostorSpaceCoordinate;
varying mediump vec3 normalizedViewCoordinate;

uniform mat4 modelViewProjMatrix;
uniform mediump mat4 orthographicMatrix;
uniform mediump float sphereRadius;

void main()
{
    vec4 transformedPosition;
    transformedPosition = modelViewProjMatrix * position;
    impostorSpaceCoordinate = inputImpostorSpaceCoordinate.xy;

    transformedPosition.xy = transformedPosition.xy + inputImpostorSpaceCoordinate.xy * vec2(sphereRadius);
    transformedPosition = transformedPosition * orthographicMatrix;

    normalizedViewCoordinate = (transformedPosition.xyz + 1.0) / 2.0;
    gl_Position = transformedPosition;
}

The inputImpostorSpaceCoordinate attribute are four coordinates that range from (-1, -1) to (1, 1) and are supplied in parallel with the four vertices that make up the rectangular billboard you're trying to present. They act to displace the vertices relative to the screen and then also can provide texture coordinates for the kind of sub-map you'd use above. I used those values to do raytracing calculations for my sphere generation, but you can ignore that part of it.