gluDisk rotation for mapping

Question

I'm trying to create sub-cursor for terrain mapping.

Basic by code: (old image, but rotation is same)

image http://www.sdilej.eu/pics/274a90360f9c46e2eaf94e095e0b6223.png

This is when i testing change glRotate ax to my numbers:

image2 http://www.sdilej.eu/pics/146bda9dc51708da54b9249706f874fc.png

What i want:

image3 http://www.sdilej.eu/pics/69721aa237608b423b635945d430e561.png

My code:

void renderDisk(float x1, float y1, float z1, float x2, float y2, float z2, float radius, int subdivisions, GLUquadricObj* quadric)
{
  float vx = x2 - x1;
  float vy = y2 - y1;
  float vz = z2 - z1;

  //handle the degenerate case of z1 == z2 with an approximation
  if( vz == 0.0f )
    vz = .0001f;

  float v = sqrt( vx*vx + vy*vy + vz*vz );
  float ax = 57.2957795f * acos( vz/v );
  if(vz < 0.0f)
    ax = -ax;

  float rx = -vy * vz;
  float ry = vx * vz;

  glPushMatrix();

  glTranslatef(x1, y1, z1);
  glRotatef(ax, rx, ry, 0.0);

  gluQuadricOrientation(quadric, GLU_OUTSIDE);
  gluDisk(quadric, radius - 0.25, radius + 5.0, subdivisions, 5);

  glPopMatrix();
}
void renderDisk_convenient(float x, float y, float z, float radius, int subdivisions)
{
  // Mouse opacity
  glColor4f( 0.0f, 7.5f, 0.0f, 0.5f );
  GLUquadricObj* quadric = gluNewQuadric();
  gluQuadricDrawStyle(quadric, GLU_LINE);
  gluQuadricNormals(quadric, GLU_SMOOTH);
  gluQuadricTexture(quadric, GL_TRUE);
  renderDisk(x, y, z, x, y, z, radius, subdivisions, quadric);
  gluDeleteQuadric(quadric);
}

renderDisk_convenient(posX, posY, posZ, radius, 20);

Answer 1

This is a simple one. In your call to renderDisk() you supply bad arguments. Looks like you copied the function from some tutorial without understanding how it works. The first three parameters control the center position, and the other three parameters control rotation using a second position which the disk is always facing. If the two positions are equal (which is your case), this line is executed:

//handle the degenerate case of z1 == z2 with an approximation
if( vz == 0.0f )
    vz = .0001f;

And setting z to nonzero makes the disc perpendicular to XZ plane, which is also the horizontal plane for your terrain. So ... to make it okay, you need to modify your function like this:

void renderDisk_convenient(float x, float y, float z, float radius, int subdivisions)
{
    // Mouse opacity
    glColor4f( 0.0f, 7.5f, 0.0f, 0.5f );
    GLUquadricObj* quadric = gluNewQuadric();
    gluQuadricDrawStyle(quadric, GLU_LINE);
    gluQuadricNormals(quadric, GLU_SMOOTH);
    gluQuadricTexture(quadric, GL_TRUE);
    float upX = 0, upY = 1, upZ = 0; // up vector (does not need to be normalized)
    renderDisk(x, y, z, x + upX, y + upY, z + upZ, radius, subdivisions, quadric);
    gluDeleteQuadric(quadric);
}

This should turn the disc into the xz plane so it will be okay if the terrain is flat. But in other places, you actually need to modify the normal direction (the (upX, upY, upZ) vector). If your terrain is generated from a heightmap, then the normal can be calculated using code such as this:

const char *p_s_heightmap16 = "ps_height_1k.png";
const float f_terrain_height = 50; // terrain is 50 units high
const float f_terrain_scale = 1000; // the longer edge of terrain is 1000 units long

TBmp *p_heightmap;
if(!(p_heightmap = p_LoadHeightmap_HiLo(p_s_heightmap16))) {
    fprintf(stderr, "error: failed to load heightmap (%s)\n", p_s_heightmap16);
    return false;
}
// load heightmap

TBmp *p_normalmap = TBmp::p_Alloc(p_heightmap->n_width, p_heightmap->n_height);
// alloc normalmap

const float f_width_scale = f_terrain_scale / max(p_heightmap->n_width, p_heightmap->n_height);
// calculate the scaling factor

for(int y = 0, hl = p_normalmap->n_height, hh = p_heightmap->n_height; y < hl; ++ y) {
    for(int x = 0, wl = p_normalmap->n_width, wh = p_heightmap->n_width; x < wl; ++ x) {
        Vector3f v_normal(0, 0, 0);
        {
            Vector3f v_pos[9];
            for(int yy = -1; yy < 2; ++ yy) {
                for(int xx = -1; xx < 2; ++ xx) {
                    int sx = xx + x;
                    int sy = yy + y;
                    float f_height;
                    if(sx >= 0 && sy >= 0 && sx < wh && sy < hh)
                        f_height = ((const uint16_t*)p_heightmap->p_buffer)[sx + sy * wh] / 65535.0f * f_terrain_height;
                    else
                        f_height = 0;

                    v_pos[(xx + 1) + 3 * (yy + 1)] = Vector3f(xx * f_width_scale, f_height, yy * f_width_scale);
                }
            }
            // read nine-neighbourhood

            /*
                0          1          2
                +----------+----------+
                |\         |         /|
                |  \       |       /  |
                |    \     |     /    |
                |      \   |   /      |
               3|_________\|/_________|5
                |        4/|\         |
                |       /  |   \      |
                |    /     |     \    |
                |  /       |       \  |
                |/         |         \|
                +----------+----------+
                6          7          8

             */

            const int p_indices[] = {
                0, 1, //4,
                1, 2, //4,
                2, 5, //4,
                5, 8, //4,
                8, 7, //4,
                7, 6, //4,
                6, 3, //4,
                3, 0 //, 4
            };

            for(int i = 0; i < 8; ++ i) {
                Vector3f a = v_pos[p_indices[i * 2]];
                Vector3f b = v_pos[p_indices[i * 2 + 1]];
                Vector3f c = v_pos[4];
                // triangle

                Vector3f v_tri_normal = (a - c).v_Cross(b - c);
                v_tri_normal.Normalize();
                // calculate normals

                v_normal += v_tri_normal;
            }
            v_normal.Normalize();
        }
        // calculate normal from the heightmap (by averaging the normals of eight triangles that share the current point)

        uint32_t n_normalmap =
            0xff000000U |
            (max(0, min(255, int(v_normal.z * 127 + 128))) << 16) |
            (max(0, min(255, int(v_normal.y * 127 + 128))) << 8) |
            max(0, min(255, int(-v_normal.x * 127 + 128)));
        // calculate normalmap color

        p_normalmap->p_buffer[x + wl * y] = n_normalmap;
        // use the lightmap bitmap to store the results
    }
}

(note this contains some structures and functions that are not included here so you won't be able to use this code directly, but the basic concept is there)

Once you have the normals, you need to sample normal under location (x, z) and use that in your function. This will still make the disc intersect the terrain where there is a steep slope next to flat surface (where the second derivative is high). In order to cope with that, you can either lift the cursor up a bit (along the normal), or disable depth testing.

If your terrain is polygonal, you could use vertex normals just as well, just take triangle that is below (x, y, z) and interpolate it's vertices normals to get the normal for the disc.

I hope this helps, feel free to comment if you need further advice ...