CODEWITHSUNDEEP
opengl-esrenderingopengl-es-2.0stencil-buffer
Dmitry
Dmitry

Reputation: 3079

Strange bugs occur while using stencil testing

I'm making an app which shows a terrain and some custom pits which should be inserted into the terrain. To insert the pits into the terrain I use stencil testing. It works almost great, but some strange bugs are present.

The code:

glEnable(GL_STENCIL_TEST);

glStencilFuncSeparate(GL_FRONT, GL_ALWAYS, 1, 255);
glStencilOpSeparate(GL_FRONT, GL_KEEP, GL_KEEP, GL_REPLACE);

glStencilFuncSeparate(GL_BACK, GL_ALWAYS, 1, 255);
glStencilOpSeparate(GL_BACK, GL_KEEP, GL_KEEP, GL_KEEP);

glEnable(GL_DEPTH_TEST);
glDepthFunc(GL_LEQUAL);
glDepthMask(GL_TRUE);

for (Pit3D* pit in pitList) {
    [pit render];
}

glStencilFuncSeparate(GL_FRONT, GL_EQUAL, 0, 255);
glStencilOpSeparate(GL_FRONT, GL_REPLACE, GL_KEEP, GL_KEEP);

glStencilFuncSeparate(GL_BACK, GL_EQUAL, 0, 255);
glStencilOpSeparate(GL_BACK, GL_REPLACE, GL_KEEP, GL_KEEP);

[terrain render];

Bug #1:

When I look through a hill to the pits I see some strange shadows or black spots. Although I look through an another hill to the pits I don't see that shadows or spots.

Bug #1

Bug #2:

Some strange pixels are present when the pits are close to the camera. They definitely belongs to the pits but they shouldn't be visible.

Unfortunately I don't have screenshot at the moment. It will be added later today.

Bug #3

Depends on range and view angle sometimes the pits are covered by the terrain. But when the pits are close to the camera the terrain cover is disappeared from the pits.

Bug #3

I have no idea how to fix these bugs. Please help me. All answers will be upvoted.

UPDATES:

I've recently discovered that the pits are covered by terrain back-face.

BUG #3 1

Upvotes: 3

Views: 318

Answers (1)

Rabbid76
Rabbid76

Reputation: 210880

To achieve what you want you have either to draw the scene from the back to the front. Note, the back faces of the geometry have to clear the stencil buffer, before the font faces are drawn.

Or you draw the scene twice, using Face Culling. When you draw the scene first, then cull all the front faces and when you draw it second then cull all the back faces.
Use the separated stencil test for front and back faces in that way, that th back faces clear the stencil buffer. (see glStencilFuncSeparate and glStencilOpSeparate)

The rendering process works as follows:

  • Enable the depth test

  • Disable the color buffer and enable the stencil test for setting the stencil mask

  • Draw the "holes". This causes that the stencil buffer is set to 1 at the positions of the holes.

  • Setup the stencil test for clearing the stencil buffer

  • Enable face culling for front faces

  • Draw the geometry. This causes that the stencil buffer is cleared at the positions where the "holes" covered from the back faces of the geometry.

  • Enable face culling for back faces

  • Enable the color buffer

  • Draw the geometry

To make tha algorithm work, you have to draw all your primitives int the same winding order. And you have to tell OpenGL the direction by glFrontFace. Either clockwise GL_CW or counterclockwise GL_CCW. 

glStencilMask(0xFF);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

glFrontFace(GL_CCW);  // depends on your geometry "GL_CCW" or "GL_CW"
glDisable(GL_CULL_FACE);

glEnable(GL_DEPTH_TEST);
glDepthFunc(GL_LESS); // default

glColorMask(GL_FALSE,GL_FALSE,GL_FALSE,GL_FALSE);

glEnable(GL_STENCIL_TEST);

glStencilFuncSeparate(GL_FRONT, GL_ALWAYS, 1, 255);
glStencilFuncSeparate(GL_BACK, GL_ALWAYS, 1, 255);
glStencilOpSeparate(GL_FRONT, GL_KEEP, GL_KEEP, GL_REPLACE);
glStencilOpSeparate(GL_BACK, GL_KEEP, GL_KEEP, GL_KEEP);

// draw the holes
// ....

glStencilFuncSeparate(GL_FRONT, GL_EQUAL, 0, 255);
glStencilOpSeparate(GL_FRONT, GL_KEEP, GL_KEEP, GL_KEEP);
glStencilFuncSeparate(GL_BACK, GL_ALWAYS, 0, 255);
glStencilOpSeparate(GL_BACK, GL_KEEP, GL_KEEP, GL_REPLACE);

glEnable(GL_CULL_FACE);
glCullFace(GL_FRONT);

// draw the geometry the 1. time ("draw" back faces)
// ....

glCullFace(GL_BACK);
glColorMask(GL_TRUE,GL_TRUE,GL_TRUE,GL_TRUE);

// draw the geometry the 2. time (draw front faces)
// ....

Upvotes: 3

Related Questions