CODEWITHSUNDEEP
opengl-esthree.jsshader
Robert Rodriguez
Robert Rodriguez

Reputation: 191

Masking fragments generated by gl_PointSize

I'm trying to mask out some particles made with points so that they don't render on some areas of my scene but I can't seem to be able to get the fragment shader right, this is what I have so far:

uniform sampler2D tDiffuse;
uniform sampler2D tStencil;

varying vec2 vTexCoord;

void main() {
    vec4 diffuseColor = texture2D( tDiffuse, vTexCoord );
    vec4 stencil = texture2D(tStencil, gl_FragCoord.xy);
    gl_FragColor = stencil * diffuseColor;
}

Vertex shader:

attribute float size;
varying vec2 vTexCoord;

void main() {
    vec3 vertex = position;
    vTexCoord = vTexCoordRef;
    gl_PointSize = size;
    vec4 mvPosition = modelViewMatrix * vec4( vertex, 1.0 );
    gl_Position = projectionMatrix * mvPosition;
}

Shader output:

Shader output

Stencil texture:

Stencil texture

Desired output made in paint:

enter image description here

Upvotes: 0

Views: 111

Answers (1)

gromiczek
gromiczek

Reputation: 3020

I believe the part you are missing is the texture coordinate for tStencil. If that texture is the same size as tDiffuse, then you can use the same vTexCoord for both.

uniform sampler2D tDiffuse;
uniform sampler2D tStencil;

varying vec2 vTexCoord;
varying vec2 vTexCoord2; // a texture coord for the stencil

void main() {
   vec4 diffuseColor = texture2D( tDiffuse, vTexCoord );
   vec4 stencil = texture2D(tStencil, vTexCoord2);
   gl_FragColor = stencil * diffuseColor;
}

If all is well there, you may want to look at your alpha values. If your point cloud is composed of the white points in the middle of the image with black clear color all around, then you want your stencil to render as an alpha channel. Do a conditional so that wherever your stencil image texels occur (whatever color or alpha they are), set the alpha of your gl_FragColor to 0.0. That should punch through the white of your point cloud to reveal the black clear color.

gl_FragColor = vec4( (gl_FragColor * diffuseColor).rgb, alphaBasedOnStencil );

Upvotes: 1

Related Questions