How can I render a textured quad so that I fade different corners?

Question

I'm drawing textured quads to the screen in a 2D environment. The quads are used as a tile-map. In order to "blend" some of the tiles together I had the idea like:

1. A single "grass" tile drawn on top of dirt would render it as a faded circle of grass; faded from probably the quarter point.
2. If there was a larger area of grass tiles, then the edges would gradually fade from the quarter point that is on the edge of the grass.

So if the entire left-edge of the quad was to be faded, it would have 0 opacity at the left-edge, and then full opacity at one quarter of the width of the quad. Right edge fade would have full opacity at the three-quarters width, and fade down to 0 opacity at the right-most edge.

I figured that setting 4 corners as "on" or "off" would be enough to have the fragment shader work it out. However, I can't work it out.

If corner0 were 0 the result should be something like this for the quad:

If both corner0 and corner1 were 0 then it would look like this:

This is what I have so far:

#version 330

layout(location=0) in vec3 inVertexPosition;
layout(location=1) in vec2 inTexelCoords;
layout(location=2) in vec2 inElementPosition;
layout(location=3) in vec2 inElementSize;
layout(location=4) in uint inCorner0;
layout(location=5) in uint inCorner1;
layout(location=6) in uint inCorner2;
layout(location=7) in uint inCorner3;

smooth out vec2 texelCoords;
flat out vec2 elementPosition;
flat out vec2 elementSize;
flat out uint corner0;
flat out uint corner1;
flat out uint corner2;
flat out uint corner3;

void main()
{
    gl_Position = vec4(inVertexPosition.x,
                       -inVertexPosition.y,
                       inVertexPosition.z, 1.0);

    texelCoords = vec2(inTexelCoords.x,1-inTexelCoords.y);

    elementPosition.x = (inElementPosition.x + 1.0) / 2.0;
    elementPosition.y = -((inElementPosition.y + 1.0) / 2.0);
    elementSize.x = (inElementSize.x) / 2.0;
    elementSize.y = -((inElementSize.y) / 2.0);
    corner0 = inCorner0;
    corner1 = inCorner1;
    corner2 = inCorner2;
    corner3 = inCorner3;
}

The element position is provided in the range of [-1,1], the corner variables are all either 0 or 1. These are provided on an instance basis, whereas the vertex position and texelcoords are provided per-vertex. The vertex y-coord is inverted because I work in reverse and just flip it here for ease. ElementSize is on the scale of [0,2], so I'm just converting it to [0,1] range.

The UV coords could be any values, not neccessarily [0,1].

Here's the frag shader

#version 330

precision highp float;

layout(location=0) out vec4 frag_colour;

smooth in vec2 texelCoords;
flat in vec2 elementPosition;
flat in vec2 elementSize;
flat in uint corner0;
flat in uint corner1;
flat in uint corner2;
flat in uint corner3;

uniform sampler2D uTexture;

const vec2 uScreenDimensions = vec2(600,600);

void main()
{
    vec2 uv = texelCoords;

    vec4 c = texture(uTexture,uv);

    frag_colour = c;

    vec2 fragPos = gl_FragCoord.xy / uScreenDimensions;

    // What can I do using the fragPos, elementPos??
}

Basically, I'm not sure what I can do using the fragPos and elementPosition to fade pixels toward a corner if that corner is 0 instead of 1. I kind of understand that it should be based on the distance of the frag from the corner position... but I can't work it out. I added elementSize because I think it's needed to determine how far from the corner the given frag is...

Answer 1

To achieve a fading effect, you have to use Blending. YOu have to set the alpha channel of the fragment color dependent on a scale:

frag_colour = vec4(c.rgb, c.a * scale);

scale has to be computed dependent on the texture coordinates (uv). If a coordinate is in range [0.0, 0.25] or [0.75, 1.0] then the texture has to be faded dependent on the corresponding cornerX variable. In the following the variables uv is assumed to be a 2 dimensional vector, in range [0, 1].

Compute a linear gradients for the left, right, bottom and top side, dependent on uv:

float gradL = min(1.0, uv.x * 4.0);
float gradR = min(1.0, (1.0 - uv.x) * 4.0);
float gradT = min(1.0, uv.y * 4.0);
float gradB = min(1.0, (1.0 - uv.y) * 4.0);

Or compute Hermite gradients by using smoothstep:

float gradL = smoothstep(0.0, 0.25, uv.x);
float gradR = 1.0 - smoothstep(0.75, 1.0, uv.x);
float gradT = smoothstep(0.0, 0.25, uv.y);
float gradB = 1.0 - smoothstep(0.75, 1.0, uv.y);

Compute the fade factor for the 4 corners and the 4 sides dependent on gradL, gradR, gradT, gradB and the corresponding cornerX variable. Finally compute the maximum fade factor:

float fade0 = float(corner0) * max(0.0, 1.0 - dot(vec2(0.707), vec2(gradL, gradT)));
float fade1 = float(corner1) * max(0.0, 1.0 - dot(vec2(0.707), vec2(gradL, gradB)));
float fade2 = float(corner2) * max(0.0, 1.0 - dot(vec2(0.707), vec2(gradR, gradB)));
float fade3 = float(corner3) * max(0.0, 1.0 - dot(vec2(0.707), vec2(gradR, gradT)));

float fadeL = float(corner0) * float(corner1) * (1.0 - gradL);
float fadeB = float(corner1) * float(corner2) * (1.0 - gradB);
float fadeR = float(corner2) * float(corner3) * (1.0 - gradR);
float fadeT = float(corner3) * float(corner0) * (1.0 - gradT);

float fade = max(
    max(max(fade0, fade1), max(fade2, fade3)),
    max(max(fadeL, fadeR), max(fadeB, fadeT)));

At the end compute the scale and set the fragment color:

float scale = 1.0 - fade; 
frag_colour = vec4(c.rgb, c.a * scale);