CODEWITHSUNDEEP
javascriptglslwebgltextures
Dan Rossi
Dan Rossi

Reputation: 51

Scaling Video texture mixed with image texture

I've tried searching and is a trivial thing to do but can't figure it out, I looked at many answers but never answered what I need with full working examples.

I am trying to scale the video texture in a fragment shader, with an image texture in the background. So can't transform vertex. The video texture needs to be resized to 200 x 200 and positioned in the top or bottom corner.

It will use MediaStream eventually for webcam, with transparent background effect. It works with bodypix on top of an image texture, using the mix shader but need to scale the video so it doesn't stretch to the viewport.

If I try a vec2 of vec2(200,200), clamp will stretch the rest of the colours. Repeat will tile it. Neither of these is the expected result.

Jsfiddle of video render but need to resize it.

https://jsfiddle.net/danrossi303/or3dk2q4/16/

The basic fragment shader I am working with.

  precision mediump float;

 uniform sampler2D background;
 uniform sampler2D frame;

uniform float texWidth;
uniform float texHeight;


void main(void) {
vec2 texCoord = gl_FragCoord.xy / vec2(texWidth,texHeight);

vec4 texel0 = texture2D(background, texCoord);
vec4 texel1 = texture2D(frame, texCoord);
 gl_FragColor = mix(texel0, texel1, 1.);
}

Upvotes: 0

Views: 125

Answers (1)

Rabbid76
Rabbid76

Reputation: 211176

You need to scale the texture coordinates for the video:

vec2 frameuv = texCoord * vec2(texWidth, texHeight) / vec2(200.0, 200.0);
vec4 texel1 = texture2D(frame, frameuv);

Discard the frame (texel1) if frameuv.x or frameuv.y is greater than 1.0 (see step):

float w = step(frameuv.x, 1.0) * step(frameuv.y, 1.0);
gl_FragColor = mix(texel0, texel1, w);

Complete fragment shader:

precision mediump float;
  
uniform sampler2D background;
uniform sampler2D frame;

uniform float texWidth;
uniform float texHeight;

void main(void) {
    vec2 texCoord = gl_FragCoord.xy / vec2(texWidth,texHeight);
   
    vec4 texel0 = texture2D(background, texCoord);
    vec2 frameuv = texCoord * vec2(texWidth, texHeight) / vec2(200.0, 200.0);
    vec4 texel1 = texture2D(frame, frameuv);
    gl_FragColor = mix(texel0, texel1, step(frameuv.x, 1.0) * step(frameuv.y, 1.0));
}

Additionally there is a bug when you load the background texture. gl.bindTexture binds a texture object to the current texture unit. Therfor the texture unit has to be set with gl.activeTexture, before calling gl.bindTexture:

img.onload = () => {
    gl.activeTexture(gl.TEXTURE0);            // <---
    gl.bindTexture(gl.TEXTURE_2D, texture);
    initBackgroundTexture();
};
img.src = "https://videos.electroteque.org/textures/virtualbg.jpg";

Upvotes: 0

Related Questions