Text in canvas being wiped out with a video canvas underneath

Question

Note: Minimal working example here. Click on change text button, and you can see the text will appear for one frame and disappear.

I wanted to add an overlay to my video, so I stacked two canvases and filled text to the transparent top canvas.

However, the text does not stick. It disappears.

To test if I was filling the text correctly, I tried using a black canvas (no video) underneath.

I just needed to fillText once and the text stayed.

However, with the video canvas underneath, the text won't stick unless I draw text in requestAnimationFrame, which I believe keeps drawing the same text in every frame, which is unnecessary.

The text canvas is supposed to be separate from the video canvas. Why is this getting wiped out without requestAnimationFrame?

How can I fix it?

Minimal working example here, but the code is also shown below.

import "./styles.css";
import { useEffect, useRef, useState } from "react";

export default function App() {
  const inputStreamRef = useRef();
  const videoRef = useRef();

  const canvasRef = useRef();
  const overlayCanvasRef = useRef();
  const [text, setText] = useState("Hello");

  function drawTextWithBackground() {
    const ctx = overlayCanvasRef.current.getContext("2d");
    ctx.clearRect(
      0,
      0,
      overlayCanvasRef.current.width,
      overlayCanvasRef.current.height
    );

    /// lets save current state as we make a lot of changes
    ctx.save();

    /// set font
    const font = "50px monospace";
    ctx.font = font;

    /// draw text from top - makes life easier at the moment
    ctx.textBaseline = "top";

    /// color for background
    ctx.fillStyle = "#FFFFFF";

    /// get width of text
    const textWidth = ctx.measureText(text).width;
    // Just note that this way of "measuring" height is not accurate.
    // You can measure height of a font by using a temporary div / span element and
    // get the calculated style from that when font and text is set for it.
    const textHeight = parseInt(font, 10);

    const textXPosition = canvasRef.current.width / 2 - textWidth / 2;
    const textYPosition = canvasRef.current.height - textHeight * 3;

    ctx.save();
    ctx.globalAlpha = 0.4;
    /// draw background rect assuming height of font
    ctx.fillRect(textXPosition, textYPosition, textWidth, textHeight);
    ctx.restore(); // this applies globalAlpha to just this?

    /// text color
    ctx.fillStyle = "#000";

    ctx.fillText(text, textXPosition, textYPosition);

    /// restore original state
    ctx.restore();
  }

  function updateCanvas() {
    const ctx = canvasRef.current.getContext("2d");

    ctx.drawImage(
      videoRef.current,
      0,
      0,
      videoRef.current.videoWidth,
      videoRef.current.videoHeight
    );

    // QUESTION: Now the text won't stick unless I uncomment this below,
    // which runs drawTextWithBackground in requestAnimationFrame.
    // Previosuly, with just a black canvas underneath, I just needed to fillText once
    // and the text stayed.
    // The text canvas is supposed to be separate. Why is this getting wiped out without requestAnimationFrame?
    // drawTextWithBackground();

    requestAnimationFrame(updateCanvas);
  }

  const CAMERA_CONSTRAINTS = {
    audio: true,
    video: {
      // the best (4k) resolution from camera
      width: 4096,
      height: 2160
    }
  };

  const enableCamera = async () => {
    inputStreamRef.current = await navigator.mediaDevices.getUserMedia(
      CAMERA_CONSTRAINTS
    );

    videoRef.current.srcObject = inputStreamRef.current;

    await videoRef.current.play();

    // We need to set the canvas height/width to match the video element.
    canvasRef.current.height = videoRef.current.videoHeight;
    canvasRef.current.width = videoRef.current.videoWidth;
    overlayCanvasRef.current.height = videoRef.current.videoHeight;
    overlayCanvasRef.current.width = videoRef.current.videoWidth;

    requestAnimationFrame(updateCanvas);
  };
  useEffect(() => {
    enableCamera();
  });

  return (
    
      
  );
}

Blindman67 · Accepted Answer

Very simple fix

The canvas is being cleared due to how React updates state.

The fix

The simplest fix is to use just the one canvas and draw the text onto the canvas used to display the video.

From the working example change the two functions updateCanvas and drawTextWithBackground as follows.

Note I have removed all the state saving in drawTextWithBackground as the canvas state is lost due to react anyway so why use the very expensive state stack functions.

  function drawTextWithBackground(ctx) {
    const can = ctx.canvas;
    const textH = 50;  
    ctx.font = textH + "px monospace";
    ctx.textBaseline = "top";
    ctx.textAlign = "center";
    ctx.fillStyle = "#FFFFFF66";     // alpha 0.4 as part of fillstyle
    ctx.fillStyle = "#000";
    const [W, X, Y] = [ctx.measureText(text).width, can.width, can.height - textH * 3];
    ctx.fillRect(X -  W / 2, Y, W, textH);
    ctx.fillText(text, X, Y);
  }

  function updateCanvas() {
    const vid = videoRef.current;
    const ctx = canvasRef.current.getContext("2d");
    ctx.drawImage(vid, 0, 0, vid.videoWidth,  vid.videoHeight);
    drawTextWithBackground(ctx);
    requestAnimationFrame(updateCanvas);
  }

Text in canvas being wiped out with a video canvas underneath

Answers (2)

Very simple fix

The fix

Related Questions