CODEWITHSUNDEEP
javascriptcanvaswebrtcmediastreamcapturestream
Sam
Sam

Reputation: 1055

High CPU Usage from Canvas CaptureStream

Context

I've been working on building a virtual-backgrounds feature for web environments, and as part of that I'm taking in a webcam feed (MediaStreamTrack), doing some image-processing in WebGL and outputting the processed video frames to a canvas.

Problem

As soon as you call captureStream() on the canvas (or, any canvas it seems), the CPU usage starts to spike (on my Linux device, from 20% to 60%).

I've re-created a simplified version of this problem here, with a simple animation and a call to canvas.captureStream():

const canvas = document.querySelector('#canvas-target');
const animation = new CanvasAnimation(canvas.getContext('2d'));
animation.start();
// Starts an animation on canvas

function captureStream(){

   let stream  = canvas.captureStream();
   const video = document.querySelector('#stream-output');

   // Create canvas stream, feed it to video
   video.srcObject = stream;

   // CPU starts to go up here
   video.play();
}

See Codepen for the full example: https://codepen.io/sb2702/pen/MWogaeo

The main factors that seem to affect CPU usage are frame-rate and resolution - so,

  1. I assume that the browser is just doing a lot of work under the hood to re-encode the video stream into a media stream.

  2. The "quick" answer is to just restrict the framerate and/or resolution

My primary question is - am I missing something else fundamental about canvas, captureStream - etc.., and is there some much more CPU efficient way of generating a MediaStreamTrack with output from a Canvas?

Upvotes: 2

Views: 370

Answers (0)

Related Questions