CODEWITHSUNDEEP
javascripthtmlcanvasvideoweb-mediarecorder
JarsOfJam-Scheduler
JarsOfJam-Scheduler

Reputation: 3169

Web API MediaRecorder: How do I force record at 60fps?

Summary

  1. The API I speak about

  2. The context

  3. The problem: Actual result + Expected result

  4. What I already have tried

  5. Clues to help you to help me

  6. Minimal and Testable Example: Prerequisites and Steps to follow + Sources

The API I speak about

https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder

The context

I have a video element that I load & play several times with different src. Between two different videos, a transition occures and each video appears with a fade in. When a video is playing, I draw it in a canvas with the transition and the fade in.

Everything I draw in the canvas is recorded using the Web API MediaRecorder. Then I download the WEBM file corresponding to this recording.

The problem

Actual result

The resulting WEBM is jerky. When I drew in the canvas, the drawing was fluid. But the WEBM resulting from the recording of the canvas drawing is jerky.

Expected result

The WEBM resulting from the recording of the canvas drawing must be as fluid as the canvas drawing itself. If it's impossible to have exactly this result, then: the WEBM must be approximately as fluid as the canvas drawing itself in a way that we quite can't say it's jerky.

What I have already tried

  1. First, I've tried to set a time slice of 1ms, of 16ms (corresponding to 60fps), of 100ms, then 1000, then 10000, etc. to the method start of the Media Recorder, but it didn't work.

  2. Second, I've tried to call requestData every 16ms (in a simple JS timeoutlistener executed every 16ms), but it didn't work.

Clues to help you to help me

Perhaps I'm wrong but it would be possible that a material limit exist on my computer (I have a HP Spectre x360 that doesn't have graphic card, but just a little graphics chip), or a logical limit on my Chromium browser (I have Version 81.0.4044.138 on my Ubuntu 16.04 LTS).

If you can confirm this it would solve this question. If you can explain how to deal with this problem it would be very good (alternative solution to the Web API Media Recorder, or something else).

Minimal and Testable Example

Prerequisites and Steps to follow

  1. Have at least 2 WEBM videos (which will be the input videos, remember: we want to output a new video that contains the canvas drawing, which itself contains these two input videos plus transitions and colors effects etc.)

  2. Have a HTTP server and a file called "index.html" that you will open with Chromium v.81 e.g.. Copy/paste the following sources inside ; click on the "Start" button (you won't need to click on the "Stop" button) ; it will draw both videos on the canvas, with the transitions & colors effects, it will record the canvas drawing, and a "Download the final output video" will appear, allowing you to download the output video. You will see that it's jerky.

  3. In the following sources, copy/paste the paths of your videos in the JS array called videos.

Sources

<html>
<head>
    <title>Creating Final Video</title>
</head>

<body>
<button id="start">Start</button>
<button id="stop_recording">Stop recording</button>

<video id="video" width="320" height="240" controls>
    <source type="video/mp4">
</video>
<canvas id="canvas" width=3200 height=1608></canvas>

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>

<script>
var videos = [];  // Populated by Selenium
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
var video = document.querySelector("video");

startRecording();
$('#start').click(function() {
    showSomeMedia(0, 0);
});

function showSomeMedia(videos_counter) {
    resetVideo();

    if(videos_counter == videos.length) {
            $('#stop_recording').click();
            return;
    } else {
            setVideoSrc(videos_counter);
            setVideoListener();
            videos_counter++;
    }

    video.addEventListener('ended', () => {
         showSomeMedia(videos_counter);
    }, false);
}

function resetVideo() {
    var clone = video.cloneNode(true);
    video.remove();
    video = clone;
}

function setVideoSrc(videos_counter) {
    video.setAttribute("src", videos[videos_counter]);
    video.load();
    video.play();
}

function setVideoListener() {
    var alpha = 0.1;
    video.addEventListener('playing', () => {
      function step() {
        if(alpha < 1) {
            ctx.globalAlpha = alpha;
        }

        ctx.drawImage(video, 0, 0, canvas.width, canvas.height)
        requestAnimationFrame(step)

        if(alpha < 1) {
            alpha += 0.00001;
        }
      }
      requestAnimationFrame(step);
    }, false)
}

function startRecording() {
  const chunks = [];
  const stream = canvas.captureStream();
  const rec = new MediaRecorder(stream);
  rec.ondataavailable = e => chunks.push(e.data);

  $('#stop_recording').click(function() {
        rec.stop();
  });
  rec.onstop = e => exportVid(new Blob(chunks, {type: 'video/webm'}));

   window.setTimeout(function() {
            rec.requestData();
   }, 1);
  rec.start();
}

function exportVid(blob) {
  const vid = document.createElement('video');
  vid.src = URL.createObjectURL(blob);
  vid.controls = true;
  document.body.appendChild(vid);
  const a = document.createElement('a');
  a.download = 'my_final_output_video.webm';
  a.href = vid.src;
  a.textContent = 'Download the final output video';
  document.body.appendChild(a);
}
</script>

</body>

</html>

Upvotes: 4

Views: 653

Answers (1)

JarsOfJam-Scheduler
JarsOfJam-Scheduler

Reputation: 3169

The problem is solved by using my gaming laptop (RoG), wich has a good graphic card, able to record at higher frequency than my Spectre x360 development laptop.

Upvotes: 1

Related Questions