CODEWITHSUNDEEP
node.jshtml5-canvasvideo-streaminghtml5-video
J Does
J Does

Reputation: 1

Send video played on canvas to websocket

I made multiple videos running on HTML5 canvas. Success. I made node.js websocket and its clients on browser. Success. Now I am trying to combine these projects. The idea is to send video played on canvas (admin.html) through nodejs websocket (server) to browser clients (spectactor.html). I am failed.

Please help. Here is the script. runVid called when an admin clicked a row in playlist after establishing ws connection. runVid then called draw() and sendImage. sendImage not triggered at all. What's wrong? I also put sendImage() within draw() function, but this also failed.

var port = 8080;
var ws = new WebSocket('ws://127.0.0.1:' + port,['soap', 'xmpp']);
ws.binaryType = 'arraybuffer';


  function runVid(){
    var video = document.getElementById('video');
    var canvas = document.getElementById('canvas');

    var context = canvas.getContext('2d');

    var cw = Math.floor(canvas.clientWidth);
    var ch = Math.floor(canvas.clientHeight);
    canvas.width = cw;
    canvas.height = ch;

    video.addEventListener('play', function(){
      draw(this, context, cw, ch);
      sendImage(context, cw, ch);
    }, false);
  }


  function draw(video,canvas,width,height) {
    if(video.paused || video.ended) return false;
    canvas.drawImage(video,0,0,width,height);

    setTimeout(draw,20,video,canvas,width,height);

  }

  function sendImage(context, width,height){
    var imageData = context.getImageData(0, 0, width, height);
    var canvasPixelArray = imageData.data;
    var canvasPixelLen = canvasPixelArray.length;

    for(var j = 0; j<canvasPixelLen;j++){
      byteArray[j] = canvasPixelArray[j];
    }

    ws.send(byteArray.buffer);
    console.log("byteArray sent");
  }

Upvotes: 0

Views: 2596

Answers (1)

Brad
Brad

Reputation: 163272

The idea is to send video played on canvas (admin.html) through nodejs websocket (server) to browser clients (spectactor.html).

You could use WebSocket, but why would you want to? With your approach, you must get all of that data from the canvas, buffer it, and in your case you're leaving it raw (which is huuuuuge and impractical for transmission over the internet), then have the server copy all of that data to all its listening clients in your JS, and then undo the process on the receiving end. The overhead of this is massive, and is only appropriate if you have excellent internet connections, you require lossless video, and you expect low frame rates and/or frame sizes.

What you should do instead is start with the CanvasCaptureMediaStream. Once you're capturing a Canvas as a MediaStream, you can use MediaRecorder to get an encoded version of that video, send it to your server by WebSocket, and then let the server send it to all other clients with normal HTTP!

Even better... use WebRTC. You'll sacrifice some quality for lower latency by default, but it will take your new MediaStream and send it right to clients without involving your server at all. (Unless of course you want to stream one to many... and then eventually you'll want servers involved to help handle the load.)

Either of these solutions are better than what you're attempting to do now.

Upvotes: 4

Related Questions