Reputation: 21917
I am streaming video over a WebSocket by sending each frame in the raw ImageData format (4 bytes per pixel in RGBA order). When I receive each frame on the client (as an ArrayBuffer
), I want to paint this image directly onto the canvas as efficiently as possible, using putImageData.
This is my current solution:
// buffer is an ArrayBuffer representing a properly-formatted image
var array = new Uint8ClampedArray(buffer);
var image = new ImageData(array, width, height);
canvas.putImageData(image, 0, 0);
But it is rather slow. My theories as to why:
the array (which is ~1MB in size) is being copied thrice, once into the Uint8ClampedArray
, once into the ImageData
, and lastly into the canvas, each frame (30 times per second).
I am using new
twice for each frame, which may be a problem for the garbage collector.
Are these theories correct and if so, what tricks can I employ to make this as fast as possible? I am willing to accept an answer that is browser-specific.
Upvotes: 7
Views: 13417
Reputation: 137171
No, both your ImageData image
and your TypedArray array
share the exact same buffer buffer
.
These are just pointers, your original buffer is never "copied".
var ctx = document.createElement('canvas').getContext('2d');
var buffer = ctx.getImageData(0,0,ctx.canvas.width, ctx.canvas.height).data.buffer;
var array = new Uint8ClampedArray(buffer);
var image = new ImageData(array, ctx.canvas.width, ctx.canvas.height);
console.log(array.buffer === buffer && image.data.buffer === buffer);
For your processing time issue, the best way would be to simply send directly the video stream to a videoElement and use drawImage
.
Upvotes: 5