CODEWITHSUNDEEP
three.jshtml5-videogpu
Davey
Davey

Reputation: 11

Three.js video textures and the GPU

I can load HTML 5 video elements on a page, map them to THREE.Texture's, and display video textures on cubes with MeshBasicMaterial in 3d. (All the videos are set to autoplay and loop. ) Each animate/render cycle, for each texture I check :

if (video.HAVE_ENOUGH_DATA)
   texture.needsUpdate = true

As I load more and more videos to play in this manner the framerate continues to drop. What if anything can I do to improve performance? Do I need to load the video as a DOM element? Is there anyway I could leverage fragment shaders/GPU to parallelize?

Upvotes: 0

Views: 1466

Answers (1)

brianchirls
brianchirls

Reputation: 7853

Here are a few things I'd try...

  1. Load the video elements in Javascript, unattached to the DOM. It may not make too much of a difference if the videos are display: none, but it couldn't hurt.

  2. Make the videos smaller. In this case, having good compression is not enough, as the bottleneck is likely going to be copying the uncompressed video frame to GPU memory.

  3. On each frame, check to see if .currentTime on each video has changed. It may not be necessary to update the texture if you're between frames. You may end up being off by one frame, since it sometimes takes a couple milliseconds to update the video frame even after currentTime is updated. But this is usually only noticeable if the video is paused. Usually, I have a special case where, if the video is paused, I keep updating on every frame for another half second or so.

  4. You might try setting your video dimensions to powers of two. This probably depends on the GPU hardware, but it couldn't hurt to test it.

  5. If the videos are all the same length and play on the same timeline, it might help to combine them into one, load it all in a single texture and set the UV coordinates accordingly.

More generally, you should probably be concerned with:

  • copying the video frame into GPU memory. No more often and no bigger than necessary.
  • reducing the browser's overhead in managing the video files

Upvotes: 4

Related Questions