Controlling a HTML5 Video via offset-position ends in a performance fiasko

Question

I am sitting on one of those bigass scrolling-sites with video Background. The Video performs horribly bad in chrome, safari and firefox. I've put the project in a codepen for you to see the problem.

I tried exchanging the video for another one, I tried a 60fps version and a 24fps version.

I think the Problem could be with this line (88 in JS):

$j('#v0')[0].currentTime = myPageoffset/500;

It is from this example: http://codepen.io/anon/pen/hfaLE Could someone explain to me why he chose to divide the position by 400?

What do I do wrong?

Here is my pen: http://codepen.io/anon/pen/yYaKLw

Answer 1

Preamble

Based solely on my observation, this is not meant to play the video, but to use it as a dynamic background that is updated when the page is scrolled.

Playing a video by seeking through it with scroll events can not be expected to be smooth.

Unless the issue is that it jumps too quickly through the frames. Then the rest of the answer should clarify why that is.

On a sidenote, I'm running Firefox 40.0.3 and the second codepen shows no video, with the console returning this:

Specified "type" attribute of "video/webm; codecs=vp8, vorbis" is not supported. Load of media resource http://randlerbank.de/video_1_3s_24fps_720p.webm failed.
Specified "type" attribute of "video/ogg; codecs=theora, vorbis" is not supported. Load of media resource http://randlerbank.de/video_1_3s_24fps_720p.ogv failed.
Specified "type" attribute of "video/mp4; codecs=avc1.42E01E, mp4a.40.2" is not supported. Load of media resource http://randlerbank.de/video_1_3s_24fps_720p.mp4 failed.

Removing the type attribute of the source element in the video element solves this.

Performance

This might really just be a matter of encoding. The ogv version of the video seems to run a little better than the webm, which itself seems to run better than the mp4.

As said in the preamble, playing a video via seeking is not a good idea. Depending on how the player fetches frames, this could introduce some overhead and glitches, especially on videos with large key frame intervals.

I won't go into the details of video compression, but a basic example is this:

[I][P][P][P][P]

• [I] is a key frame, it has all the data necessary to show the picture, like a single image would, such as a jpeg or png.

• [P] is an inter frame and depends on the previous frame. It only contains partial data about the picture. That dependency goes all the way up to the previous key frame.

  If you were to try to access the 6th frame of this example directly, you would have to fetch the nearest previous key frame, then every inter frame after that until reaching the one you want, while reconstructing the picture at every frame, essentially playing all 6 frames in memory to show only the last one.

I won't go into B frames, that would make things even worse.

See Video compression picture types on Wikipedia for details.

This mechanism is an essential part of what makes high compression of digital video possible, at the obvious drawback of seeking individual frames, which requires to decode an increasing number of frames the further from a key frame the requested frame is.

On regular playback, the last frame is presumably kept in memory for reference but seeking is not playback, it's very well possible that the player tries to reconstruct the whole chain.

Depending on the key frame interval, the type of compression, the level of compression and the power of the processor, this process can range from being invisible to appearing as a slideshow (with a forecast of macro blocks).

If you want to minimize this issue, you would ideally use a video that contains only keyframes, at the price of a higher bitrate.

Or you could reverse the logic by setting the scroll position depending on the current timestamp of the video.

Mitigate slightly

Jquery has an option called jQuery.fx.interval that you could set to the millisecond equivalent of your video framerate to avoid unnecessary function calls in your scrolling animation.

jQuery.fx.interval = 1000 / framerate;

jQuery.fx.interval

Returns: Number

Description: The rate (in milliseconds) at which animations fire.

version added: 1.4.3

This property can be manipulated to adjust the number of frames per second at which animations will run. The default is 13 milliseconds. Making this a lower number could make the animations run smoother in faster browsers (such as Chrome) but there may be performance and CPU implications of doing so.

Since jQuery uses one global interval, no animation should be running or all animations should stop for the change of this property to take effect.

But this will hardly solve the fundamental issue.

The analysis

MediaInfo has this to say about that first video file:

Format                : WebM
Format version        : Version 2
File size             : 1.93 MiB
Duration              : 34s 178ms
Overall bit rate mode : Variable
Overall bit rate      : 473 Kbps

Note the duration of 34s 178ms.

The CSS in that codepen sets the height of the document at 13500px:

#set-height {
    display: block;
    height: 13500px;
}

Let's do some simple arithmetic:

13500 / 400 = 33.75

Well, 33.75 looks awfully close to the video duration of 34s 178ms.

My conclusion is that the height of the document is meant to match the duration of the video, like a kind of vertical seekbar.

Meanwhile, your codepen uses a video with a duration of 2s 850ms, your virtualScrollbar has a height of 3000px and your divider is 500, therefore:

3000 / 500 = 6

Your vertical seekbar is meant for a duration of 6 seconds, which is why your video reaches its end when the scrollbar is around half the height of the document.

As to why 400, this could be considered a sensitivity setting. Scrolling through a document with the mouse wheel jumps a lot more than 1 pixel. A higher sensitivity (and let's not forget to adapt the page height as well), will make smaller steps in the video.

Example with a 60 seconds long, 24fps video (1440 frames):

• height:6000px with sensitivity 100 = 60 seconds

  • scroll 100px: 100 / 100 = 1, jumps 1 seconds, 24 frames
  • scroll 200px: 200 / 100 = 2, jumps 2 seconds, 48 frames
  • scroll 400px: 400 / 100 = 4, jumps 4 seconds, 96 frames

• height:24000px with sensitivity 400 = 60 seconds

  • scroll 100px: 100 / 400 = 0.25, jumps 0.25 second, 6 frames
  • scroll 200px: 200 / 400 = 0.5, jumps 0.5 second, 12 frames
  • scroll 400px: 400 / 400 = 1, jumps 1 second, 24 frames