CODEWITHSUNDEEP
videostreamingplaybackrate
John Smith
John Smith

Reputation: 61

Video player playback rate

I understand that video players in Android and iOS devices decode streaming content at video encoding rate. Is there any relation between the video encoding and playback rate? That is, does normal playback (without using settings such as x2,x0.5) also occur at the encoding rate?

If not, then how is playback rate defined?

Thanks! Regards, John

Upvotes: 1

Views: 977

Answers (1)

Mick
Mick

Reputation: 25491

I'm not entirely sure what you mean by the encoding rate in this context, but it probably will help to understand that most video encoding formats include two time concepts:

  • decoding time
  • presentation time

If you look at some discussions you will often see PTS and DTS being mentioned - these are the decode time stamp and the presentation time stamp for a frame.

The presentation time stamp is the simple one to understand - this is the time the frame needs to be displayed so that the viewer sees the video as intended.

The reason there is a decode time stamp also is because many encoders compress frames by taking reference frames and describing other frames by how they change with respect to these frames. This can save considerable bandwidth for frames where the background remains fairly constant, as you can imagine.

The reference frames can be both forward and backwards - in other words a given frame 'X' might be described by referring to its deltas from frame 'X-3' and 'X+3'.

This means that the video player needs to have frame 'X+3' when it wants to present frame X, hence the time it needs to decode frame 'X+3' is in advance of the time it needs to present it.

There is a good description here in a classic ffmpeg tutorial: http://dranger.com/ffmpeg/tutorial05.html

The diagram from the tutorial helps to explain it pictorially - the P frames are the reference frames and the B frames are the ones described by their difference to the reference frame:

  PTS:    1 4 2 3
  DTS:    1 2 3 4
  Stream: I P B B

The I frame is a full image, the P frame is a delta based on the previous I frame and the B frames are deltas which depend on information in the previous and following frames.

The viewer will (hopefully!) see the video in the order of the PTS times: i.e. 1, 2, 3, 4 in the example above, but the frames need to be decoded in the order of the DTS and it also makes sense to stream them in this order.

Upvotes: 1

Related Questions