CODEWITHSUNDEEP
iosopengl-esvideo-streaming
R. Jari
R. Jari

Reputation: 305

How to play Video using OpenGL in iOS?

I am trying to play a video using OpenGL ES 2.0 in iOS. I am not able to get a sample code or starting point of how to achieve this. Can anybody help me with this?

Upvotes: 2

Views: 709

Answers (1)

Matic Oblak
Matic Oblak

Reputation: 16774

What you are looking for is getting a raw buffer for the video in real time. I believe you need to look into AVFoundation and somehow extract the CVPixelBufferRef. If I remember correctly you have a few ways; one is on demand at specific time, another for processing where you will get a fast iteration of the frames in a block, and the one you probably need is to receive the frames in real time. So with this you can extract a raw RGB buffer which needs to be pushed to the texture and then drawn to the render buffer.

I suggest you create a texture once (per video) and try making it as small as possible but ensure that the video frame will fit. You might need the POT (power of two) textures so to get the texture dimension from video width you need something like:

GLInt textureWidth = 1.0f;
while(textureWidth<videoWidth) textureWidth <<= 1; // Multiplies by 2

So the texture size is expected to be larger then the video. To push the data to the texture you then need to use texture subimage glTexSubImage2D. Which expects a pointer to your raw data and rectangle parameters where to save the data which are then (0, 0, sampleWidth, sampleHeight). Also then the texture coordinates must computed so they are not in range [0, 1] but rather for x: [0, sampleWidth/textureWidth].

So then you just need to put it all together:

  • Have a system to keep generating the video raw sample buffers
  • Generate a texture to fit video size
  • On new sample update the texture using glTexSubImage2D (watch out for threads)
  • After the data is loaded into the texture draw the texture as full screen rectangle (watch out for threads)

You might need to watch out for video orientation, transformation. So if possible do test your system with a few videos that have been recorded on the device in different orientations. I think there is now a support to receive the buffers already correctly oriented. But by default the sample at least used to be "wrong"; the portrait recorded video still had the samples in landscape but a transformation matrix or orientation was given with the asset.

Upvotes: 2

Related Questions