Get frames from a video into a matrix

Question

I'm currently trying to implement a compression algorithm(frame prediction) for an assignment. I am not looking for thumbnail files, or even just a shell command to generate something for me. My problem is specifically integrating it with a golang program.

I just started and I'm already stuck. I'm supposed to get each frame out of a video,divide it into I P and B frames and perform inter-coding(compress the frame itself), then perform intra-coding(between the frames).

Right now I cannot even get started on the above problems, because I have no idea how to read the video as something I could use in code. Apparently, the only library I can think of is ffmpeg. FFMPEG can get separate frames, apparently even i p and b frames.

ffmpeg -i <inputfile> -vf '[in]select=eq(pict_type\,B)[out]' b.frames.mp4

But this is just another video output, that I do not know how to open. What I was thinking of was outputting frames into bitmaps(?), then reading each bitmap separately, to reconstruct three 3D matrixes, of i frames, p frames and b frames. However this seems like quite a feat. Someone, somewhere has definitely tried to parse a video into a 3D matrix and has found a better solution than what I'm thinking of.

To be concise, I have a video, I need a 3D matrix. The 3D matrix is a matrix of 2D matrixes, which represent a frame in the video. Each point in a 3D matrix is a pixel(or whatever the equivalent is in videos).

Answer 1

I/P/B frames only exist in the raw bitstream. Once the video is decoded, all frames are I frames. You probably want to use ffmprg to decode to something like yuv4mpegpipe then parse the output in your golang program.