CODEWITHSUNDEEP
python-3.xffmpegpyffmpeg
Hattori
Hattori

Reputation: 371

read one 16bits video frame at a time with ffmpeg-python

i have a video encoded in 16bits. I cannot use opencv as it does not support 16bits video. So i tried ffmpeg-python. The video is around 1Go and each frame is 1M pixels. So, i would like to extract one frame at a time to process it, otherwise, it won't fit in memory. I mean it can, but i want to limit the amount of RAM used by my process.

I looked at examples here. I guess what is closer than what i want is the following, but instead of filling the numpy array with the whole video, i want to read one frame at a time. Or maybe it is possible to give ffmpeg a callback so that is returns a numpy array with all the frames already processed ?

out, _ = (
    ffmpeg
    .input('test.16b.raw.avi')
    .output('pipe:', format='rawvideo', pix_fmt='gray16le')
    .run(capture_stdout=True)
)
video = (
    np
    .frombuffer(out, np.uint16)
    .reshape(-1, height, width, 2)
)

ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.1 (GCC) 20200726
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libgsm --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, avi, from 'test.16b.raw.avi':
  Metadata:
    encoder         : Lavf58.45.100
  Duration: 00:00:14.23, start: 0.000000, bitrate: 46453 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), gray16le, 640x400, 46553 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'pipe:':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: rawvideo (Y1[0][16] / 0x10003159), gray16le, 640x400, q=2-31, 122880 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.91.100 rawvideo
frame=  427 fps=103 q=-0.0 Lsize=  213500kB time=00:00:14.23 bitrate=122880.0kbits/s speed=3.44x    
video:213500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

Upvotes: 0

Views: 1787

Answers (1)

pncanas333
pncanas333

Reputation: 106

You can do this:

def extract_frame(input_vid, frame_num):
    out, _ = (
        ffmpeg
        .input(input_vid)
        .filter_('select', 'gte(n,{})'.format(frame_num))
        .output('pipe:', format='rawvideo', pix_fmt='gray16le', vframes=1)
        .run(capture_stdout=True, capture_stderr=True)
    )
    return np.frombuffer(out, np.uint16).reshape(-1, height, width, 2)

for i in range($totalFrameNumber):
    frame = extract_frame($videoPath,i)

Change $totalFrameNumber and $videoPath.

Upvotes: 1

Related Questions