CODEWITHSUNDEEP
pythonopencvvideo-encodingmkv
py_ml
py_ml

Reputation: 488

How to read MKV bytes as video?

I am receiving fragment of MKV video in bytes. I need to take 1st frame of it for later processing without saving video in disk. For similar problem with image I use OpenCV or PIL and everything works fine, however, I am not able to do the same with video data. Any tips how to read video from bytes to memory object that I could use for later processing with OpenCV or some other library?

Upvotes: 5

Views: 8084

Answers (2)

FirefoxMetzger
FirefoxMetzger

Reputation: 3260

Mark Setchell got it right; though - in 2022 - you don't have to drop down all the way to get_reader anymore to do this.

Setup:

Create a test video:

ffmpeg -f lavfi -i testsrc -t 5 testsrc.mkv

create a MKV bytes string

from pathlib import Path

mkv_bytes = Path("testsrc.mkv").read_bytes()

Read the first frame:

ImageIO<2.15.0

import imageio as iio

req = iio.core.Request(mkv_bytes, "r")

# note: this is a bit of a hack, see
# https://github.com/imageio/imageio/issues/686
req._extension = ".mkv"

first_frame = iio.imread(req)

ImageIO>2.15.0 (releases this week - W6 2022)

import imageio as iio

first_frame = iio.v3.imread(mkv_bytes, format_hint=".mkv")

Upvotes: 0

Mark Setchell
Mark Setchell

Reputation: 207718

As I don't have your bytes buffer, I just created an MKV video file with ffmpeg like this:

ffmpeg -i SomeVideo.avi -f matroska -vcodec libx264 video.mkv

I then installed imageio with:

pip install imageio

Then I loaded the entire MKV video into memory so I have something that must look pretty much the same as the bytes object you receive stored in my variable content:

import imageio

# Get bytes of MKV video
with open('video.mkv', 'rb') as file: 
    content = file.read()

Now I should be set up and looking the same as you. Just for reference, the first few bytes of content look like this:

b'\x1aE\xdf\xa3\x01\x00\x00\x00\x00\x00\x00#B\x86\x81\x01B\xf7\x81\x01'

So, let's continue.

# Wrap the content in a BytesIO and get an ffmpeg reader to read it 
vid = imageio.get_reader(BytesIO(content),  'ffmpeg')

I can now print the mean of each frame like this:

for num, image in enumerate(vid.iter_data()): 
    print(image.mean())

Or get the metadata and print it like this:

metadata = vid.get_meta_data()                                                             

print(metadata)

{'plugin': 'ffmpeg',
 'nframes': 750,
 'ffmpeg_version': '4.1 built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)',
 'fps': 25.0,
 'source_size': (1280, 720),
 'size': (1280, 720),
 'duration': 30.0}

Keywords: Python, video, frame, individual frame, FFmpeg, imageio, single frame, BytesIO, bytes, MKV, Matroska.

Upvotes: 7

Related Questions