CODEWITHSUNDEEP
pythonopencvvideo-processing
Tim
Tim

Reputation: 57

OpenCV read video files with multiple streams/tracks

I have a video file that contains multiple streams as shown below using VLC media player: Video Information

When I try to read it using Python + OpenCV using the following code:

vidObj = cv2.VideoCapture("video.avi")
ret, frame = vidObj.read()

I can only read the first track of the video. How can I read all the video tracks at the same time?

Upvotes: 3

Views: 2083

Answers (1)

Tronic
Tronic

Reputation: 1336

As far as I could tell, OpenCV does not allow choosing video stream, so this is not possible. However, you can do it rather easily with ffmpeg command line utilities:

import numpy as np
import json
import subprocess

def videoInfo(filename):
    proc = subprocess.run([
        *"ffprobe -v quiet -print_format json -show_format -show_streams".split(),
        filename
    ], capture_output=True)
    proc.check_returncode()
    return json.loads(proc.stdout)

def readVideo(filename):
    cmd = ["ffmpeg", "-i", filename]
    streams = 0
    for stream in videoInfo(filename)["streams"]:
        index = stream["index"]
        if stream["codec_type"] == "video":
            width = stream["width"]
            height = stream["height"]
            cmd += "-map", f"0:{index}"
            streams = streams + 1
    cmd += "-f", "rawvideo", "-pix_fmt", "rgb24", "-"
    shape = np.array([streams, height, width, 3])
    with subprocess.Popen(cmd, stdout=subprocess.PIPE) as proc:
        while True:
            data = proc.stdout.read(shape.prod())  # One byte per each element
            if not data:
                return
            yield np.frombuffer(data, dtype=np.uint8).reshape(shape)

Note that the code reads all video streams and assumes that each has the same resolution. It lacks proper error handling but got the job done in my scientific project.

For example, reading stereoscopic stream:

import matplotlib.pyplot as plt

for left, right in readVideo("testvideo.mkv"):
    plt.imshow(left)
    plt.show()
    plt.imshow(right)
    plt.show()

Upvotes: 1

Related Questions