how to make cv2.videoCapture.read() faster?

Question

My question :

I was working on my computer vision project. I use opencv(4.1.2) and python to implement it.

I need a faster way to pass the reading frame into image processing on my Computer(Ubuntu 18.04 8 cores i7 3.00GHz Memory 32GB). the cv2.VideoCapture.read() read frame (frame size : 720x1280) will take about 120~140ms. which is too slow. my processing module take about 40ms per run. And we desire 25~30 FPS.

here is my demo code so far:

import cv2
from collections import deque
from time import sleep, time
import threading


class camCapture:
    def __init__(self, camID, buffer_size):
        self.Frame = deque(maxlen=buffer_size)
        self.status = False
        self.isstop = False
        self.capture = cv2.VideoCapture(camID)


    def start(self):
        print('camera started!')
        t1 = threading.Thread(target=self.queryframe, daemon=True, args=())
        t1.start()

    def stop(self):
        self.isstop = True
        print('camera stopped!')

    def getframe(self):
        print('current buffers : ', len(self.Frame))
        return self.Frame.popleft()

    def queryframe(self):
        while (not self.isstop):
            start = time()
            self.status, tmp = self.capture.read()
            print('read frame processed : ', (time() - start) *1000, 'ms')
            self.Frame.append(tmp)

        self.capture.release()

cam = camCapture(camID=0, buffer_size=50)
W, H = 1280, 720
cam.capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
cam.capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)


# start the reading frame thread
cam.start()

# filling frames
sleep(5)

while True:
  frame = cam.getframe() # numpy array shape (720, 1280, 3)

  cv2.imshow('video',frame)
  sleep( 40 / 1000) # mimic the processing time

  if cv2.waitKey(1) == 27:
        cv2.destroyAllWindows()
        cam.stop()
        break

What I tried :

1. multiThread - one thread just reading the frame, the other do the image processing things. It's NOT what I want. because I could set a buffer deque saving 50 frames for example. but the frame-reading thread worked with the speed ~ frame/130ms. my image processing thread worked with the speed ~ frame/40ms. then the deque just running out. so I've been tried the solution. but not what I need.

2. this topic is the discussion I found out which is most closest to my question. but unfortunately, I tried the accepted solutions (both of two below the discussion).

One of the solution (6 six thumbs up) point out that he could reading and saving 100 frames at 1 sec intervals on his mac. why my machine cannot handle the frame reading work? Do I missing something? my installation used conda and pip conda install -c conda-forge opencv, pip install opencv-python(yes, I tried both.)

The other of the solution(1 thumb up) using ffmpeg solution. but it seem's work with video file but not camera device?

3. adjust c2.waitKey() : the parameter just controls the frequency when video display. not a solution.

Then, I know I just need some keywords to follow.

code above is my demo code so far, I want some method or guide to make me videoCapture.read() faster. maybe a way to use multithread inside videoCapture object or other camera reading module.

Any suggestions?

Answer 1

This is the best solution ever

From This

capture = cv2.VideoCapture(0)

to

capture = cv2.VideoCapture(0, cv2.CAP_DSHOW)

I searched for much time but not found the best solution but this solution works faster open cv2 video capture try now hope useful for you. just use cv2.VideoCapture(0, cv2.CAP_DSHOW)

Answer 2

In my case, I have set camera like this (in c++ code):

// cam is cv::VideoCapture object
cam.set(cv::CAP_PROP_BUFFERSIZE, 1);
cam.set(cv::CAP_PROP_FPS, 20); // set fps before set fourcc
cam.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('M', 'J', 'P', 'G'));

CAP_PROP_BUFFERSIZE doesn't matter, but CAP_PROP_FPS must be set before CAP_PROP_FOURCC.
By this setup, my camera fps increases from 5 to 20.

Answer 3

This comes a bit late, but I was wondering this with my Logitech C920 HD Pro USB-camera on Ubuntu 20.04 and OpenCV. I tried to command the capture session to run Full HD @ 30 FPS but the FPS was fluctuating between 4-5 FPS.

The capture format for my camera defaulted as "YUYV 4:2:2". No matter how I tried to alter the video capture settings, OpenCV did not magically change the video format to match e.g. the desired FPS setting.

When I listed the video formats for my Logitech C920, it revealed:

ubuntu:~$ v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'YUYV' (YUYV 4:2:2)
<clip>
        Size: Discrete 1600x896
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 2304x1296
            Interval: Discrete 0.500s (2.000 fps)
    [1]: 'MJPG' (Motion-JPEG, compressed)
<clip>
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

The solution was to manually command the OpenCV capture device to use the compressed 'MJPG' format:

import numpy as np
import cv2

capture = cv2.VideoCapture(0)
W, H = 1920, 1080
capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'))
capture.set(cv2.CAP_PROP_FPS, 30)

Answer 4

Long.

I checked using the following settings and somehow if you increase the frame size opencv will reduce the total fps. Maybe this is a bug.

1920x1080 : FPS: 5.0, Width: 1920.0, Height: 1080.0 , delay = 150ms

https://i.sstatic.net/1ZQ7v.jpg

1280x720 : FPS: 10.0, Width: 1280.0, Height: 720.0, delay = 60ms

https://i.sstatic.net/tDV3R.jpg

640x480 : FPS: 30.0, Width: 640.0, Height: 480.0, delay = 5ms

https://i.sstatic.net/cfMDc.jpg

But by using other applications such as cheese, we still get a full 30fps at 1920x1080 resolution. Please be notified that set the CAP_PROP_BUFFERSIZE value won't help either.

So a question will arise: "How can we overcome this?". At this stage, you only have 2 choices :

1. Reduce the frame resolution into 640x480
2. Use other framework

Hope this help.