CODEWITHSUNDEEP
pythonpython-3.xffmpegopencv
Maxime Debarbat
Maxime Debarbat

Reputation: 479

Stream images from python openCV with ffmpeg

In order to try an embedded AI, I want to stream an image dataset through a rtsp stream. What I tried to do is to read one image every X seconds and send it to the stream and infere my AI on it. I tried to use this github repo :https://gist.github.com/takidog/2c981c34d5d5b41c0d712f8ef4ac60d3#file-main-py

This is what I tried so far :

import cv2
import time

import subprocess as sp 

import glob, os

__PATH = "./DATASET"


os.chdir(__PATH)
IMG_LIST = glob.glob("*.jpg")
IMG_LIST_LEN = len(IMG_LIST)
IMG_INDEX = 0
IMG_DELAY = 2

IMG_WIDTH = 1280
IMG_HEIGHT = 720

IMG_SIZE = str(IMG_WIDTH)+"x"+str(IMG_HEIGHT)
FPS = 5

RTSP_SERVER = "rtsp://localhost:31415/stream"

COMMAND = ['ffmpeg',
           '-re',
            '-s', IMG_SIZE,
            '-r', str(FPS),
            '-i', '-',
            '-bufsize', '64M',
            '-maxrate', "4M",
            '-rtsp_transport', 'tcp',
            '-muxdelay','0.1',
            RTSP_SERVER]

process = sp.Popen(COMMAND,stdin=sp.PIPE)

while(True):

    CURRENT_IMG = cv2.imread(IMG_LIST[IMG_INDEX])
    IMG_INDEX = (IMG_INDEX+1)%IMG_LIST_LEN
    while(CURRENT_IMG.shape[0]!=720): #We dump images with a bad format
        CURRENT_IMG = cv2.imread(IMG_LIST[IMG_INDEX])
        IMG_INDEX = (IMG_INDEX+1)%IMG_LIST_LEN

        
    _,FRAME = cv2.imencode('.png', CURRENT_IMG)

    process.stdin.write(FRAME.tobytes())

    time.sleep(1/FPS)

Surprise surprise this does not work and gives me this error :

Input #0, png_pipe, from 'pipe:':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: png, rgb24(pc), 1280x720, 25 fps, 25 tbr, 25 tbn, 25 tbc
[NULL @ 0x55ba3fe1b860] Unable to find a suitable output format for 'rtsp://localhost:31415/stream'
rtsp://localhost:31415/stream: Invalid argument
Traceback (most recent call last):
  File "main.py", line 47, in <module>
    process.stdin.write(FRAME.tobytes())
BrokenPipeError: [Errno 32] Broken pipe

Upvotes: 1

Views: 8098

Answers (1)

Rotem
Rotem

Reputation: 32084

Here is a reproducible sample - hoping you can copy paste and execute, but nothing is promised...

The example applies the following stages:

  • Create 10 synthetic JPEG images in ./test_dataset folder, to be used as input.
  • Execute FFplay sub-process as RTSP listener.
    When using TCP protocol we should start the TCP server first (FFplay is used as a TCP server in out case).
    We also need the receiver process, because without it, FFmpeg streamer process halts after the first frame.
  • Execute FFmpeg sub-process for RTSP streaming.
    Cyclically read JPEG image to NumPy array (in BGR color format), and write the array as raw video frame to stdin pipe.
    Note: It is more efficient to write raw video frames, than encoding each frame to PNG (as used by your reference sample).

Here is the code:

import cv2
#import time
import subprocess as sp 
import glob
import os

img_width = 1280
img_height = 720


test_path = './test_dataset'  # Folder with synthetic sample images.

os.makedirs(test_path, exist_ok=True)  # Create folder for input images.

os.chdir(test_path)

ffmpeg_cmd = 'ffmpeg'  # May use full path like: 'c:\\FFmpeg\\bin\\ffmpeg.exe'
ffplay_cmd = 'ffplay'  # May use full path like: 'c:\\FFmpeg\\bin\\ffplay.exe'


# Create 10 synthetic JPEG images for testing (image0001.jpg, image0002.jpg, ..., image0010.jpg).
sp.run([ffmpeg_cmd, '-y', '-f', 'lavfi', '-i', f'testsrc=size={img_width}x{img_height}:rate=1:duration=10', 'image%04d.jpg'])


img_list = glob.glob("*.jpg")
img_list_len = len(img_list)
img_index = 0

fps = 5

rtsp_server = 'rtsp://localhost:31415/live.stream'

# You will need to start the server up first, before the sending client (when using TCP). See: https://trac.ffmpeg.org/wiki/StreamingGuide#Pointtopointstreaming
ffplay_process = sp.Popen([ffplay_cmd, '-rtsp_flags', 'listen', rtsp_server])  # Use FFplay sub-process for receiving the RTSP video.


command = [ffmpeg_cmd,
           '-re',
           '-f', 'rawvideo',  # Apply raw video as input - it's more efficient than encoding each frame to PNG
           '-s', f'{img_width}x{img_height}',
           '-pixel_format', 'bgr24',
           '-r', f'{fps}',
           '-i', '-',
           '-pix_fmt', 'yuv420p',
           '-c:v', 'libx264',
           '-bufsize', '64M',
           '-maxrate', '4M',
           '-rtsp_transport', 'tcp',
           '-f', 'rtsp',
           #'-muxdelay', '0.1',
           rtsp_server]

process = sp.Popen(command, stdin=sp.PIPE)  # Execute FFmpeg sub-process for RTSP streaming


while True:
    current_img = cv2.imread(img_list[img_index])  # Read a JPEG image to NumPy array (in BGR color format) - assume the resolution is correct.
    img_index = (img_index+1) % img_list_len  # Cyclically repeat images

    process.stdin.write(current_img.tobytes())  # Write raw frame to stdin pipe.

    cv2.imshow('current_img', current_img)  # Show image for testing

    # time.sleep(1/FPS)
    key = cv2.waitKey(int(round(1000/fps)))  # We need to call cv2.waitKey after cv2.imshow

    if key == 27:  # Press Esc for exit
        break


process.stdin.close()  # Close stdin pipe
process.wait()  # Wait for FFmpeg sub-process to finish
ffplay_process.kill()  # Forcefully close FFplay sub-process
cv2.destroyAllWindows()  # Close OpenCV window

Upvotes: 5

Related Questions