CODEWITHSUNDEEP
gstreamer
신대규
신대규

Reputation: 21

How to combine opencv and gstreamer?

I completed video streaming jetson tx1 and My PC

now I have opencvcode of video processing

so, I want to adapt opencv code to gstreamer in jetson

Next code is PC-jetson tx1 streaming and opencv code

//jetson code//


CLIENT_IP=10.100.0.70

gst-launch-1.0 nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 \

! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \

omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! \

h264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false


// PC code//

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! xvimagesink sync=false async=false -e



opencv code


#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>

int main()
{
    cv::Mat img,img_gray;

    cv::VideoCapture input(0);

    for(;;)
    {
    if (!input.read(img))
        break;

    cv::cvtColor(img, img_gray, CV_RGB2GRAY);

    cv::imshow("img", img);
    cv::imshow("gray", img_gray);
    char c = cv::waitKey(30);

    if (c == 27)
        break;
    }
}

Upvotes: 2

Views: 1619

Answers (1)

Alper Kucukkomurler
Alper Kucukkomurler

Reputation: 1794

OpenCV support VideoCapture and VideoWriter gstreamer pipes support gstreamer pipes. You need to pass your pipe as a parameter finishing with appsink/ starting for appsrc.

char* inputPipe = "nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! videoconvert ! appsink";
char* outputPipe = "appsrc ! videoconvert ! omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! 264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false";

cv::VideoCapture input(inputPipe);
cv::VideoWriter  writer(outputPipe, 0, 25.0, cv::Size(1920,1080));
for(;;)
{
if (!input.read(img))
    break;

cv::cvtColor(img, img_gray, CV_RGB2GRAY);
//cv::imshow("img", img);
//cv::imshow("gray", img_gray);
// instead of imshow pass the frame to VideoWriter
writer.write(img_gray);

char c = cv::waitKey(30);

if (c == 27)
    break;
}

Performance tip:

OpenCv uses VideoCapture uses BGR colorspace and nvcamerasrc provides output at I420 colorspace. If you use the following pipe videoconvert element will use too much CPU.

char* inputPipe = "nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! videoconvert ! appsink";

Instead, if you convert colorspace to BGRx or RGBA with nvvidconv, the colorspace conversion will be done on specialized hardware and the CPU usage will be significantly lower 

 char* inputPipe = "nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)BGRx, framerate=(fraction)30/1' ! videoconvert ! appsink";

Upvotes: 1

Related Questions