CODEWITHSUNDEEP
video-streaminggstreamerrtsp
Ahmed Alnaqbi
Ahmed Alnaqbi

Reputation: 11

Video streaming, RTSP and GStream

I am trying to understand the concept of video streaming and when I googled it there are different terms that I came across, According to what I understood (correct me if I am wrong), when we want to stream a video, first of all we need a camera connected to jeston or raspberry pi, then we need a encoder then a link then a decoder. What is the role of decoder and encoder? Is RTSP and Gstreamer encoder and decoder? And why do we need a Clint and a server?

Upvotes: 1

Views: 2300

Answers (1)

Daniel
Daniel

Reputation: 1418

Let's get the terminology straight.

Gstreamer is...

"an extremely powerful and versatile framework for creating streaming media applications."

In simple words, Gstreamer allows you to create very complex media piplines and run them in your terminal, or using the GStramer API (which gives you more capabilities).
An example of such a pipeline is:
grabbing frames from a camera => reducing the framerate => cropping => resizing => encoding to h.264 => storing as a local .mp4 file

Such a pipeline can be created in the terminal with:

gst-launch-1.0 v4l2src device="/dev/video0" ! \
  videorate ! video/x-raw,framerate=10/1 ! \
  videocrop top=6 left=302 right=198 bottom=9 ! \
  videoscale ! video/x-raw,width=640,height=480 ! \
  x264enc ! h264parse ! matroskamux ! \
  filesink location=/tmp/video.mp4 -e

Which takes me to your next question: The pipeline element x264enc is an H.264 encoder.
An encoder is a complex mechanism that compresses your video by (usually) significant factors while slightly reducing the image quality.
It takes advantage of the fact that in most videos there is not much change between consecutive frames, and not too many changes in a single frame between adjacent pixeles (think of a video with a white wall in the background). The encoder therefore only encodes the changes! Which results in a very small file in case you're storing it to disk, or reduced bandwidths if you are transmitting it over the network.

A decoder does the reverese. It can read an encoded file and decode these (above-mentioned) "changes" into a video stream (similar to the original one). In simple words, it uncompresses the video.

RTSP is a streaming protocol (one of many) which allows sending commands like play/pause and receiving back a video stream.

So that means that if you wish to stream video from your device, you'll need an RTSP server running on it.

Luckily, GStreamer has an RTSP server you can easily build. It supports launching gstreamer video pipelines, which will eventually stream the video to the clients.

On my Ubuntu 18.04 the following script compiles a toy example of a server (test-launch.c):

wget https://raw.githubusercontent.com/GStreamer/gst-rtsp-server/1.14/examples/test-launch.c
sudo apt-get update && sudo apt-get install libgstrtspserver-1.0-0 libgstrtspserver-1.0-dev
gcc -o test-launch  ./test_launch.c  `pkg-config --cflags --libs gstreamer-rtsp-server-1.0`
  • This assumes you already have gstreamer (and plugins) installed.
  • I intentionally downloaded the 1.14 (test-launch.c) version to match my gstreamer version.

now you can launch the (toy) server like this:

./test-launch "( videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96 )"

And launch the client like this:

vlc rtsp://127.0.0.1:8554/test
  • The IP and PORT are hardcoded in test-launch.c

BTW, if you are using a jetson, you can search Nvidia Forum for examples. (e.g this)

Upvotes: 2

Related Questions