CODEWITHSUNDEEP
pythontimestampsender
Alex
Alex

Reputation: 21

Absolute timestamps from the sender report (SR) for RTSP video stream

I get video streams from ip cameras using rtsp in python and want to get absolute timestamp for each frame in the sender reports. If I read a stream the camtime obviously starts from zero: ret, image_np = cap.read() camtime = cap.get(cv2.CAP_PROP_POS_MSEC)/1000. I need to know when the frames are recorded since the cameras are located in different time zones and the frames are received with different delays. Can someone advise what library to use and how the python code should look like? Thanks! P.S. I can get camera streaming start time from ffmpeg in SDP (option "o").

Upvotes: 2

Views: 3142

Answers (1)

Stamatis Tzanos
Stamatis Tzanos

Reputation: 93

After a solid three weeks of experimenting I managed to do this with GStreamer.

GStreamer applications are made up of pipelines of elements where each element has different uses and processes the data flow differently. Read the documentation to get a grasp on what is happening and this tutorial. In short, data is passed from element to element within buffers and is processed accordingly before passed to the next element.

When you understand how GStreamer works, setup a pipeline like this:

rtspsrc -> rtph264depay -> h264parse -> avdec_h264 -> videoconvert -> capsfilter -> videoconvert -> fakesink

It is a matter of learning each element's properties to set up your pipeline correctly but the most significant thing here is to set up your rtspsrc correctly:

rtsp_source = Gst.ElementFactory.make("rtsprc", "rtsp_source")
rtsp_source.set_property("location", RTSP_URL)

Then get the sink pad of your rtph264depay which has the RTP data and setup a probe for it:

pad = element.get_static_pad("sink")
pad.add_probe(Gst.PadProbeType.BUFFER, callback_func, None)

This means that whenever data passes this pad, a callback function is called which receives the data in a buffer. Access the buffer:

r, map_info = buffer.map(Gst.MapFlags.READ)
raw_data = map_info.data
buffer.unmap(map_info)

Now you have the RTP packet with it's header and payload in raw_data. From it, you can now extract the RTP timestamp within your callback. You can either do it yourself, or even better, I preferred to do it with this amazing library.

Then, if you also need the frame, setup another probe in your fakesink's sink pad. In the same way, your buffer map_info now has frame data which you can convert to a frame with numpy for example:

rgb_frame = np.ndarray(
            shape=(height, width, 3),
            dtype=np.uint8,
            buffer=map_info.data,
        )

Do what you will with the frame and timestamp. Good luck!

Upvotes: 4

Related Questions