CODEWITHSUNDEEP
pythonc++opencvrtsp
AhmedAly
AhmedAly

Reputation: 156

Synchronizing multiple RTSP streams with OpenCV?

I'm currently using OpenCV to digest an RTSP stream with OpenCV and the Python bindings. However, i'm trying to expand this to capture multiple RTSP streams and then perform operations on each frame and perform additional logic on the aggregate results (in parallel). I need to synch these live streams to within ~1 second accuracy. If all my streams were the same profile, this wouldn't be as challenging. However, some of the streams i am digesting are variable frame rate (VFS) so the FPS can drop from the max FPS to the min FPS depending on the bandwidth, while others may be constant.This could be simplified drastically if there was a way to get an embedded timestamp from the RTSP live stream, which is part of the protocol. But, i don't believe there is a way to get this timestamp using OpenCV....

I'm wondering if this has been done before using OpenCV? I know there are commercial broadcasting tools that can do this, but that seems impractical and over kill for my use case. Someone also recommended http://www.ros.org/about-ros/, but again that may be too much for what i am trying to do...

Upvotes: 1

Views: 2910

Answers (1)

Osama AlKoky
Osama AlKoky

Reputation: 214

Look at NTP settings from the RTSP cameras to get a synchronized timestamp on the frames from all cameras. Check this out:

https://ipcamtalk.com/threads/network-time-protocol-ntp.1287/

After that you can start a thread for each camera and store the frames along with their timestamps in a small buffer. Then you can compare the time stamps and drop frames from the faster camera to match the frames as much as possible. I have written a code to do this but I cannot share it at the moment, I will update the answer if that becomes possible in the future.

Upvotes: 1

Related Questions