CODEWITHSUNDEEP
gstreamermultiqueue
Steve Broberg
Steve Broberg

Reputation: 4394

How to configure gstreamer multiqueue to ensure that we grab the same number of frames from each source

We're trying to capture footage from multiple live cameras. We want to end up with a video for each camera where frame N from each video corresponds to the same moment in time (+/- a few milliseconds). From our understanding, multiqueue should do this for us. However, the following pipeline doesn't do what we want:

multiqueue max-size-buffers=1 sync-by-running-time=true name=mqueue \
   tcambin serial=33810032 name=source \
   ! timecodestamper post-messages=true \
   ! timeoverlay shading-value=255 shaded-background=true font-desc=\"Sans, 10\" time-mode=0 \
   ! video/x-raw,format=BGRx,width=1920,height=1080,framerate=30/1 \
   ! mqueue.sink_1 \
   tcambin serial=35910547 name=source1 \
   ! timecodestamper post-messages=true \
   ! timeoverlay shading-value=255 shaded-background=true font-desc=\"Sans, 10\" time-mode=0 \
   ! video/x-raw,format=GRAY8,width=1920,height=1080,framerate=15/1 \
   ! mqueue.sink_2 \
   mqueue.src_1 \
   ! videoconvert \
   ! x264enc speed-preset=1 pass=5 bitrate=100000 threads=6 \
   ! mp4mux \
   ! filesink location=out1.mp4 \
   mqueue.src_2 \
   ! videoconvert \
   ! x264enc speed-preset=1 pass=5 bitrate=100000 threads=6 \
   ! mp4mux \
   ! filesink location=out2.mp4

Note that one camera is set for 30fps, the other for 15. Because we have a queue depth of 1 on the multiqueue, I would expect that the downstream encoding parts of the pipeline (src_1 & src_2) should be pulling off one frame each, which would unblock the upstream sinks, which would then get whatever image is current on the camera.

Instead, we're getting videos where src_1 is has twice as many frames as src_2. Our assumption is that multiqueue should prevent this behavior.

Upvotes: 0

Views: 2630

Answers (1)

philipp
philipp

Reputation: 1813

According to the documentation, sync-by-running-time only refers to syncing between linked and non-linked queues. I.e. if you feed two queues, and only one queues src pad is connected to a consumer, syncing will control how fast buffers in the non-linked queue are removed.

A queue that has sink and src pad linked will never discard any buffers.

Upvotes: 0

Related Questions