CODEWITHSUNDEEP
gstreamergstreamer-1.0nvidia-jetson
spicyruby
spicyruby

Reputation: 1

Cannot build gstreamer pipeline with tee, appsink and file sink

using: Jetson Orin Nano Jetpack 5.1.2 L4T 35.4.1 Ubuntu 20.04

I am attempting to build the following pipeline: video source → tee t.→ video sink t.-> appsink

each of the pipeline with just one sink, even with the tee works:

TEE videotestsrc - filesink

gst-launch-1.0 videotestsrc ! tee name=t ! \
queue ! nvvidconv ! 'video/x-raw, format=(string)RGBA' ! nvvidconv ! 'video/x-raw, format=(string)NV12' ! x264enc speed-preset=fast ! h264parse ! qtmux ! filesink location=demoVideos/videotestsrc.mov -e

TEE videotestsrc - appsink

gst-launch-1.0 videotestsrc ! tee name=t ! \
queue ! nvvidconv ! 'video/x-raw, format=(string)RGBA' ! appsink -e

However, when attaching both sinks to the tee, i’m getting ‘queue_dataflow gstqueue.c:1516:gst_queue_loop: queue is empty’ using GST_DEBUG=3,queue_dataflow:5`

TEE with app sink and filesink

gst-launch-1.0 videotestsrc ! tee name=t ! \
queue ! nvvidconv ! 'video/x-raw, format=(string)RGBA' ! nvvidconv ! 'video/x-raw, format=(string)NV12' ! x264enc speed-preset=fast ! h264parse ! qtmux ! filesink location=demoVideos/videotestsrc.mov t. ! \
queue ! nvvidconv ! 'video/x-raw, format=(string)RGBA' ! appsink -e

Thank you

i’ve tried:

  1. appsink drop=true
  2. increasing queue max-size-buffer
  3. fakesink instead of app sink

Upvotes: 0

Views: 389

Answers (1)

Akkshay
Akkshay

Reputation: 140

  1. The tee element create two branches of the stream, and they need to stay in sync.
  2. The encoders usually need more than one frame to initialise properly, and wont start pushing the buffers downstream until it initialises and starts encoding.

In your example, when the second branch receives the first buffer, it pushes it to downstream element immediately and waits for first branch to do the same.

But the first branch has encode element x264enc in it - which hasn't completed the initialisation yet and is waiting for second buffer to do that. And the second buffer doesn't arrive until the second branch is ready to receive as well.

This creates the deadlock condition and it stalls the flow of the both pipelines. This is working with the individual pipelines because there is no sync issue.

To fix this, you can either add an encoder in second pipeline so that both of the pipelines wait for second frame and end up in sync. for example

gst-launch-1.0 videotestsrc ! tee name=t ! \
queue ! videoconvert ! 'video/x-raw, format=(string)RGBA' ! videoconvert ! 'video/x-raw, format=(string)NV12' ! x264enc speed-preset=fast ! h264parse ! qtmux ! filesink location=/tmp/videotestsrc.mov t. ! \
queue ! videoconvert ! 'video/x-raw, format=(string)RGBA' ! videoconvert ! 'video/x-raw, format=(string)NV12' ! x264enc ! fakesink -e

OR

The fix that you probably are looking for is - add a queue element in the second branch before the appsink so that it can store the first buffer in it instead of pushing it to downstream element (filesink) and hence doesn't expect the first branch to push the first buffer.

Then both branches are ready to receive the next buffer which lets the encoder initialise properly and continues the data flow.

gst-launch-1.0 videotestsrc ! tee name=t ! \
queue ! videoconvert ! 'video/x-raw, format=(string)RGBA' ! videoconvert ! 'video/x-raw, format=(string)NV12' ! x264enc speed-preset=fast ! h264parse ! qtmux ! filesink location=/tmp/videotestsrc.mov t. ! \
queue ! videoconvert ! 'video/x-raw, format=(string)RGBA' ! queue ! fakesink -e

Also, it is recommended to use queue elements before heavy processing elements such as x264enc and filesink or appsink. I suggest something like

gst-launch-1.0 videotestsrc ! tee name=t ! \
queue ! videoconvert ! 'video/x-raw, format=(string)RGBA' ! videoconvert ! 'video/x-raw, format=(string)NV12' ! queue ! x264enc speed-preset=fast ! h264parse ! qtmux ! queue ! filesink location=/tmp/videotestsrc.mov t. ! \
queue ! videoconvert ! 'video/x-raw, format=(string)RGBA' ! queue ! fakesink -e

Note that - to try to run example quickly, I have used fakesink instead of appsink and videoconvert instead of nvvidconv

Upvotes: 0

Related Questions