Gstreamer recording multiple segments of RTP stream to file

Question

I'm writing a c++ application with gstreamer and am trying to achieve the following: connect to an rtp audio stream (opus), write one copy of the entire stream to an audio file, and then additionally, based on events triggered by the user, create a separate series of audio files consisting of segments of the rtp stream (think a start/stop record toggle button).

Currently using udpsrc -> rtpbin -> rtpopusdepay -> queue -> tee (pipeline splits here)

tee_stream_1 -> queue -> webmmux -> filesink

tee_stream_2 -> queue -> webmmux -> filesink

tee_stream_1 should be active during the entire duration of the pipeline. tee_stream_2 is what should generate multiple files based on user toggle events.

An example scenario:

1. pipeline receive rtp audio stream, tee_stream_1 begins writing audio to full_stream.webm
2. 2 seconds into rtp audio stream, user toggles "start recording". tee_stream_2 begins writing audio to stream_segment_1.webm
3. 5 seconds into rtp audio stream, user toggles "stop recording". tee_stream_2 finishes writing audio to stream_segment_1.webm and closes file.
4. 8 seconds into rtp audio stream, user toggles "start recording". tee_stream_2 begins writing audio to stream_segment_2.webm
5. 9 seconds into rtp audio stream, user toggles "stop recording". tee_stream_2 finishes writing audio to stream_segment_2.webm and closes file.
6. 10 seconds into rtp audio stream, stream ends, full_stream.webm finishes writing audio and closes.

End result being 3 audio files, full_stream.webm with 10 seconds of audio, stream_segment_1.webm with 3 seconds of audio, and stream_segment_2.webm with 1 second of audio.

Attempts to do this so far have been met with difficulty since the muxers seem to require an EOS event to finish properly writing the stream_segment files, however this EOS is propogated to the other elements of the pipeline which has the undesired effect of ending all of the recordings. Any ideas on how to best accomplish this? I can provide code if it would be helpful.

Thank you for any and all assistance!

Answer 1

For such case, I'd suggest to give a try to RidgeRun's open source gstd and interpipe plugins that provide high level control of dynamic pipelines.

You may install with something like:

# Some required packages to be installed, not exhaustive...
# If not enough you would see errors and figure out any other missing package
sudo apt install libsoup2.4-dev libjson-glib-dev libdaemon-dev libjansson-dev libreadline-dev gtk-doc-tools python3-pip

# Get gstd sources from github
git clone --recursive https://github.com/RidgeRun/gstd-1.x.git

# Configure, build and install (meson may be better, but here using autogen/configure
cd gstd-1.x
./autogen.sh 
./configure 
make -j $(nproc)
sudo make install
cd ..


# Get gst-interpipe sources from github
git clone --recursive https://github.com/RidgeRun/gst-interpipe.git

# Configure, build and install (meson may be better, but here using autogen/configure
cd gst-interpipe
./autogen.sh 
./configure 
make -j $(nproc)
sudo make install
cd ..

# Tell gstreamer about the new plugins interpipesink and interpipesrc
# First clear gstreamer cache (here using arm64, you would adapt for your arch)
rm ~/.cache/gstreamer-1.0/registry.aarch64.bin

# add new plugins path
export GST_PLUGIN_PATH=/usr/local/lib/gstreamer-1.0

# now any gstreamer command would rebuild the cache, so if ok this should work
gst-inspect-1.0 interpipesink

interpipes need a daemon that manages, so in a first terminal you would just start it. It will display some operations and errors if any:

gstd

Now in a second terminal you would try this script (here recording into directory /home/user/tmp/tmp2...adjust for your case):

#!/bin/sh

gstd-client pipeline_create rtpopussrc udpsrc port=5004 ! application/x-rtp,media=audio,encoding-name=OPUS,clock-rate=48000 ! queue ! rtpbin ! rtpopusdepay ! opusparse ! audio/x-opus ! interpipesink name=opussrc

gstd-client pipeline_create audio_record_full interpipesrc name=audiofull listen-to=opussrc is-live=true allow-renegotiation=true stream-sync=compensate-ts ! queue ! audio/x-opus ! opusparse ! webmmux ! filesink location=/home/user/tmp/tmp2/full_stream.webm


gstd-client pipeline_play rtpopussrc
gstd-client pipeline_play audio_record_full

sleep 2

gstd-client pipeline_create audio_record_1 interpipesrc name=audio_rec1 listen-to=opussrc is-live=true allow-renegotiation=true stream-sync=compensate-ts ! queue ! audio/x-opus ! opusparse ! webmmux ! filesink location=/home/user/tmp/tmp2/stream_segment_1.webm
gstd-client pipeline_play audio_record_1

sleep 3

gstd-client pipeline_stop audio_record_1
gstd-client pipeline_delete audio_record_1

sleep 3

gstd-client pipeline_create audio_record_2 interpipesrc name=audio_rec2 listen-to=opussrc is-live=true allow-renegotiation=true stream-sync=compensate-ts ! queue ! audio/x-opus ! opusparse ! webmmux ! filesink location=/home/user/tmp/tmp2/stream_segment_2.webm
gstd-client pipeline_play audio_record_2

sleep 1

gstd-client pipeline_stop audio_record_2
gstd-client pipeline_delete audio_record_2

sleep 1

gstd-client pipeline_stop audio_record_full
gstd-client pipeline_delete audio_record_full
gstd-client pipeline_stop rtpopussrc
gstd-client pipeline_delete rtpopussrc

echo 'Done'

and check resulting files.