CODEWITHSUNDEEP
gstreamerpython-gstreamer
tamworth10
tamworth10

Reputation: 21

Adding 10 second wav file to gstreamer pipeline that is already playing

I have a gstreamer pipeline created from the python gst bindings, which is set up to play a headset's microphone back to the headset's speaker. This works fine and is playing in a pipeline like this:

JackAudioSrc -> GstAudioMixer -> Queue -> GstJackAudioSink

Then many seconds later I want to play a short 10 second .wav file into the pipeline so the wav file is mixed with the microphone and heard on the headset. To do this, a GstFileSrc is dynamically added to the GstAudioMixer to mix in a short 10 second wav file to the headset's speaker, which gives pipeline like this:

GstJackAudioSrc             -> GstAudioMixer -> Queue -> GstJackAudioSink
                             /
Gstfilesrc -> Gstwavparse ->/

When the Gstfilesrc and Gstwavparse file is dynamically added to a sink pad of the mixer, at a time say 6 seconds since the start of the pipeline, only the last 4 seconds of the wav is heard.

The problem seems to be that the wav file seeks to the time relative to when the pipeline started PLAYING.

I have tried changing "do-timestamp" in a multifilesrc, and GstIndentity "sync"=True, and can't find a way to set "live" on a filesrc, and many others but to no avail.

However, the whole 10 second wav file will play nicely if the pipeline is set to Gst.State.NULL then back to Gst.State.PLAYING when the filesrc is added at 6 seconds. This works as the pipeline time gets set back to zero, but this produces a click on the headset, which is unacceptable.

How can I ensure that the wav file starts playing from the start of the wav file, so that the whole 10 seconds is heard on the headset, if added to the pipeline at any random time?

An Update:

I can now get the timing of the wave file correct by adding a clocksync and setting its timestamp offset, before the wavparse:

nanosecs = pipeline.query_position(Gst.Format.TIME)[1]
clocksync.set_property("ts-offset", nanosecs)

Although the start/stop times are now correct, the wav audio is corrupted and heard as nothing but clicks and blips, but at least it starts playing at the correct time and finishes at the correct time. Note that without the clocksync the wav file audio is perfectly clear, it just starts and stops at the wrong time. So the ts-offset is somehow corrupting the audio.

Why is the audio being corrupted?

Upvotes: 0

Views: 541

Answers (1)

tamworth10
tamworth10

Reputation: 21

So I got this working and the answer is not to use the clocksync, but instead request a mixer sink pad, then call set_offset(nanosecs) on the mixer sink pad, before linking the wavparse to the mixer:

    sink_pad = audio_mixer.get_request_pad("sink_%u")
    nanosecs = pipeline.query_position(Gst.Format.TIME)[1]
    sink_pad.set_offset(nanosecs)
    sink_pad.add_probe(GstPadProbeType.IDLE, wav_callback)

    def wav_callback(pad, pad_probe_info, userdata):
        wavparse.link(audio_mixer)
        wav_bin.set_state(Gst.State.PLAYING)
        return Gst.PadProbeReturn.REMOVE

Then if the wav file needs to be rewound/replayed:

    def replay_wav():
        global wav_bin
        global sink_pad
        wav_bin.seek_simple(Gst.Format.TIME, Gst.SeekFlags.FLUSH, 0)
        nanosecs = pipeline.query_position(Gst.Format.TIME)[1]
        sink_pad.set_offset(nanosecs)

Upvotes: 2

Related Questions