CODEWITHSUNDEEP
c#.netgstreameravaloniauigstreamer-sharp
Jiří Darmovzal
Jiří Darmovzal

Reputation: 1

How to add gdkpixbufoverlay to Pipeline in gstreamer-sharp-netcore

I have a .NET application with Avalonia UI, where I render the stream from the webcam according to this sample: Rudimentary video player made with Avalonia UI. The code works as expected, but I am unable to add gdkpixbufoverlay to the Pipeline to add a PNG overlay over the video stream.

Setting up the Pipeline:

Pipeline = new Pipeline("playback");
Playbin = Gst.ElementFactory.Make("playbin", "playbin");
Playbin["uri"] = AppController.Configuration["keying:videoStream"];
Pipeline.Add(Playbin);

VideoSink = new AppSink("videoSink");
VideoSink["caps"] = Caps.FromString("video/x-raw,format=RGBA");
VideoSink.Drop = true;
VideoSink.Sync = true;
VideoSink.MaxLateness = (1000 / 30) * 1000000;
VideoSink.MaxBuffers = 1;
VideoSink.Qos = true;
VideoSink.EnableLastSample = false;

Playbin["video-sink"] = VideoSink;

Pipeline.SetState(Gst.State.Playing);

Rendering:

private void OnTick(TimeSpan time)
{
    if (System.Threading.Interlocked.CompareExchange(ref renderLock, 1, 0) == 0)
    {
        try
        {
            PullVideoSample();
        }
        finally
        {
            System.Threading.Interlocked.Decrement(ref renderLock);
        }
    }
}

private void PullVideoSample()
{
    AppSink? sink = VideoSink;

    if (sink == null)
        return;

    using Sample sample = sink.TryPullSample(0);
    if (sample == null)
        return;

    using Caps caps = sample.Caps;
    Structure cap = caps[0];

    string format;
    format = cap.GetString("format");
    cap.GetInt("width", out int width);
    cap.GetInt("height", out int height);

    using Gst.Buffer buffer = sample.Buffer;
    if (format == "RGBA" && buffer.Map(out MapInfo map, MapFlags.Read))
    {
        FrameImage.UpdateImage(ref map, width, height);
        buffer.Unmap(map);
    }
}

I tried different ways of adding it to the Pipeline, but everything ended up with either a black output or opening the output in a new Direct3D11 renderer window.

Thank you for your advice.

Upvotes: 0

Views: 302

Answers (1)

Jiří Darmovzal
Jiří Darmovzal

Reputation: 1

Finally, I was able to insert the overlay into the pipeline as follows.

Setting up the Pipeline:

Pipeline = Parse.Launch($"uridecodebin uri={AppController.Configuration["keying:videoStream"]} ! queue ! videoconvert ! videoscale ! video/x-raw,width=3840,height=2160 ! gdkpixbufoverlay location={overlayImage} ! appsink name=mysink") as Pipeline ?? throw new Exception("Unable to create Pipeline");

VideoSink = Pipeline.GetChildByName("mysink") as AppSink ?? throw new Exception("Unable to create VideoSink");

The rest of the code remained the same. I wasn't able to build the pipeline from objects, but this is good enough for me.

Upvotes: 0

Related Questions