CODEWITHSUNDEEP
gstreamer
Yuniesky Vasconcelo
Yuniesky Vasconcelo

Reputation: 161

How to get the correct video frame number or time position from appsink buffer

Thanks in advance.

I want to record a video from an rtsp video camera and at the same time process the video frame obtained from appsink throught new-sample signal. Then, in a separate application I read the recorded video and show the information related to the frames processed.

Documentation say that buffer->offset have the video frame number, but doesnt work for me, it allways have the same number.

I have this pipeline:

rtspsrc location=rtsp://10.0.0.1:554/video.sdp latency=100 ! rtph264depay ! tee name=t
! queue ! vaapidecodebin ! vaapipostproc format=rgba  !  appsink name=appsink t.
! queue ! h264parse ! mp4mux ! filesink sync=false name=filer location=/home/VideoDB/2017-09-04_16:33:46.mp4

Code example:

GstFlowReturn GstVideoSourcePrivate::newSample(GstAppSink* sink, gpointer user_data)
{
....
    GstSample* sinkSample = gst_app_sink_pull_sample(GST_APP_SINK(sink));
    if (sinkSample) {
        GstBuffer* buffer = gst_sample_get_buffer(sinkSample);

       // I need this position to be the same as the recorded video
       // or get the frame video sequence number, so that we
        GstClockTime pos;
        gst_element_query_position(self->pipeline(), GST_FORMAT_TIME, &pos);
        ...
    }
    ...
}

Upvotes: 1

Views: 3252

Answers (1)

Yuniesky Vasconcelo
Yuniesky Vasconcelo

Reputation: 161

Thanks for your answer.

I did what you told me, but I can not get the expected result.

Then I discovered that when a videorate element is inserted into the pipeline, buffer->offset begins to display the correct sequence of video frame. But again, I can not get a good sync for a few milliseconds.

So, I read the doc one more time and I made this code to get a better result. It seems that there are few latencies that need to be compensated.

https://gstreamer.freedesktop.org/documentation/application-development/advanced/clocks.html

https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/clock.html

...
int64_t timestam = GST_BUFFER_TIMESTAMP(buffer);
GstSegment* segment = gst_sample_get_segment(sinkSample);
gint64 pos = gst_segment_to_stream_time(segment, GST_FORMAT_TIME, timestam);
GstQuery*q = gst_query_new_latency();
if (gst_element_query (self->m_pipeline, q)) {
    gboolean live;
    GstClockTime minlat, maxlat;
    gst_query_parse_latency (q, &live, &minlat, &maxlat);
    pos+= minlat;
}
gst_query_unref (q);
...

Upvotes: 1

Related Questions