CODEWITHSUNDEEP
windowsopenglrgbmp4ms-media-foundation
JStefanelli
JStefanelli

Reputation: 143

How to have custom video media/stream sink request RGB32 frames in media foundation?

I am trying to make a custom media sink for video playback in an OpenGL application (without the various WGL_NV_DX_INTEROP, as I am not sure if all my target devices support this).

What I have done so far is to write a custom stream sink that accepts RGB32 samples and set up playback with a media session, however i encountered a problem with initial testing of playing an mp4 file:

If, however, I configure the stream sink to receive NV12 samples, everything seems to work fine.

My best guess is the color converter MFT generated by the TopologyLoader needs some more configuration, but I don't know how to do that, considering that I need to keep this entire process indipendent from the original file types.

Upvotes: 2

Views: 465

Answers (1)

mofo77
mofo77

Reputation: 1515

I've made a minimal test case, that demonstrate the use of a custom video renderer with a classical Media Session.

I use big_buck_bunny_720p_50mb.mp4, and i don't see any problems using RGB32 format.

Sample code here : https://github.com/mofo7777/Stackoverflow under MinimalSinkRenderer.

EDIT

Your program works well with big_buck_bunny_720p_50mb.mp4. I think that your mp4 file is the problem. Share it, if you can.

I just made a few changes :

You Stop on MESessionEnded, and you Close on MESessionStopped.

case MediaEventType.MESessionEnded:
    Debug.WriteLine("MediaSession:SesssionEndedEvent");
    hr = mediaSession.Stop();
    break;
case MediaEventType.MESessionClosed:
    Debug.WriteLine("MediaSession:SessionClosedEvent");
    receiveSessionEvent = false;
    break;
case MediaEventType.MESessionStopped:
    Debug.WriteLine("MediaSession:SesssionStoppedEvent");
    hr = mediaSession.Close();
    break;
default:
    Debug.WriteLine("MediaSession:Event: " + eventType);
    break;

Adding this to wait for the sound, and to check sample is ok :

internal HResult ProcessSample(IMFSample s)
{
    //Debug.WriteLine("Received sample!");

    CurrentFrame++;

    if (s != null)
    {
        long llSampleTime = 0;
        HResult hr = s.GetSampleTime(out llSampleTime);

        if (hr == HResult.S_OK && ((CurrentFrame % 50) == 0))
        {
            TimeSpan ts = TimeSpan.FromMilliseconds(llSampleTime / (10000000 / 1000));
            Debug.WriteLine("Frame {0} : {1}", CurrentFrame.ToString(), ts.ToString());
        }

        // Do not call SafeRelease here, it is done by the caller, it is a parameter
        //SafeRelease(s);
    }

    System.Threading.Thread.Sleep(26);

    return HResult.S_OK;
}

In

public HResult SetPresentationClock(IMFPresentationClock pPresentationClock)

adding

SafeRelease(PresentationClock);

before

if (pPresentationClock != null)
    PresentationClock = pPresentationClock;

Upvotes: 2

Related Questions