Encoding a D3D Surface obtained through Desktop Duplication using Media Foundation

Question

I want to encode Desktop Duplication API frames to send over the network after encoding them with Media Foundation. I'm stuck with a E_NOTIMPL error when I call IMFTransform::ProcessInput, leaving me a little in the dark.

These are the steps I've done up until now, I'm detailing them because it took me days to gather everything from the scarce scattered info across the web, so if it's resolved it will hopefully help others. Everything below is met with an S_OK:

• I'm obtaining the surface through Duplication API, creating an IMFSample from it using MFCreateVideoSampleFromSurface
• I'm getting a video encoder using IMFActivate::ActivateObject from an IMFActivate initialized with MFT_CATEGORY_VIDEO_ENCODER and MFVideoFormat_H264
• I'm initializing IMFMediaType on the input with bitrate, framerate, aspect ratio, etc.. and most importantly: MFVideoFormat_NV12, seems to be the only one to work with the DXGI_FORMAT_B8G8R8A8_UNORM of Desktop Duplication API.
• I'm setting IMFMediatype on the output with the same as the above, aside from MFVideoFormat_H264 as SUB_TYPE.
• I'm calling the IMFTransform::SetOutputType then IMFTransform::SetInputType with the 2 above.
• I'm setting the IMFSample time at 0 as it seems it's not being set by MFCreateVideaSampleFromSurface. Also setting the SampleDuration with MFFrameRateToAverageTimePerFrame with the input FPS.

After all of this, I call the MFTransform::ProcessInput with the IMFSample created above, and get a "E_NOTIMPL not implemented" as an HRESULT. I've read that I should set an IMFDXGIDeviceManager to my IMFTransform encoder above, so I did that using:

• MFCreateDXGIDeviceManager from my ID3D11Device used with Desktop Duplication API and an arbitrary reset token.
• Doing an IMFDXGIDeviceManager::ResetDevice with the device and token.
• Calling IMFTransform::ProcessMessage(MFT_MESSAGE_SET_D3D_MANAGER, reinterpret_cast(m_pDXDeviceManager).

On this last step I get another "E_NOTIMPL not implemented" on my HRESULT, and that's where I don't know what I'm doing wrong anymore or what needs to be done.

Answer 1

The first thing to do, is to check if your encoder is MF_SA_D3D11_AWARE

I think it does not, reading this : MFT_MESSAGE_SET_D3D_MANAGER

If an MFT does not support this message, it should return E_NOTIMPL

But one thing is strange : DirectX Surface Buffer

Call IMFMediaBuffer::Lock. Generally this is not recommended, because it forces the object to copy memory from the Direct3D surface and then back again.

ProcessInput should have worked.