Memory increase with SharpDX bitmap drawing

Question

I am pretty new to DirectX and Direct2D. I am experimenting with SharpDx to unload the work on the hardware rather than consuming the CPU for constant rendering.

I have a media foundation pipeline. I implemented a custom renderer. My application is in C# and WPF.

I get the raw pixels from my pipeline and push them off to WritableBitamps which is updating an Image control. Apparently there is a known issue with Image control which holds onto brushes. My application can throw Memory Exceptions though to this problem. If the original frame is not big the problem is not noticible. Lets say 800x600 video.

I run into more issues if a 1920 by 1080 video is being played out.

I tried to create one BitmapSource and update the back buffer (to prevent constant bitamp creation) and that did not help. In some cases the memory grows like crazy. As of now I am creating new bitamps for each drawing. I know this is a temp solution for testing.

So now what I am experimenting with is SharpDx. I am using a wrapper called D2DControl which wraps a D3DImage and setup code and allows to create a WPF control which you can plug into your view (Similar to D3DImage. D2DControl is derived from D3DImage) I know some people hate external links but here is the link in case someone is interested.

https://github.com/dalance/D2dControl/tree/master/D2dControl

What this wrapper does is that it hides a lot of details. It exposes a Render() method which allows the drawing to happen.

Here is what I have.

public class MediaDisplayControl : D2dControl.D2dControl
{
    private Bitmap backBufferBmp;        
    public FrameRenderer Renderer { get; set; }

    public override void Render(RenderTarget target)
    {
        var sampleData = Renderer.Samples.FirstOrDefault();

        if (sampleData != null)
        {
            if (backBufferBmp == null)
                backBufferBmp = new Bitmap(target, new SharpDX.Size2(sampleData.Width, sampleData.Height), new BitmapProperties(target.PixelFormat));

            backBufferBmp.CopyFromMemory(sampleData.Pixels, sampleData.Stride);

            target.Transform = new RawMatrix3x2 { M11 = target.Size.Width / backBufferBmp.Size.Width, M22 = target.Size.Height / backBufferBmp.Size.Height };
            target.DrawBitmap(backBufferBmp, 1f, BitmapInterpolationMode.Linear);
        }
    }
}

I tried variations of creating new bitamp on each render or reusing it. On a 1920x1080 video frame, the memory jumps 300mg at a time until it hits 1.2GB and it stops.

I am scaling the bitmap since the D3DImage is not scaling the bitamp. With Image controls, you can write a large bitamp and it will scale to the view size. With this code, the image gets cropped if it is larger than the container.

Regardless, the transformation does not change the outcome. The act of CopyFromMemory and DrawBitamp increases the memory.

So my questions is: 1) Does what I am doing make any sense? 2) Are there other things I can pursue to prevent memory issues?

Thanks for taking the time to read this.

Answer 1

Update:

Thanks a lot for @Evgeny Pereguda detailed answers. It helped me realize some of the thinks I was overlooking. I took the route of wrapping Direct3D9 interfaces in Cli/C++.

In his sample project, he used ComImport to pull in the Direct3D9 into C# side. This requires some knowledge regarding the dlls itself.

Per COM documentation on C# side, you cannot partially pull in an interface. It is either all or just the Class Guida and name. In the Direct3D9 case, there are many functions and trying to define all of them (since I most likely use 1 method out of the whole class) is an overkill.

His method uses reflection to pull in the method based on the address of the method from the dll which requires hard coding the method address. I opted to right a wrapper for the Direct3D9 classes I needed using the header files and D3D9.lib since I a bit of nit picker about hard coding things.

I created a complete user control which can be plugged in to our applications.

Thanks for the guidance on the memory issues. The best route was to use unmanaged memory and I did not realize I was using managed. Switching that helped increase performance and memory managed by a lot.

SharpDx is good, but many of the details are hidden and I would like to have more control.

As of now, I write unmanaged pixels to my pipeline and the memory is performing properly.

Answer 2

I see that you try resolve your task by copy of Unmanaged memory into the Managed memory and again into Unmanaged memory. Firstly, you get raw pixels for rendering - it is not good idea, because code cannot use DirectX Video Acceleration for acceleration of decoding - it decodes into the CPU memory. Secondly, in your code there is Bitmap backBufferBmp - it is a managed memory object and you cannot clear it from memory fast, but you can try GC.Collect Method (). Thirdly, in the rendering process pixels from backBufferBmp in system memory copy into the video memory of WPF. So, in your solution three are many bottlenecks. I can recommend try System.Windows.Interop.D3DImage - it allows attach Direct3DSurface9 into the rendering process WPFwithout copy from memory, then you can use Direct3DSurface9 as a rendering target for using DirectX Video Acceleration. However, it needs write a new custom renderer for Media Foundation pipeline.

So, there is not magic code line for resolving your problem - you need rewrite your code from the scratch.

Regards.

P.S. you can find example of using of System.Windows.Interop.D3DImage by link - WPFViewerEVRDisplay