How do I rework an existing Direct2D-based library to function accessed from multiple threads at once?

Question

I am using an existing graphics library, one implementation of which uses Direct2D. (It can also use GDI+, Quartz, etc.) It has bitmap objects, pens, brushes etc, all implemented as wrapper code around Direct2D objects. The library is closed source and unfortunately I probably can't post extensive portions of it here.

I have found that when using multiple threads - one thread producer, creating bitmaps and drawing to them, one thread consumer, drawing them onscreen - sometimes the bitmap the producer thread produces is completely blank: the image data (internally a texture) is all zeroes. (A bitmap is a wrapper object around a texture, but representing a bitmap like a you might think of a GDI+ bitmap object, perhaps. I'm sure you're familiar with the general paradigm.) Note that it never crashes, just sometimes silently fails. A typical symptom will be drawing to the bitmap data, calling Unmap to copy from the buffer back to the texture, and finding when you later try to draw it that the texture is blank.

What I think is happening is:

• To write directly to the pixel data, as my producer thread is doing, the library provides Map and Unmap functions to access the pixel data directly. There is one DirectX10 device, a "shared device", and it is used to call CopyResource to copy from the internal texture to a buffer, which can then be written to freely, and then unmapping copies back the other way. I think sometimes this silently fails.
• This approach of shared devices is used throughout the library: there is a shared render target (ID2D1RenderTarget) used to create a shared bitmap (ID2D1Bitmap), etc. The approach seems to be that graphics / bitmap objects have their own textures, of course, but all operations on them are done with the one device, one render target, etc.
• When multiple threads use "bitmaps" at the same time several operations can fail: CopyResource (mentioned above), which returns void not HRESULT, appears to fail sometimes; ID2D1RenderTarget::DrawBitmap also silently fails.

So, I would like to solve the problem and hack into the library enough thread safety I can achieve my producer-consumer thread system reliably. How do I do this?

I've been doing a lot of reading about the right way to use Direct2D from multiple threads.

• All factories are already created with the D2D1_FACTORY_TYPE_MULTI_THREADED flag.

• I tried using the ID2D1MultiThread interface for synchronization via the factories. However, this only seems to be available on Windows 7 and above: I need to use DirectX10-level, Vista+ APIs.

• I have also tried using the ID3D10Multithread interface for synchronizing. I've run into a few problems with this:

  I'm not sure where I should be synchronizing, or around what. If it is fine-grained, such as around Map and Unmap calls, or around Present as MSDN indicates is required, the synchronization has no effect: the above method calls still fail. If it is coarser-grained, entering the lock when calling ID2D1RenderTarget::BeginDraw, holding it for all drawing, and leaving the lock after ID2D1RenderTarget::EndDraw, ID2D1RenderTarget::Flush and/or IDXGISwapChain::Present are called, it seems to work brilliantly... until after a few seconds I enter a deadlock. MSDN describes this,

  Be careful that you never have the message-pump thread wait on the render thread when you use full-screen swap chains. For instance, calling IDXGISwapChain1::Present1 (from the render thread) may cause the render thread to wait on the message-pump thread. When a mode change occurs, this scenario is possible if Present1 calls ::SetWindowPos() or ::SetWindowStyle() and either of these methods call ::SendMessage(). In this scenario, if the message-pump thread has a critical section guarding it or if the render thread is blocked, then the two threads will deadlock. - Source

  ...but has no guidance on how to avoid it. My deadlocks appear to be caused by two threads both trying to access graphics, eg calling BeginDraw, at the same time. There appears to be a second lock in action, since if there was just the one lock then there would be no deadlock.

• I have tried keeping per-thread instances of the "shared" device, render target, etc. That is, I have one instance per thread of each type of object, which is then used by all code running in that thread. This works well, except for textures: it seems that accessing a texture created in another thread context (by another device) doesn't work - at all. Quite often while coding this technique I have run into problems where methods fail - the most common being CreateSharedBitmap, which fails with D2DERR_UNSUPPORTED_OPERATION, where a texture came from another thread (thus, another device.) This is the key bit, because a bitmap object (wrapping a texture) created in one thread needs to be able to be drawn in another thread. Depending on how it's coded, there may or may not be a 1:1 relationship between threads and devices, render targets, etc.

  If I can resolve this, I think the rest of what I've coded works sufficiently well for me to achieve what I need to. Is it possible - and how? - to cross ID3D10Texture2Ds between ID3D10Device1s? Is it necessary to have separate devices across threads?

All up I'm a bit stumped and am seeking advice from those familiar with Direct3D 10 and Direct2D about the best approach. I think there are two viable possibilities:

1. Figure out the ID3D10Multithread interface locking. There isn't much on Google, nor much advice about avoiding deadlocks on MSDN.

2. Continue with the mechanism where every thread has its own device, shared render target, etc. (How much of this is necessary - is it possible, say, for there to be one device only but individual render targets per thread?) If so, how can texture resources cross from one thread to another, where that may mean from one device to another? Basically, a wrapper bitmap object which uses a texture internally created in one thread needs to be useable to draw on a second.

but am keen to hear about any other possibilities too. Any advice whatsoever about correct Direct2D threading implementations will be very gratefully heard.

Answer 1

The model that works for me is single D2D and D3D factory and per-thread D2D render targets. Synchronization is done via my own critical sections. I don't need to share bitmaps between threads though.

What should work in your case is either having separate everything (factory, render targets, etc.) per every thread and work with shared resources. This also answers your question on how to cross ID3D10Texture2Ds between ID3D10Device1s. You'll need those functions: IDXGIResource::GetSharedHandle ID3D10Device::OpenSharedResource IDXGIKeyedMutex::AcquireSync

Or create just one factory, one render target, etc. per process and sync via your own critical sections. In this case, synchronization needs to be done around every D2D or D3D call that touches a batch (BeginDraw-EndDraw) which uses another thread's bitmaps.

I found enabling both D2D and D3D debug layers and using Windows 8.1 SDK very helpful while working on multi-threading issues. New Windows 8.1 SDK has new messages for multi-threading issues. You don't need to be on Windows 8.1 to use 8.1 SDK.