Main thread waiting on [NSManagedObjectContext(_NSInternalAdditions) lockObjectStore] while background thread is executing

Question

I'm executing an expensive fetch (~5 seconds, around 30,000 objects) on a background thread (using GCD) via a newly-created NSManagedObjectContext that's owned by the background thread.

However, I'm not getting the benefit of doing this in the background, as the main thread is waiting for a lock on the persistent store and thus the UI is frozen. Here's the stack trace:

* thread #1: tid = 0x1c03, 0x3641be78 CoreData`-[NSManagedObjectContext(_NSInternalAdditions) lockObjectStore], stop reason = breakpoint 1.1
    frame #0: 0x3641be78 CoreData`-[NSManagedObjectContext(_NSInternalAdditions) lockObjectStore]
    frame #1: 0x36432f06 CoreData`-[_PFManagedObjectReferenceQueue _processReferenceQueue:] + 1754
    frame #2: 0x36435fd6 CoreData`_performRunLoopAction + 202
    frame #3: 0x35ab9b1a CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 18
    frame #4: 0x35ab7d56 CoreFoundation`__CFRunLoopDoObservers + 258
    frame #5: 0x35ab80b0 CoreFoundation`__CFRunLoopRun + 760
    frame #6: 0x35a3b4a4 CoreFoundation`CFRunLoopRunSpecific + 300
    frame #7: 0x35a3b36c CoreFoundation`CFRunLoopRunInMode + 104
    frame #8: 0x376d7438 GraphicsServices`GSEventRunModal + 136
    frame #9: 0x33547cd4 UIKit`UIApplicationMain + 1080
    frame #10: 0x000f337a MyApp`main + 90 at main.m:16

I (believe) I've confirmed that I'm not accessing the main thread's NSManagedObjectContext while this background thread is doing its work. And from the stack trace, it's clear that I'm not directly doing anything with Core Data. But something has triggered a call to _processReferenceQueue: which is causing the attempt to lock the store. Does anyone happen to know what this method does and how/if I can prevent it from getting called while my background thread is doing its work?

EDIT

After I kick off this background fetch, I am not doing ANY Core Data reads or writes on the main thread. That's why this is so puzzling. I would expect there to be contention if the main thread was also trying to do some more work, but it's not - at least, I haven't asked it to. No reads, no writes, no FRC. That's why I would like to know if anyone's familiar with this _processReferenceQueue method. Why is it getting called? What could I be doing that is causing it to run?

EDIT

As a test, I tried to get the MT's MOC into a state where it had no pending changes before I set the BT off to do the fetch, in the hopes that it wouldn't need to do any work in _processReferenceQueue that would require a lock on the store.

Prior to kicking off the BT, I noticed that there was a single object in the [MOC updatedObjects] set. There were no objects in the inserted or deleted sets.

After calling [MOC save], the [MOC updatedObjects] set was empty, as expected.
However, once I kicked off the BT, the the MT still attempted to lock the store inside _processReferenceQueue, even though nothing should be dirty in the MOC.

The next thing I tried (strictly as a test) was to instead call [MOC reset] before kicking off the BT. Again, the [MOC updatedObjects] set was empty after the reset, as expected. At this point in the code I'm not touching any managed objects on the MT until the BT is finished its work (so I'm not running into any problems due to the reset invalidating managed objects I already have references to). This time, however, the MT did NOT try to lock the persistent store in _processReferenceQueue.

This behaviour suggests to me that I'm not doing anything explicit with the MOC on the MT after I've kicked off the BT. Otherwise, the MT would have requested a lock at some point (for a read or a write), inside or outside of _processReferenceQueue. But it didn't.

I'm not sure why a recently-saved MOC requires a subsequent lock in _processReferenceQueue whereas a recently-reset MOC does not.

I'll keep digging.

Thanks! Brian

Answer 1

Without more information, all I can do is take a guess.

Unfortunately, Core Data does not implement reader/writer locks for MOCs using the same persistent store coordinator. That means a read on one thread will block other threads that use the same persistent store coordinator.

Thus, your long-running fetch on the background will prevent the main thread from fetching.

You have several options.

Break up that long-running fetch so that it fetches small pieces in sequence. You want to do multiple small fetches, not issuing the next piece until the current one is done.

This will allow a fetch on the main thread to get an opportunity to be processed in between the multiple small fetches on the background thread.

The other alternative is to create a separate persistent store coordinator, connect your background MOC to that, and run your fetch. You can have multiple readers at the same time if you utilize multiple persistent store coordinators, and they will not permanently block each other.

EDIT

I've definitely thought about breaking it up. But this isn't a case where I'm trying to make two simultaneous fetches play nicely together. The MT shouldn't be touching Core Data at this point. – BrianJStafford

Are you using UIManagedDocument, or have you created a MOC on the main thread, or with NSMainQueueConcurrencyType?

I ask because Core Data makes sure all "user events" are processed each time through the run loop. I've never delved to determine exactly how this is done, but I imagine they add a handler to the run loop for MOC that is on the main thread.

Core Data is dead simple to use, except when you have multiple MOCs. Then, the interaction is so complex that it is difficult to determine what is happening without actual code.

If you can, post the code that creates any/all MOCs, and the code that uses those MOCs.

I sincerely doubt they would have the run-loop-processing actually acquire a lock on the store if there was nothing to do with the current context, so maybe there is some hidden interaction going on with the background MOC. My bet is still something in your app that you don't think you are doing that is somehow crossing the background work with the main MOC.

If you want, you can easily track down how [_PFManagedObjectReferenceQueue _processReferenceQueue:] is called.

Test 1. Create a simple project, check the Core Data box to get a simple Core Data template project. Put a breakpoint at _processReferenceQueue: just to make sure you can get that breakpoint.

Test 2. Comment out the part that instantiates the core data stuff, and see if you hit the breakpoint just by linking against the framework.

Test 3. Create the MOC, but do not do anything else with it. Just create it. Repeat to see if you hit the breakpoint.

Test 4. Get rid of the "default MOC" so you basically are like Test 2. Now, create a private-queue MOC, do a simple transaction with it via performBlock and see if you hit the breakpoint.

You can see where this is going. Basically, determine what combination of very simple use cases cause you to hit your breakpoint in the main thread.

That will, at least, let you know if/when the framework itself is causing this to happen, which should, in turn, give you an idea of what your complex app is doing.

EDIT

Curiosity got the better of me. I just ran a few simple tests, setting breakpoints at both:

-[NSManagedObjectContext(_NSInternalAdditions) lockObjectStore]
-[_PFManagedObjectReferenceQueue _processReferenceQueue:]

I see neither hit unless I am actually messing with the MOC. Each MOC is confinement concurrency type. I created a database with 100,000 simple entities.

The main thread uses a FRC to load the data. I added a button that, when pressed, kicks off a dispatch_async and fetches all 100,000 objects.

I see both breakpoints hit in the supplementary thread, but neither are hit in the main thread. If I fetch or update with the MOC on the main thread, of course, I hit those breakpoints.

I am running on the simulator.

So, my conclusion is that CoreData, by itself, is not responsible for the behavior you are seeing.

Possibly, it is due to some configuration on the MOC, persistent store coordinator, persistent store, or some unknown interaction between your contexts.

Without further information on the configuration of your objects and actual code, my best conclusion is that you are doing "something" in your code to cause this to happen.

You should set these breakpoints, and see what is happening in your code when they are hit.