CODEWITHSUNDEEP
ioscore-data
Code Monkey
Code Monkey

Reputation: 41

Problematic NSManagedObject accumulation over time

I have an application which continuously receives a stream of XML messages from a TCP/IP endpoint. Upon receipt of each message, the application digests it's contents into a set of core data entities. This is accomplished via a three context structure:

This arrangement keeps the stream processing off the main thread. The application typically receives anywhere from 10 - 150 messages every second or two. Saving of the Stream context takes place after each message is deconstructed and persisted. CPU usage is typically short of 15% on an A6 level device.

My problem however is memory. If I hook up an NSFetchedResultsController to the Main context I get a nice flow of the messages as they arrive. However, if I profile I notice that my NSManagedObject count gradually increases. Eventually memory pressure will cause the app to terminate.

After 12 minutes of profiling, the app has consumed 6300 XML messages and parsed 121,000 properties. This consumes 7.8MB for the properties, 438KB for the messages and the total app size is now 54MB. Obviously this isn't sustainable.

Instruments notes that all of the objects are still live. Trolling around the interwebs leads me to believe I might have a retain cycle causing the objects to not be faulted. However, the suggestion of using "refreshObject" isn't clear in the documentation that it would apply here.

Once the XML has been received, a Message entity is created. Next a Type entity is created using the root node of the XML as it's name - and associated bits. Similarly for each element and sub element of those elements and any inline properties of the XML a property element is created. This is the fun part as it has a reference to the message (for a flat representation of all properties) as well as a hierarchal childProperties relationship to itself. At the end of this process the context is saved and the Main context picks it up and the FRC displays the new row.

One thought was to reset the Stream context after a save every few hundred of messages persisted. If I disconnect the FRC, I can stay basically level - however this feels wrong and doesn't solve the problem when I wire the FRC back up.

Any thoughts would be appreciated.

Upvotes: 4

Views: 784

Answers (1)

eofster
eofster

Reputation: 2047

I would suggest configuring your Stream context with the same persistent store coordinator that is used for the Master context. And maybe periodically reset the stream context.

In the current configuration Stream context fill put additional pressure to its parents. And if big updates are happening in the Stream context, this pressure becomes more visible.

First, when the Stream context needs to do something that requires a lock, it will lock both parents.

Second, when save happens in the Stream context, all the changes are pushed back to the parent, the Main context. And you don’t have control over it. If there is a fetched results controller in the Main context, then on save it will replay all the changes one-by-one. And if the update is big, it will bring a big overhead. Definitely in CPU and probably in memory.

I think the best pattern for handling big updates in the background and refreshing the UI (especially with the fetched results controller) is to configure the context that does big updates directly with persistent store coordinator. And then, when big a update happens, just refetch in the UI context. And don’t forget to set fetch batch size on the fetch request to some meaningful to your case value. You could start with the number of cells visible on the screen.

This pattern is more efficient but comes with complexity cost. You need to think how to refresh the data in other contexts. You need to take care of this because Core Data doesn’t touch objects that are fully realized. (Setting setShouldRefreshRefetchedObjects doesn’t help either because of the bug confirmed to me by Apple.)

For example, you fetched some object in the Main context, accessed its property for displaying it on the screen. This object is not a fault any more. Then your Stream context (now configured with the persistent store coordinator directly) updated the same property. Even if you refetch in the Main context and the object will be in the search results, object properties will not be updated.

So you could use something like this:

- (void)refreshObjectsOnContextDidSaveNotification:(NSNotification *)notification {
    NSSet *updatedObjects = notification.userInfo[NSUpdatedObjectsKey];
    NSSet *updatedObjectIDs = [updatedObjects valueForKey:@"objectID"];

    [self.mainContext performBlock:^{
        for (NSManagedObject *object in [self.mainContext registeredObjects]) {
            if (![object isFault] && [updatedObjectIDs containsObject:[object objectID]]) {
                [self.mainContext refreshObject:object mergeChanges:YES];
            }
        }
    }];

    [self.masterContext performBlock:^{
        for (NSManagedObject *object in [self.masterContext registeredObjects]) {
            if (![object isFault] && [updatedObjectIDs containsObject:[object objectID]]) {
                [self.masterContext refreshObject:object mergeChanges:YES];
            }
        }
    }];
}

This will refresh updated objects in main and master contexts.

When the save in the Stream context is not huge you could simply merge changes using the standard merge method into other two contexts. When fetched results controller is used, you’ll be able to see nice cell animations on object deletion and insertion. The number of objects affected in a save you can get from NSInsertedObjectsKey, NSUpdatedObjectsKey, and NSDeletedObjectsKey keys of the user info in the context-did-save notification.

And after each big save you could reset the Stream context. Just don’t forget that you can’t access any previously fetched objects in this context after the reset.

Upvotes: 5

Related Questions