Coredata background saving, sync strategies and conflict resolution

Question

This is more of an architectural question so I'm not sure if it is a good fit for this site, but I thought I'd give it a try anyway.

Upon login, one of our apps fetches Item updates from the server based on the latest timestamp saved locally. So any records newer than the newest local timestamp get downloaded and imported into the database.

This all happens in a background thread using a local managed object context.

The app also downloads ItemCollection instances from the server (created via CMS), which are essentially collections of Item instances.

While new Item instances are being downloaded/created in the background, another process is fetching ItemCollection items from the same server.

Each ItemCollection has a name attribute and an array of ItemIDs imported as a Coredata relationship.

Sometimes the ItemCollection import will reference an ItemUniqueID that has not yet been created by the background process. In this case we create a placeholder Item that only contains its ItemUniqueID and is flagged as placehoder/pending processing.

So between these two imports, there is a high probability of duplicate items being created.

The ItemCollection import process may create an Item with ID=1, while the same item could be created on the background thread that is downloading and processing Items from the server, and we will end up with two Item instances referencing the same ID==1.

Because the two processing are running concurrently, in separate threads and using separate managed object contexts, the only time these duplicate objects will be aware of each other is when we call save on the managedObjectContext in the main thread.

So the TL;DR version of my question is how can I ensure two separate managed object contexts do not create/update/delete NSManagedObject duplicates and when that happens, what is the correct way to deal with it using some sort of conflict resolution?

Thanks, Rog

Answer 1

You are using your own unique id. That's good. However, each context could have its own view of the data, unknown to the other because the store has not been updated. This could cause false readings/interpretations.

You can try to manage this by watching NSManagedObjectContextObjectsDidChangeNotification which will allow each context to know about what the other is doing, and thus know when objects with the same unique ID have appeared. However, this may be more effort than it's worth.

Why do you have to do the downloads at the same time?

Why not perform one download, and when that one is done, perform the other download? It makes the architecture much simpler, and the simpler the better. Your implementation also benefits from having less code, and less chance for bugs.

Answer 2

I don't really like your solution very much :(. The two calls/threads you are doing to the server are very dependent on each other and from my point of view things should be kept easy. What is the point of using two threads, were the data are "tightly coupled"?

When you retrive the ItemsCollection, can you at the same time retrive your Item data? (I mean in the same server call). That would be the perfect solution (JSON/XML with ItemCollection and Items), and you will avoid a lot of headaches.

How can you be sure that the items in each ItemCollection are (or will be) in the CoreData store? These are the kind of questions you will have to handle.

In case you can't get all the info a the same time, I would store the ItemCollection data temporaly in an struct until all the items from the first call are saved.

Regards, Angel