Synchronising old data with NSPersistentCloudKitContainer

Question

I'm using NSPersistentCloudKitContainer to synchronise data between different devices with CloudKit. It works perfectly well with a new project, however when I'm using it with old projects the old data which was added with NSPersistentContainer does not synchronise.

What I would like to achieve is to synchronise old data that was added with NSPersistentContainer after changing it to NSPersistentCloudKitContainer. Is it possible?

Answer 1

This is an old question, but if you must support iOS 13 to 16 versions and want to stay away from the complexity of a full-blown CloudKit implementation, you need NSPersistentCloudKitContainer and I couldn’t find a good answer to this issue it anywhere else, so here it is my approach for those that are still looking for a similar solution.

Problem

As described in the question, the problem is that records created prior to setting up NSPersistentCloudKitContainer and the respective cloud store are not going to sync, but adding a “sync with iCloud” property is not reliable. This works in controlled scenarios, but even with NSPersistentHistoryTrackingKey set as true, if the user logs off iCloud, deletes the app and reinstalls it, or switches iCloud accounts, you are likely to see inconsistencies over time. And when you have hundreds of thousands of users, weird things happen.

Solution

TL;DR: Use a simple CloudKit function to read the synced records created with NSPersistentCloudKitContainer on iCloud and compare them to the app’s local records. If a local record is missing on the cloud, delete it on the app and add it again, this will trigger the proper cloud sync. That’s it. If you know CloudKit and know your way around the CloudKit Console, you’re good to go.

Details:

Create a new iCloud container in Xcode under [your target]>iCloud>Signing & Capabilities. I chose not to reuse the existing one to avoid propagating old sync issues and having unexpected behaviors.

Setup your NSPersistentCloudKitContainer and run the app for the first time so it can populate the new iCloud container on your developer account.

// Set these two keys so when the user turns on iCloud sync, all Core Data changes made during off-sync are synced to iCloud:
            cloudStoreDescription.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
            cloudStoreDescription.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey)

// Set these two properties for prompt updates and merge priority: 
        container.viewContext.automaticallyMergesChangesFromParent = true
        container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy

Go to your CloudKit Console and into the newly created cloud container. Under Schema and Record Types, check the name of the record type that contains your record fields. It should have a “CD_” prefix on the name of your data model entity. Let’s call it here “CD_Entity”.

Under “CD_Entity” check the name of the record field that you will use for your NSPredicate. It also has a “CD_” prefix, let’s call it “CD_id”. Note: You can’t use NSPredicate(value: true) because the automatically created recordName is not Queryable and this approach focuses on not editing the synced iCloud container on the console or the app.

To fetch all records from the iCloud container use this:

func fetchCloudStoreRecords(container: String, recordType: String, completion: @escaping ([CKRecord]?, Error?) -> Void) {
    let ckContainer = CKContainer(identifier: “your.container.name”)
    let query = CKQuery(recordType: “CD_Entity”, predicate: NSPredicate(format: "CD_id > 0"))
    let db = ckContainer.privateCloudDatabase
    
    db.perform(query, inZoneWith: nil) { (records, error) in
        completion(records, error)
    }
}

After mapping your cloud and local fields, compare the cloud records with your local records by the id or whatever field you use to differentiate each record. The subset of local records that are not in the cloud are the ones that can’t sync. All you have to do is insert another object with the exact properties and then delete the original local record:

func resyncObject(_ origObj: Entity) {
    let context = fetchedResultsController.managedObjectContext
    guard let entity = fetchedResultsController.fetchRequest.entity else { return }
    
    let obj = Entity(entity: entity, insertInto: context)
    
    obj.id = origObj.id
    
        .
        .
        .
    // Map all your fields
    
    managedObjectContext?.delete(origObj)
    managedObjectContext?.save()
}

The new record will now be synced with both the local and cloud stores.

Answer 2

I've found a solution that works for my Core Data database - and mine is quite complex with multiple many-to-many relationships (A surgery/anaesthesia logbook app called Somnus)

I started by creating a new attribute for all my Core Data Entities called sentToCloud and setting it to FALSE by default in the Core Data model.

On the first load for an existing user:

• Fetch request using the predicate "sentToCloud == FALSE" for each Entity type
• Change sentToCloud to TRUE for each Object then save the MOC
• This triggers NSPersistentCloudKitContainer to start syncing

I've done this in order of 'priority' that works for my database, assuming the iCloud sync sessions match the order in which Core Data is modified. In my testing this seems to be the case:

• I first sync all child (or most child-like) Entities
• Then sync their parents, and so on, up the tree
• I sync the Object the user interacts with last, once everything else is in place so the relationships are intact and they don't think their data is borked while we wait for NSPersistentCloudKitContainer to reconnect all the relationships
• I also leave any binary data (magically turned into a CKAsset behind-the-scenes) to last as it's not the most important part of my database

My database synced successfully from my iPad to iPhone and all relationships and binary data appear correct.

Now all I need is a way to tell the user when data is syncing (and/or some sort of progress) and for them to turn it off entirely.

ADDENDUM

So I tried this again, after resetting all data on the iCloud dashboard and deleting the apps on my iPhone & iPad.

Second time around it only synced some of the data. It seems like it still has a problem dealing with large sync requests (lots of .limitExceeded CKErrors in the console).

What's frustrating is that it's not clear if it's breaking up the requests to try again or not - I don't think it is. I've left it overnight and still no further syncing, only more .limitExceeded CKErrors.

Maybe this is why they don't want to sync existing data?

Personally, I think this is silly. Sometimes users will do a batch process on their data which would involve updating many thousands of Core Data objects in one action. If this is just going to get stuck with .limitExceeded CKErrors, NSPersistentCloudKitContainer isn't going to be a very good sync solution.

They need a better way of dealing with these errors (breaking up the requests into smaller requests), plus the ability to see what's going on (and perhaps present some UI to the user).

I really need this to work because as it stands, there is no way to synchronise many-to-many Core Data relationships using CloudKit.

I just hope that they're still working on this Class and improving it.