Deleting/removing records in Core Data very very slow

Question

I am putting the finishing touches on an App, and have difficulties with removing records in bulk. On the hit of a button a set of approx. 3500 records need to be added to the database. This is not a problem, and takes approx. 3-4 seconds.

But sometimes (not often, but the option needs to be there) all these records need to be removed. I just ran this operation, and it took 20 minutes. What could be wrong here? There is only one dependency, all the records are children of a particular Collection.

I add all the items to a set, removed them from the Collection and then delete one by one. Every 5% I update the dialogue and when everything is done I commit the changes. But removing the items just takes ages (as I can see the progress dialogue progress very slowly)

- (void) deleteList:(DOCollection *) collection {

    // For the progress dialogue
    NSMutableDictionary* dict = [NSMutableDictionary dictionaryWithObject:@"Clearing vocabulary list!" forKey:@"message"];
    float totalItems = [collection.items count];
    float progress = 0;
    float nextProgressRefresh = 0.05;         

    NSMutableSet* itemsSet = [NSMutableSet set];
    for (DOItem* item in collection.items) {
        [itemsSet addObject:(NSNumber*)[NSNumber numberWithInt:[item.itemId intValue]]];
    }

    // Remove all of them from the collection
    [managedObjectContext performBlockAndWait:^{
        [collection setItems:[NSSet set]];
    }];    

    for (NSNumber* itemId in itemsSet) {
        DOItem* item = [itemController findItem:[itemId intValue]];
        if (item != nil) {
           [[self itemController] removeItem:item];
        }

        progress++;
        if((nextProgressRefresh < (progress / totalItems))){  
            NSString* sProgress = [NSString stringWithFormat:@"%f", (progress / totalItems) * 0.85];
            //[dict setValue:@"Saving the database...!" forKey:@"message"];
            [dict setValue:sProgress forKey:@"progress"];
            [[NSNotificationCenter defaultCenter] postNotificationName:kUpdatePleaseWaitDialogue object:dict]; 
            nextProgressRefresh = nextProgressRefresh + 0.05;
        }        
    }

    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
        [managedObjectContext performBlockAndWait:^{
            [[self collectionController] commitChanges];
        }];
        [[NSNotificationCenter defaultCenter] postNotificationName:kSavingDataComplete object:nil]; 
    });  

    //NSLog(@"Wait 2!");
    [NSThread sleepForTimeInterval:1];    

}

in DOItemController:

- (void) removeItem: (NSManagedObject*) item {
    [[self managedObjectContext] deleteObject:item];
}

Answer 1

Not sure about how you architected your data models. But I would set it up to cascade delete your objects. If the DOItem Objects are unique to the DOCollection, then you can set the delete rule to cascade. That will automagically delete the associated DOItem as well as remove it from the DOCollection item set object.

To delete the DOItem objects from the DOCollection, check your DOCollection.h file, you should have a method along the lines of

-(void)removeDOItemObjects:(NSSet *)value

if not, they may still be dynamically generated by Core Data for you. In your header file you should have something along the lines of:

@property(nonatomic,retain) DOItem *items

then in your implementation file, you should have something along the lines of:

@synthesize items

The appropriate methods that should be generated for these automatically:

-(void)addItemsObject:(DOItem*)value
-(void)addItems:(NSSet *)values
-(void)removeItemsObject:(DOItem *)value
-(void)removeItems:(DOItem *)values
-(NSSet *)items

See "Custom To-Many Relationship Accessor Methods" here for more info.

This method is provided for you when you create the data model and associated implementation files and should be highly optimized by Core Data. Then all you have to do to remove the objects is something along the lines of:

- (void) deleteList:(DOCollection *) collection {

    // Remove all of them from the collection
    [managedObjectContext performBlockAndWait:^{
        // Updated 01/10/2012
        [collection removeItems:collection.items];
        NSError *error = nil;
        if (![managedObjectContext save:&error]) { 
           NSLog(@"Core Data: Error saving context."); }
       };
    }];

}

You might want to check the performance of the delete using this method and continue to provide the user with feedback. If performance is an issue, consider striding, where you divide the set up into chunks and perform the above method fore each one of the strides, update the user interface, etc.

Again, am not sure about your application architecture, but on first glance, this looks like the issue.

Good Luck!