Paging results from Core Data requests

Question

I have a relatively simple core data sqlite database. I am trying to get results from DB one page at a time.

    NSFetchRequest* request = [[[NSFetchRequest alloc] init] autorelease];
    [request setEntity:[...]];

    [request setPredicate:[NSPredicate predicateWithFormat:@"flaggedTime != nil"]];

    NSSortDescriptor* sortDescriptor = [NSSortDescriptor sortDescriptorWithKey:@"flaggedTime" ascending:NO];
    [request setSortDescriptors:[NSArray arrayWithObject:sortDescriptor]];

    [request setFetchLimit:pageSize];
    [request setFetchOffset:((pageIndex - 1) * pageSize)];    

    NSArray* results = [self.context executeFetchRequest:request error:NULL];

pageSize is 30, pageIndex on testing data is 1, 2, 3 or 4 (there are about 80 items in DB, so pageIndex = 4 should return no items). Predicate and sorting works fine, results are successfully returned. Fetch limit works fine, too. No errors are returned.

Problem: I always get results from the first page, as if fetchOffset was not set. I tried to remove the predicate and the sorting but to no avail. The only situation when I could make fetchOffset work was when I used values under 30. Of course, that is meaningless for paging...

Does anybody know why? I will be really thankful for every answer.

Update: I am speaking about iOS. Tested on 4.2 and 5.0.

Update 2: To simplify the problem.

    NSFetchRequest* request = [[[NSFetchRequest alloc] init] autorelease];
    [request setEntity:[...];

    NSError* error = nil;

    NSManagedObjectContext* context = [...];

    NSUInteger count = [context countForFetchRequest:request error:&error];
    assert(error == nil);

    NSLog(@"Total count: %u", count);

    request.fetchOffset = 0;    
    request.fetchLimit = 30;

    NSLog(@"Fetch offset: %u, limit: %u", request.fetchOffset, request.fetchLimit);

    NSArray* page1 = [context executeFetchRequest:request error:&error];
    assert(error == nil);

    NSLog(@"Page 1 count: %u", page1.count);

    request.fetchOffset = 30;    
    request.fetchLimit = 30;

    NSLog(@"Fetch offset: %u, limit: %u", request.fetchOffset, request.fetchLimit);

    NSArray* page2 = [context executeFetchRequest:request error:&error];
    assert(error == nil);

    NSLog(@"Page 2 count: %u", page2.count);

gives:

Total count: 34
Fetch offset: 0, limit: 30
Page 1 count: 30
Fetch offset: 30, limit: 30
Page 2 count: 30 (ERROR: should give 4)

Answer 1

Core Data has built in paging and it's super simple, just set fetchBatchSize on the fetch request. When set the fetch just queries for all the objects as faults, basically just a pointer and its row index which is minimal memory into a special kind of array called a batch faulting array. I'd imagine this is fine in most cases although I haven't looked into exactly how much memory it would use for a huge number of rows. Then when you loop this array when you access a faulted record's property it queries the database for that record and the next records up to the number you set for the batch size. This allows you to loop your results one by one as normal, but in the background it loads in the data in batches. As of iOS 9.2 the minimum batch size is 4 so no point setting a number lower than that.

Answer 2

The pageOffset property of NSFetchRequest is (sometimes?) ignored when you run fetch requests against an unsaved context. Note how in the code by OP context is never saved, while in the code snippet attached to an answer by @kcharwood it is actually saved. Here is an example:

- (void) printArrayOfTestEntities:(NSArray *)array{
    NSMutableString * s = [NSMutableString new];
    NSArray * numbers = [array valueForKey:@"someField"];
    for (NSNumber * number in numbers){
        [s appendString:[NSString stringWithFormat:@"%d ", [number intValue]]];
    }

    NSLog(@"fetched objects: %@ \rcount: %d", s, array.count);
}

/* example itself */

NSManagedObjectContext *context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
[context setPersistentStoreCoordinator:persistentStoreCoordinator];

for(int i = 0; i < 34;i++){
    NSManagedObject * object = [NSEntityDescription insertNewObjectForEntityForName:@"TestEntity"
                                                      inManagedObjectContext:context];
    [object setValue:@(i) forKey:@"someField"];
}

NSFetchRequest * request = [NSFetchRequest fetchRequestWithEntityName:@"TestEntity"];
request.sortDescriptors = @[[NSSortDescriptor sortDescriptorWithKey:@"someField" ascending:YES]];
request.fetchLimit = 30;

NSArray * result = [context executeFetchRequest:request error:nil];
[self printArrayOfTestEntities:result];

request.fetchOffset = 30;
result = [context executeFetchRequest:request error:nil];
[self printArrayOfTestEntities:result];

[context save:&error];

request.fetchOffset = 0;
result = [context executeFetchRequest:request error:nil];
[self printArrayOfTestEntities:result];

request.fetchOffset = 30;
result = [context executeFetchRequest:request error:nil];
[self printArrayOfTestEntities:result];

The log:

2014-03-01 11:30:06.986 coredatatest[19771:70b] fetched objects: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29  
count: 30
2014-03-01 11:30:06.990 coredatatest[19771:70b] fetched objects: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29  
count: 30
2014-03-01 11:30:06.995 coredatatest[19771:70b] fetched objects: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29  
count: 30
2014-03-01 11:30:06.997 coredatatest[19771:70b] fetched objects: 30 31 32 33   
count: 4

Answer 3

I just created a demo project that tries to recreate your scenario at its simplest form. I created a blank project, added 34 objects, then queried it with the exact same code you listed above. Below is the example:

CDAppDelegate * delegate = (CDAppDelegate*)[[UIApplication sharedApplication] delegate];
NSManagedObjectContext * context = [delegate managedObjectContext];
for(int i = 0; i < 34;i++){
    CDObject * object = [NSEntityDescription insertNewObjectForEntityForName:@"CDObject"
                                                      inManagedObjectContext:context];
    [object setValue:i];
}
[delegate saveContext];

NSFetchRequest* request = [[[NSFetchRequest alloc] init] autorelease];
[request setEntity:[NSEntityDescription entityForName:@"CDObject"
                               inManagedObjectContext:context]];

 NSError* error = nil;

 NSUInteger count = [context countForFetchRequest:request error:&error];
 assert(error == nil);

 NSLog(@"Total count: %u", count);

 request.fetchOffset = 0;    
 request.fetchLimit = 30;

 NSLog(@"Fetch offset: %u, limit: %u", request.fetchOffset, request.fetchLimit);

 NSArray* page1 = [context executeFetchRequest:request error:&error];
 assert(error == nil);

 NSLog(@"Page 1 count: %u", page1.count);

 request.fetchOffset = 30;    
 request.fetchLimit = 30;

 NSLog(@"Fetch offset: %u, limit: %u", request.fetchOffset, request.fetchLimit);

 NSArray* page2 = [context executeFetchRequest:request error:&error];
 assert(error == nil);

 NSLog(@"Page 2 count: %u", page2.count);

[request release];

The log is as shown:

2011-11-04 14:53:04.530 CDCoreDataTest[77964:207] Total count: 34
2011-11-04 14:53:04.531 CDCoreDataTest[77964:207] Fetch offset: 0, limit: 30
2011-11-04 14:53:04.532 CDCoreDataTest[77964:207] Page 1 count: 30
2011-11-04 14:53:04.533 CDCoreDataTest[77964:207] Fetch offset: 30, limit: 30
2011-11-04 14:53:04.533 CDCoreDataTest[77964:207] Page 2 count: 4

Using your code, I was able to get it up and running no problem. This was done with iOS 5.0 running on the simulator. Your code looks right to me for you are trying to accomplish, so there must be something going on with the fetch request or the context itself...