downloading data from several pffile's at once asynchronosly

Question

If I have an array of Message objects, each with a PFile containing data, is it possible to download the data for every single message by queuing them up asynchronously like so:

for (int i = 0; i < _downloadedMessages.count; i++) {
    PFObject *tempMessage = (PFObject *)[_downloadedMessages objectAtIndex:i];    
    [[tempMessage objectForKey:@"audio"] getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
            [self persistNewMessageWithData:data];
    }];
}

This seems to cause my app to hang, even though this should be done in the background...

Using the solution below:

NSMutableArray* Objects = ...

[self forEachPFFileInArray:Objects retrieveDataWithCompletion:^BOOL(NSData* data, NSError*error){
    if (data) {
        PFObject *tempObj = (PFObject *)Object[someIndex...];
        [self persistNewMessageWithData:data andOtherInformationFromObject:tempObj];
        return YES;
    }
    else {
         NSLog(@"Error: %@", error);
         return NO; // stop iteration, optionally continue anyway
    }
} completion:^(id result){
     NSLog(@"Loop finished with result: %@", result);    
}];

Answer 1

What you are possibly experiencing is, that for a large numbers of asynchronous requests which run concurrently, the system can choke due to memory pressure and due to network stalls or other accesses of resources that get exhausted (including CPU).

You can verify the occurrence of memory pressure using Instruments with the "Allocations" tool.

Internally (that is, in the Parse library and the system) there might be a variable set which sets the maximum number of network requests which can run concurrently. Nonetheless, in your for loop you enqueue ALL requests.

Depending of what enqueuing a request means in your case, this procedure isn't free at all. It may cost a significant amount of memory. In the worst case, the network request will be enqueued by the system, but the underlying network stack executes only a maximum number of concurrent requests. The other enqueued but pending requests hang there and wait for execution, while their network timeout is already running. This may lead to cancellation of pending events, since their timeout expired.

The simplest Solution

Well, the most obvious approach solving the above issues would be one which simply serializes all tasks. That is, it only starts the next asynchronous task when the previous has been finished (including the code in your completion handler). One can accomplish this using an asynchronous pattern which I name "asynchronous loop":

The "asynchronous loop" is asynchronous, and thus has a completion handler, which gets called when all iterations are finished.

typedef void (^loop_completion_handler_t)(id result);
typedef BOOL (^task_completion_t)(PFObject* object, NSData* data, NSError* error);

- (void) forEachObjectInArray:(NSMutableArray*) array 
   retrieveDataWithCompletion:(task_completion_t)taskCompletionHandler
   completion:(loop_completion_handler_t)completionHandler 
{
    // first, check termination condition:
    if ([array count] == 0) {
        if (completionHandler) {
            completionHandler(@"Finished");
        }
        return;
    }
    // handle current item:
    PFObject* object = array[0];
    [array removeObjectAtIndex:0];
    PFFile* file = [object objectForKey:@"audio"];
    if (file==nil) {
        if (taskCompletionHandler) {
            NSDictionary* userInfo = @{NSLocalizedFailureReasonErrorKey: @"file object is nil"}
            NSError* error = [[NSError alloc] initWithDomain:@"RetrieveObject"
                                                        code:-1 
                                                    userInfo:userInfo]; 
            if (taskCompletionHandler(object, nil, error)) {                    
                // dispatch asynchronously, thus invoking itself is not a recursion
                dispatch_async(dispatch_get_global(0,0), ^{ 
                    [self forEachObjectInArray:array 
                    retrieveDataWithCompletion:taskCompletionHandler
                             completionHandler:completionHandler];
                });
            }
            else {
                if (completionHandler) {
                    completionHandler(@"Interuppted");
                }
            }
        }
    }
    else {
        [file getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
            BOOL doContinue = YES;
            if (taskCompletionHandler) {
                doContinue = taskCompletionHandler(object, data, error);
            }
            if (doContinue) {
                // invoke itself (note this is not a recursion")
                [self forEachObjectInArray:array 
                retrieveDataWithCompletion:taskCompletionHandler
                         completionHandler:completionHandler];
            }
            else {
                if (completionHandler) {
                    completionHandler(@"Interuppted");
                }
            }
        }];
    }
}

Usage:

// Create a mutable array 
NSMutableArray* objects = [_downloadedMessages mutableCopy];

[self forEachObjectInArray:objects 
retrieveDataWithCompletion:^BOOL(PFObject* object, NSData* data, NSError* error){
    if (error == nil) {
        [self persistNewMessageWithData:data andOtherInformationFromObject:object];
        return YES;
    }
    else {
         NSLog(@"Error %@\nfor PFObject %@ with data: %@", error, object, data);
         return NO; // stop iteration, optionally continue anyway
    }
} completion:^(id result){
     NSLog(@"Loop finished with result: %@", result);    
}];