NSURLSession not downloading full data, but finnish with success

Question

Scenario: I am downloading some big attachments (30-50 mb) from EWS API, by using NSURLSession. And saving the downloded xml data into files.

I made HTTP class which uses NSURLSession, handles delegate callbacks and has a completion handler. The HTTP class creates its own NSURLSession and start downloading the data. Here is my HTTP.m

//
//  HTTP.m
//  Download
//
//  Created by Ankush Kushwaha on 7/6/18.
//  Copyright © 2018 Ankush Kushwaha. All rights reserved.
//

#import "HTTP.h"

typedef void (^httpCompletionBlock)(NSData* result);

@interface HTTP()

@property (nonatomic) NSMutableData * data;
@property (nonatomic) NSString *fileNametoSaved;
@property (nonatomic) httpCompletionBlock completion;

@end

@implementation HTTP

- (instancetype)initWithAttachmntId:(NSString *)attachmentId
                         fileName:(NSString *)fileName
                         completion:(void (^)(NSData* result))completion
{
    self = [super init];
    if (self) {
        self.data = [NSMutableData data];

        self.completion = completion;

        self.fileNametoSaved = fileName;

        NSURL *requestUrl = [NSURL URLWithString:@"https://outlook.office365.com/EWS/Exchange.asmx"];
        NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:requestUrl];
        request.HTTPMethod = @"POST";

        NSString *soapXmlString = [NSString stringWithFormat:@"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"
                                   "<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n"
                                   "xmlns:m=\"http://schemas.microsoft.com/exchange/services/2006/messages\"\n"
                                   "xmlns:t=\"http://schemas.microsoft.com/exchange/services/2006/types\"\n"
                                   "xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">\n"
                                   "<soap:Body>\n"
                                   "<m:GetAttachment>\n"
                                   "<m:AttachmentIds>\n"
                                   "<t:AttachmentId Id=\"%@\"/>\n"
                                   "</m:AttachmentIds>\n"
                                   "</m:GetAttachment>\n"
                                   "</soap:Body>\n"
                                   "</soap:Envelope>\n",attachmentId];
        if (soapXmlString)
        {
            NSString *xmlLength = [NSString stringWithFormat:@"%ld", (unsigned long)soapXmlString.length];
            request.HTTPBody = [soapXmlString dataUsingEncoding:NSUTF8StringEncoding];
            [request addValue:@"text/xml; charset=utf-8" forHTTPHeaderField:@"Content-Type"];
            [request addValue:xmlLength forHTTPHeaderField:@"Content-Length"];
        }

        dispatch_async(dispatch_get_main_queue(), ^{

            NSURLSessionConfiguration *defaultConfiguration = [NSURLSessionConfiguration ephemeralSessionConfiguration];

            NSURLSession *defaultSession = [NSURLSession sessionWithConfiguration:defaultConfiguration
                                                                         delegate:self
                                                                    delegateQueue:nil];

            NSURLSessionDataTask *dataTask = [defaultSession dataTaskWithRequest:request];

            [dataTask resume];
        });
    }
    return self;
}

-(void)URLSession:(NSURLSession *)session
             task:(NSURLSessionTask *)task
didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge
completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *))completionHandler
{

    if (challenge.previousFailureCount == 0)
    {
        NSURLCredential* credential;

        credential = [NSURLCredential credentialWithUser:@"MY_OUTLOOK.COM EMAIL" password:@"PASSWORD" persistence:NSURLCredentialPersistenceForSession];

        [[challenge sender] useCredential:credential forAuthenticationChallenge:challenge];

        completionHandler(NSURLSessionAuthChallengeUseCredential,credential);
    }
    else
    {
        // URLSession:task:didCompleteWithError delegate would be called as we are cancelling the request, due to wrong credentials.

        completionHandler(NSURLSessionAuthChallengeCancelAuthenticationChallenge, nil);
    }

}

-(void)URLSession:(NSURLSession *)session
         dataTask:(NSURLSessionDataTask *)dataTask
didReceiveResponse:(NSURLResponse *)response
completionHandler:(void (^)(NSURLSessionResponseDisposition))completionHandler
{
    completionHandler(NSURLSessionResponseAllow);
}

-(void)URLSession:(NSURLSession *)session
         dataTask:(NSURLSessionDataTask *)dataTask
   didReceiveData:(NSData *)data
{
    [self.data appendData:data];
//    NSLog(@"data : %lu", (unsigned long)self.data.length);

}

-(void)URLSession:(NSURLSession *)session
             task:(NSURLSessionTask *)task
didCompleteWithError:(NSError *)error
{

    NSLog(@"didCompleteWithError: %@", error);

    if (error)
    {

        NSLog(@"Error: %@", error);
    }
    else
    {
        NSData *data;

        if (self.data)
        {
            data = [NSData dataWithData:self.data];
        }

        NSLog(@"Success : %lu", (unsigned long)self.data.length);
        NSString  *filePath = [NSString stringWithFormat:@"/Users/startcut/Desktop/xxx/%@",
                               self.fileNametoSaved];
        NSString *xmlString = [[NSString alloc] initWithData:self.data encoding:NSUTF8StringEncoding];

        [xmlString writeToFile:filePath atomically:YES encoding:NSUTF8StringEncoding error:nil];

        self.completion ? self.completion(self.data) : nil;

    }

    [session finishTasksAndInvalidate]; // We must release the session, else it holds strong referance for it's delegate (in our case EWSHTTPRequest).
    // And it wont allow the delegate object to free -> cause memory leak
}

- (void)URLSession:(NSURLSession *)session
              task:(NSURLSessionTask *)task
willPerformHTTPRedirection:(NSHTTPURLResponse *)response
        newRequest:(NSURLRequest *)request
 completionHandler:(void (^)(NSURLRequest * _Nullable))completionHandler;

{

    NSString *redirectLocation = request.URL.absoluteString;

    if (response)
    {
        completionHandler(nil);
    }
    else
    {
        completionHandler(request); // new redirect request
    }
}

@end

In My ViewController I am making 5 HTTP requests, to download 5 diffrent attachments.

    HTTP *http = [[HTTP alloc] initWithAttachmntId:@"AAAaAGFua3VzaC5zdGFyY3V0QG91dGxvb2suY29tAEYAAAAAACd30qZd6oFAvoaMby5vOMUHAOsTbManU6VPoeQkUTl4/J0AAWUo5o0AAOsTbManU6VPoeQkUTl4/J0AAWruTj0AAAESABAAWGs6REUQc02OHF0x6uYJ+g=="
                                      fileName:@"http1"
                                    completion:^(NSData *result) {
                                        NSLog(@"Completion 1");
                                    }];
HTTP *http2 = [[HTTP alloc] initWithAttachmntId:@"AAAaAGFua3VzaC5zdGFyY3V0QG91dGxvb2suY29tAEYAAAAAACd30qZd6oFAvoaMby5vOMUHAOsTbManU6VPoeQkUTl4/J0AAWUo5o0AAOsTbManU6VPoeQkUTl4/J0AAWruTjsAAAESABAAP8zebUI1fkSiE8tQ+RtwiQ=="
                                       fileName:@"http2"
                                    completion:^(NSData *result) {
                                        NSLog(@"Completion 2");
                                    }];

HTTP *http3 = [[HTTP alloc] initWithAttachmntId:@"AAAaAGFua3VzaC5zdGFyY3V0QG91dGxvb2suY29tAEYAAAAAACd30qZd6oFAvoaMby5vOMUHAOsTbManU6VPoeQkUTl4/J0AAWUo5o0AAOsTbManU6VPoeQkUTl4/J0AAWruTjkAAAESABAAiPaJIPjp/k6iQHSMpi6aDw=="
                                       fileName:@"http3"
                                     completion:^(NSData *result) {
                                         NSLog(@"Completion 3");
                                     }];

HTTP *http4 = [[HTTP alloc] initWithAttachmntId:@"AAAaAGFua3VzaC5zdGFyY3V0QG91dGxvb2suY29tAEYAAAAAACd30qZd6oFAvoaMby5vOMUHAOsTbManU6VPoeQkUTl4/J0AAWUo5o0AAOsTbManU6VPoeQkUTl4/J0AAWruTjwAAAESABAA86vBkFlTNU2oEVq/eRtLGQ=="
                                       fileName:@"http4"
                                     completion:^(NSData *result) {
                                         NSLog(@"Completion 4");
                                     }];
HTTP *http5 = [[HTTP alloc] initWithAttachmntId:@"AAAaAGFua3VzaC5zdGFyY3V0QG91dGxvb2suY29tAEYAAAAAACd30qZd6oFAvoaMby5vOMUHAOsTbManU6VPoeQkUTl4/J0AAWUo5o0AAOsTbManU6VPoeQkUTl4/J0AAWruTjoAAAESABAAND6qbOQbnkyoyg0K17T9/Q=="
                                       fileName:@"http5"
                                     completion:^(NSData *result) {
                                         NSLog(@"Completion 5");
                                     }];

Problem: As the files or data are being downloaded parallelly with 5 separate HTTP objects, At the end when NSUrlSession session delegate gets called I save data into files in my HTTP.m's -(void)URLSession (NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error Method. Most of the times the downloaded data (files) does not contain the full data (e.g If the Size of the attachment is 30 mb, my code downloads the data 4 mb or 10 mb or 3.2 mb etc. The numbers are not consistent). It seems that NSURLSession terminates or stop the data downloading in between and close the connection successfully. If I download 1 attachment at a time (Instead of making 5 HTTP objects in my view controller, I just make 1 object at a time) in most of the cases it works and downloads full data content.

Any help is appreciated guys. I am stuck in this from 2 days.

Answer 1

Finally. I found the cause. Not the solution :(

It was not from iOS code. There might be some code improvement needed as @dgatwood mentioned (Thanks), but even after improvements then I was facing the same problem.

Actually, the EWS exchange is getting throttled by large data download. Due to which EWS server terminates the connection in between. Here is the blog

Answer 2

In no particular order:

• You should not be creating a new session for each request. That prevents the OS from limiting the number of simultaneous requests correctly, and will likely cause other issues down the road. Similarly, you should not be calling finishTasksAndInvalidate after each task completes.
• You must retain a reference to the session until there are no more outstanding requests. If that doesn't fit easily into your app's architecture, you might consider using the default session instead of providing your own session.
• Your Content-Length header value is incorrect. It should be a byte count, not a character count. Convert the string to an NSData with encoding first, and send the length of that as Content-Length. Otherwise, it will fail as soon as you get a single multi-byte character in the body.
• Your didReceiveResponse: method should ideally be clearing your data storage so that it handles multipart responses correctly (with the last one winning), rather than concatenating them.
• Your authentication challenge handler, as written, is likely to cause serious problems. You should be checking the protection space of the challenge to see if it is one that you care about, and if not, you should be triggering default handling. Without that your app will fail if the user is behind any sort of proxy, among other things.

Fix those issues, and if it still isn't working, ask a new question about whatever is still not working. :-)