iOS Swift download lots of small files in background

Question

In my app I need to download files with the following requirements:

• download lots (say 3000) of small PNG files (say 5KB)
• one by one
• continue the download if the app in background
• if an image download fails (typically because the internet connection has been lost), wait for X seconds and retry. If it fails Y times, then consider that the download failed.
• be able to set a delay between each download to reduce the server load

Is iOS able to do that? I'm trying to use NSURLSession and NSURLSessionDownloadTask, without success (I'd like to avoid starting the 3000 download tasks at the same time).

EDIT: some code as requested by MwcsMac:

ViewController:

class ViewController: UIViewController, URLSessionDelegate, URLSessionDownloadDelegate {

    // --------------------------------------------------------------------------------
    // MARK: Attributes

    lazy var downloadsSession: URLSession = {

        let configuration = URLSessionConfiguration.background(withIdentifier:"bgSessionConfigurationTest");
        let session = URLSession(configuration: configuration, delegate: self, delegateQueue:self.queue);

        return session;
    }()

    lazy var queue:OperationQueue = {

        let queue = OperationQueue();
        queue.name = "download";
        queue.maxConcurrentOperationCount = 1;

        return queue;
    }()

    var activeDownloads = [String: Download]();

    var downloadedFilesCount:Int64 = 0;
    var failedFilesCount:Int64 = 0;
    var totalFilesCount:Int64 = 0;

    // --------------------------------------------------------------------------------



    // --------------------------------------------------------------------------------
    // MARK: Lifecycle

    override func viewDidLoad() {

        super.viewDidLoad()

        startButton.addTarget(self, action:#selector(onStartButtonClick), for:UIControlEvents.touchUpInside);

        _ = self.downloadsSession
        _ = self.queue
    }

    // --------------------------------------------------------------------------------



    // --------------------------------------------------------------------------------
    // MARK: User interaction

    @objc
    private func onStartButtonClick() {

        startDownload();
    }

    // --------------------------------------------------------------------------------



    // --------------------------------------------------------------------------------
    // MARK: Utils

    func startDownload() {

        downloadedFilesCount = 0;
        totalFilesCount = 0;

        for i in 0 ..< 3000 {

            let urlString:String = "http://server.url/\(i).png";
            let url:URL = URL(string: urlString)!;

            let download = Download(url:urlString);
            download.downloadTask = downloadsSession.downloadTask(with: url);
            download.downloadTask!.resume();
            download.isDownloading = true;
            activeDownloads[download.url] = download;

            totalFilesCount += 1;
        }
    }

    // --------------------------------------------------------------------------------



    // --------------------------------------------------------------------------------
    // MARK: URLSessionDownloadDelegate

    func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {

        if(error != nil) { print("didCompleteWithError \(error)"); }

        failedFilesCount += 1;
    }

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {

        if let url = downloadTask.originalRequest?.url?.absoluteString {

            activeDownloads[url] = nil
        }

        downloadedFilesCount += 1;

        [eventually do something with the file]

        DispatchQueue.main.async {

            [update UI]
        }

        if(failedFilesCount + downloadedFilesCount == totalFilesCount) {

            [all files have been downloaded]
        }
    }

    // --------------------------------------------------------------------------------



    // --------------------------------------------------------------------------------
    // MARK: URLSessionDelegate

    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {

        if let appDelegate = UIApplication.shared.delegate as? AppDelegate {

            if let completionHandler = appDelegate.backgroundSessionCompletionHandler {

                appDelegate.backgroundSessionCompletionHandler = nil

                DispatchQueue.main.async { completionHandler() }
            }
        }
    }

    // --------------------------------------------------------------------------------
}

AppDelegate:

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    var window: UIWindow?
    var backgroundSessionCompletionHandler: (() -> Void)?

    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
        // Override point for customization after application launch.
        return true
    }

    func applicationWillResignActive(_ application: UIApplication) {
        // Sent when the application is about to move from active to inactive state. This can occur for certain types of temporary interruptions (such as an incoming phone call or SMS message) or when the user quits the application and it begins the transition to the background state.
        // Use this method to pause ongoing tasks, disable timers, and invalidate graphics rendering callbacks. Games should use this method to pause the game.
    }

    func applicationDidEnterBackground(_ application: UIApplication) {
        // Use this method to release shared resources, save user data, invalidate timers, and store enough application state information to restore your application to its current state in case it is terminated later.
        // If your application supports background execution, this method is called instead of applicationWillTerminate: when the user quits.
    }

    func applicationWillEnterForeground(_ application: UIApplication) {
        // Called as part of the transition from the background to the active state; here you can undo many of the changes made on entering the background.
    }

    func applicationDidBecomeActive(_ application: UIApplication) {
        // Restart any tasks that were paused (or not yet started) while the application was inactive. If the application was previously in the background, optionally refresh the user interface.
    }

    func applicationWillTerminate(_ application: UIApplication) {
        // Called when the application is about to terminate. Save data if appropriate. See also applicationDidEnterBackground:.
    }

    func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {

        backgroundSessionCompletionHandler = completionHandler
    }
}

Download:

class Download: NSObject {

    var url: String
    var isDownloading = false
    var progress: Float = 0.0

    var downloadTask: URLSessionDownloadTask?
    var resumeData: Data?

    init(url: String) {
        self.url = url
    }
}

What's wrong with this code:

• I'm not sure that the background part is working. I followed this tutorial: https://www.raywenderlich.com/110458/nsurlsession-tutorial-getting-started. It says that the app screenshot should be updated if I press home and then double tap home to show the app switcher. Does not seems to work reliably. It is updated when I re-open the app though. Having an iPhone since yesterday, I don't know if this is the normal behavior?
• the 3000 downloads are started in the startDownload method. The maxConcurrentOperationCount of the queue does not seem to be respected: downloads are running concurrently
• the downloadsSession.downloadTask(with: url); call takes 30ms. Multiplied by 3000, it takes 1mn30, that's a big problem :/ . Waiting a few seconds (2-3) would be ok.
• I can't set a delay between two downloads (that's not a big problem. Would be nice though, but if I can't it will be OK)

Ideally, I would run the startDownload method asynchronously, and download the files synchronously in the for loop. But I guess I can't do that in background with iOS?

Answer 1

So here is what I finally did:

• start the download in a thread, allowed to run for a few minutes in background (with UIApplication.shared.beginBackgroundTask)
• download files one by one in a loop with a custom download method allowing to set a timeout
• before downloading each file, check if UIApplication.shared.backgroundTimeRemaining is greater than 15
• if yes, download the file with a timeout of min(60, UIApplication.shared.backgroundTimeRemaining - 5)
• if no, stop downloading and save the download progress in the user defaults, in order to be able to resume it when the user navigates back to the app
• when the user navigates back to the app, check if a state has been saved, and if so resume the download.

This way the download continues for a few minutes (3 on iOS 10) when the user leaves the app, and is pause just before these 3 minutes are elapsed. If the user leaves the app in background for more than 3 minutes, he must come back to finish the download.