Android worker - update and preserve state across retries

Question

Kotlin/Android novice here :). I'm playing around with chunked uploads using a CoroutineWorker and don't see a built-in way to maintain state for my worker in case a retry happens, but I'm having sort of a hard time believing smth like that would be missing...

My use case is the following:

1. Create the worker request with the path to the file to upload as input data
2. Worker loops over the file and performs uploads in chunks. The latest uploaded chunkIndex is being tracked.
3. In case of an error and subsequent Retry(), the worker somehow retrieves the current chunk index and resumes rather than starting from at the beginning again.

So basically, I really just need to preserve that chunkIndex flag. I looked into setting progress, but this seems to be hit or miss on retries (worked once, wasn't available on another attempt).

override suspend fun doWork(): Result {
    try {
        // TODO check if we are resuming with a given chunk index
        chunkIndex = ...

        // do the work
        performUpload(...)

        return Result.success()

    } catch (e: Exception) {
        // TODO cache the chunk index


        return Result.retry()
    }
}

Did I overlook something, or would I really have to store that index outside the worker?

Answer 1

As stated in GB's answer, there seems to be no way to cache data with in the worker, or do a Result.retry(data). I ended up just doing a quick hack with SharedPreferences instead.

Solution below. Take it with a grain of salt, I have a total of ~10 hours of Kotlin under my belt ;)

var latestChunkIndex = -1

override suspend fun doWork(): Result = withContext(Dispatchers.IO) {
    try {
        // get cached entry (simplified - no checking for fishy status or anything)
        val transferId = id.toString()
        var uploadInfo: UploadInfo = TransferCache.tryGetUpload(applicationContext, transferId) ?: TransferCache.registerUpload(applicationContext, transferId, TransferStatus.InProgress)

        if(uploadInfo.status != TransferStatus.InProgress) {
            TransferCache.setUploadStatus(applicationContext, transferId, TransferStatus.InProgress)
        }

        // resolve the current chunk - this will allow us to resume in case we're retrying
        latestChunkIndex = uploadInfo.latestChunkIndex

        // do the actual work
        upload()

        // update status and complete
        TransferCache.setUploadStatus(applicationContext, id.toString(), TransferStatus.Success)
        Result.success()
    } catch (e: Exception) {
        if (runAttemptCount > 20) {
            // give up
            TransferCache.setUploadStatus(applicationContext, id.toString(), TransferStatus.Error)
            Result.failure()
        }

        // update status and schedule retry
        TransferCache.setUploadStatus(applicationContext, id.toString(), TransferStatus.Paused)
        Result.retry()
    }
}

Within my upload function, I'm simply keeping track of my cache (I could also just do it in the exception handler of the doWork method, but I'll use the cache entry for status checks as well, and it's cheap):

private suspend fun upload() {
    while ((latestChunkIndex + 1) * defaultChunkSize < fileSize) {

        // doing the actual upload
        ...

        // increment chunk number and store as progress
        latestChunkIndex += 1
        TransferCache.cacheUploadProgress(applicationContext, id.toString(), latestChunkIndex)
    }
}

and the TransferCache looking like this (note that there is no housekeeping there, so without cleanup, this would just continue to grow!)

class UploadInfo() {
    var transferId: String = ""
    var status: TransferStatus = TransferStatus.Undefined
    var latestChunkIndex: Int = -1

    constructor(transferId: String) : this() {
        this.transferId = transferId
    }
}


object TransferCache {

    private const val PREFERENCES_NAME = "${BuildConfig.APPLICATION_ID}.transfercache"
    private val gson = Gson()

    fun tryGetUpload(context: Context, transferId: String): UploadInfo? {
        return getPreferences(context).tryGetUpload(transferId);
    }


    fun cacheUploadProgress(context: Context, transferId: String, transferredChunkIndex: Int): UploadInfo {
        getPreferences(context).run {
            // get or create entry, update and save
            val uploadInfo = tryGetUpload(transferId)!!
            uploadInfo.latestChunkIndex = transferredChunkIndex
            return saveUpload(uploadInfo)
        }
    }


    fun setUploadStatus(context: Context, transferId: String, status: TransferStatus): UploadInfo {
        getPreferences(context).run {
            val upload = tryGetUpload(transferId) ?: registerUpload(context, transferId, status)
            if (upload.status != status) {
                upload.status = status
                saveUpload(upload)
            }

            return upload
        }
    }


    /**
     * Registers a new upload transfer. This would simply (and silently) override any
     * existing registration.
     */
    fun registerUpload(context: Context, transferId: String, status: TransferStatus): UploadInfo {
        getPreferences(context).run {
            val upload = UploadInfo(transferId).apply {
                this.status = status
            }
            return saveUpload(upload)
        }
    }


    private fun getPreferences(context: Context): SharedPreferences {
        return context.getSharedPreferences(
            PREFERENCES_NAME,
            Context.MODE_PRIVATE
        )
    }


    private fun SharedPreferences.tryGetUpload(transferId: String): UploadInfo? {
        val data: String? = getString(transferId, null)
        return if (data == null)
            null
        else
            gson.fromJson(data, UploadInfo::class.java)
    }


    private fun SharedPreferences.saveUpload(uploadInfo: UploadInfo): UploadInfo {
        val editor = edit()
        editor.putString(uploadInfo.transferId, gson.toJson(uploadInfo))
        editor.apply()
        return uploadInfo;
    }
 }

Answer 2

You have a pretty good use-case but unfortunately you cannot cache data within Worker class or pass on the data to the next Worker object on retry! As you suspected, you will have to store the index outside of the WorkManager provided constructs!

Long answer,

The Worker object can receive and return data. It can access the data from getInputData() method. If you chain tasks, the output of one worker can be input for the next-in-line worker. This can be done by returning Result.success(output) (see below code)

public Result doWork() {
        int chunkIndex = upload();

        //...set the output, and we're done!
        Data output = new Data.Builder()
            .putInt(KEY_RESULT, result)
            .build();
        return Result.success(output);
}

So the problem is we cannot return data for the retry case, only for failure and success case! (Result.retry(Data data) method is missing!)

Reference: official documentation and API.