Multiple delays in Javascript/Nodejs Promise

Question

I'm working on a proxy that caches files and I'm trying to add some logic that prevents multiple clients from downloading the same files before the proxy has a chance to cache them.

Basically, the logic I'm trying to implement is the following:

Client 1 requests a file. The proxy checks if the file is cached. If it's not, it requests it from the server, caches it, then sends it to the client.

Client 2 requests the same file after client 1 requested it, but before the proxy has a chance to cache it. So the proxy will tell client 2 to wait a few seconds because there is already a download in progress.

A better approach would probably be to give client 2 a "try again later" message, but let's just say that's currently not an option.

I'm using Nodejs with the anyproxy library. According to the documentation, delayed responses are possible by using promises.

However, I don't really see a way to achieve what I want using Promises. From what I can tell, I could do something like this:

module.exports = {
  *beforeSendRequest(requestDetail) {
    if(thereIsADownloadInProgressFor(requestDetail.url)) {
        return new Promise((resolve, reject) => {
            setTimeout(() => { // delay
              resolve({ response: responseDetail.response });
            }, 10000);
        });
    }
  }
};

But that would mean simply waiting for a maximum amount of time and hoping the download finishes by then. And I don't want that.

I would prefer to be able to do something like this (but with Promises, somehow):

module.exports = {
  *beforeSendRequest(requestDetail) {
    if(thereIsADownloadInProgressFor(requestDetail.url)) {
        var i = 0;
        for(i = 0 ; i < 10 ; i++) {
            JustSleep(1000);

            if(!thereIsADownloadInProgressFor(requestDetail.url))
                return { response: responseDetail.response };
        }
    }
  }
};

Is there any way I can achieve this with Promises in Nodejs?

Thanks!

Answer 1

You don't need to send a try again response, simply serve the same data to both requests. All you need to do is store the requests somewhere in the caching system and trigger all of them when the fetching is done.

Here's a cache implementation that does only a single fetch for multiple requests. No delays and no try-laters:

export class class Cache {    
    constructor() {
        this.resultCache = {}; // this object is the cache storage
    }

    async get(key, cachedFunction) {
        let cached = this.resultCache[key];

        if (cached === undefined) { // No cache so fetch data
            this.resultCache[key] = {
                pending: [] // This is the magic, store further
                            // requests in this pending array.
                            // This way pending requests are directly
                            // linked to this cache data
            }
            try {
                let result = await cachedFunction(); // Wait for result

                // Once we get result we need to resolve all pending
                // promises. Loop through the pending array and
                // resolve them. See code below for how we store pending
                // requests.. it will make sense:
                this.resultCache[key].pending
                    .forEach(waiter => waiter.resolve(result));

                // Store the result of the cache so later we don't
                // have to fetch it again:
                this.resultCache[key] = {
                    data: result
                }

                // Return result to original promise:
                return result;

                // Note: yes, this means pending promises will get triggered
                // before the original promise is resolved but normally
                // this does not matter. You will need to modify the
                // logic if you want promises to resolve in original order
            }
            catch (err) { // Error when fetching result

                // We still need to trigger all pending promises to tell
                // them about the error. Only we reject them instead of
                // resolving them:
                if (this.resultCache[key]) {
                    this.resultCache[key].pending
                      .forEach((waiter: any) => waiter.reject(err));
                }
                throw err;
            }
        }
        else if (cached.data === undefined && cached.pending !== undefined) {
            // Here's the condition where there was a previous request for
            // the same data. Instead of fetching the data again we store
            // this request in the existing pending array.
            let wait = new Promise((resolve, reject) => {

                // This is the "waiter" object above. It is basically
                // It is basically the resolve and reject functions
                // of this promise:
                cached.pending.push({
                    resolve: resolve,
                    reject: reject
                });
            });

            return await wait; // await response form original request.
                               // The code above will cause this to return.
        }
        else {
            // Return cached data as normal
            return cached.data;
        }
    }
}

The code may look a bit complicated but it is actually quite simple. First we need a way to store the cached data. Normally I'd just use a regular object for this:

{ key : result }

Where the cached data is stored in the result. But we also need to store additional metadata such as pending requests for the same result. So we need to modify our cache storage:

{ key : {
    data: result,
    pending: [ array of requests ]
  }
}

All this is invisible and transparent to code using this Cache class.

Usage:

const cache = new Cache();

// Illustrated with w3c fetch API but you may use anything:
cache.get( URL , () => fetch(URL) )

Note that wrapping the fetch in an anonymous function is important because we want the Cache.get() function to conditionally call the fetch to avoid multiple fetch being called. It also gives the Cache class flexibility to handle any kind of asynchronous operation.

Here's another example for caching a setTimeout. It's not very useful but it illustrates the flexibility of the API:

cache.get( 'example' , () => {
    return new Promise((resolve, reject) => {
        setTimeout(resolve, 1000);
    });
});

Note that the Cache class above does not have any invalidations or expiry logic for the sake of clarity but it's fairly easy to add them. For example if you want the cache to expire after some time you can just store the timestamp along with the other cache data:

{ key : {
    data: result,
    timestamp: timestamp,
    pending: [ array of requests ]
  }
}

Then in the "no-cache" logic simply detect the expiry time:

if (cached === undefined || (cached.timestamp + timeout) < now) ...

Answer 2

You can use a Map to cache your file downloads.

The mapping in Map would be url -> Promise { file }

// Map { url => Promise { file } }
const cache = new Map()

const thereIsADownloadInProgressFor = url => cache.has(url)

const getCachedFilePromise = url => cache.get(url)

const downloadFile = async url => {/* download file code here */}

const setAndReturnCachedFilePromise = url => {
  const filePromise = downloadFile(url)
  cache.set(url, filePromise)
  return filePromise
}

module.exports = {
  beforeSendRequest(requestDetail) {
    if(thereIsADownloadInProgressFor(requestDetail.url)) {
        return getCachedFilePromise(requestDetail.url).then(file => ({ response: file }))
    } else {
        return setAndReturnCachedFilePromise(requestDetail.url).then(file => ({ response: file }))
    }
  }
};