CODEWITHSUNDEEP
typescriptasync-awaitasync.js
Jack Murphy
Jack Murphy

Reputation: 3020

How to enable async.mapLimit to work with TypeScript async / await

Is it possible to use the Async NPM module to work with async/await in TypeScript 2.2x?

Goal: Create a web scraper that spins up 10 parallel HTTP requests using Async's mapLimit function.

An example of the wrapped HTTP function would be as follows:

async callUniRest(url: string): Promise<number> {
  return new Promise<number>(resolve => {
    unirest.get(url)
      .end((response: any) => {
          resolve(cheerio.load(response.body);
      });
    });
}

Problem:

When I run:

const myList: string[] = ['http...', 'http...', 'http...', 'http...']
async.mapLimit(myList, 10, callUniRest, function(err: any, results: any {
  console.log(results);
})

the callback only gets called after the first element finishes.

Question: How do I enable async.mapLimit to work with multiple calls?

Upvotes: 2

Views: 1612

Answers (2)

Ian Routledge
Ian Routledge

Reputation: 4042

Is it possible to use the 'async' npm module to work with async / await in Typescript

Yes.

I found that mapLimit was only calling the iterator for the first set. This was because I was transpiling from TypeScript. If I used native async/await with JS then it worked. This is because, quoted from the async docs for AsyncFunction:

due to JavaScript limitations, we can only detect native async functions and not transpilied implementations.

So if you wrap your iterator with asyncify from the async library then you can return from inside your iterator and not use a callback:

If you are using async functions through a transpiler (e.g. Babel), you must still wrap the function with asyncify, because the async function will be compiled to an ordinary function that returns a promise.

Here's a somewhat contrived TypeScript example that demonstrates how to do this. Note: if you remove the asyncify call it won't work:

import { asyncify, mapLimit } from "async";

const listOfThings = "abcdefghijklmnopqrstuvwxyz".split("");

const convertToUppercase = async () =>
  await mapLimit<string, string[]>(
    listOfThings,
    5,
    asyncify(async letter => letter.toUpperCase())
  );

const init = async () => {
  const uppered = await convertToUppercase();

  console.log("done", uppered);
};

init();

Upvotes: 6

Madara&#39;s Ghost
Madara&#39;s Ghost

Reputation: 174957

No. The async module provides utilities and abstractions for folks who cannot or would not use Promises. They don't mix.

Not to mention, by the time you have myList, all of the requests were already sent.

async.mapLimit only works with functions that accept a callback, and not with ones that return a Promise.

What you might want is to use Bluebird's Promise.map() with the concurrency operator

Upvotes: 0

Related Questions