CODEWITHSUNDEEP
typescriptes6-promisehighland.js
k0pernikus
k0pernikus

Reputation: 66599

How to deal with tuples when returned from an aync function within highland streams?

I use [email protected] and highland@^2.13.0.

I have an async function that returns a tuple of the type [string, string[]].

I have worked with highland in the past, and I know that I can work with promises by creating a new highland stream with the promise, and resolving the promise through flattening it.

So what I am doing is basically: creating promises, parallelizing their consumption, and in the end I want one stream of [string, string[]] via flatten.

Yet highland actual behavior contradicts my expectation.

Here my TypeScript code showcasing my expectation:

import * as highland from "highland";

const somethingAsync = async (s: string): Promise<[string, string[]]> => {
    return [s, s.split("")];
};

const data = [
    "foo",
    "bar",
    "baz",
    "poit",
    "gnarf",
];

const stream: Highland.Stream<[string, string[]]> = highland(data)
    .map((input: string) => highland(somethingAsync(input))) // wrapping the promises into a highland stream
    .parallel(3) // processing it in parallel
    .flatten(); // flattening it to resolve the promises

stream.each((shouldBeATuple: [string, string[]]) => {
    console.log(shouldBeATuple);

    const [item1, item2] = shouldBeATuple;
    console.log(item1, item2);
});

I expect shouldBeATuple to actually be a tuple. Yet instead I get a stream of all the elements, as if flatten flattens too much.

I expect it to be on first iteration:

 ["foo", ["f","o","o"]];

and on second one:

 ["bar", ["b","a","r"]];

Yet what I get instead is:

 "foo",

and then it will be followed by: "f", "o", "o", "bar", "b", "a", "r".

So I lose the tuples completely and get one big "mess" of a stream with all the elements in it.

How can I get a stream of tuples when using async functions?

I found out that when I do not flatten the stream I get my result yet typescript will throw an error:

15 const stream: Highland.Stream<[string, string[]]> = highland(data)
         ~~~~~~
src/tuple.ts(15,7): error TS2322: Type 'Stream<Stream<[string, string[]]>>' is not assignable to type 'Stream<[string, string[]]>'.
  Type 'Stream<[string, string[]]>' is not assignable to type '[string, string[]]'.
    Property '0' is missing in type 'Stream<[string, string[]]>'.

Upvotes: 0

Views: 643

Answers (1)

Matt McCutchen
Matt McCutchen

Reputation: 30919

The type declaration for parallel is wrong. When it is corrected, your code passes the type checker without the flatten. I've submitted a pull request to fix the typings; you can add a copy of the corrected typings from there to your project.

Upvotes: 1

Related Questions