CODEWITHSUNDEEP
javaparallel-processingjava-8java-stream
Benedikt Bünz
Benedikt Bünz

Reputation: 646

Parallelism and Flatmap in Java 8 Streams

Consider the following example:

    IntStream.of(-1, 1)
             .parallel()
             .flatMap(i->IntStream.range(0,1000).parallel())
             .forEach(System.out::println);

Does it matter whether I set the inner flag to parallel? The results look very similar if I leave it away or not.

Also why does the code (ReferencePipeline) sequentialize the mapping?

I am confused by the line:

result.sequential().forEach(downstream);

Upvotes: 18

Views: 9414

Answers (2)

Dmytro Buryak
Dmytro Buryak

Reputation: 368

For anyone like me, who has a dire need to parallelize flatMap and needs some practical solution, not only history and theory.

The simplest solution I came up with is to do flattening by hand, basically by replacing it with map + reduce(Stream::concat).

Already posted an answer with details in another thread: https://stackoverflow.com/a/66386078/3606820

Upvotes: 0

Daniel Hári
Daniel Hári

Reputation: 7784

In the current JDK (jdk1.8.0_25), the answer is no, it doesn't matter you set the inner flag to parallel, because even you set it, the .flatMap() implementation set's back the stream to sequential here:

result.sequential().forEach(downstream);

("result" is the inner stream and it's sequential() method's doc says: Returns an equivalent stream that is sequential. May return itself, either because the stream was already sequential, or because the underlying stream state was modified to be sequential.)

In most cases there could be no effort to make the inner stream parallel; if outer stream has at least same number of items as number of threads that can run parallel (ForkJoinPool.commonPool().getParallelism() = 3 in my computer).

Upvotes: 17

Related Questions