CODEWITHSUNDEEP
javajava-streamstatelessstateful
Tranquility
Tranquility

Reputation: 3301

Do limit and skip operations become stateless when unordered

My question is quite simple. The intermediate limit and skip operations on java streams are lablled as being stateful operations - I assume because they by default need to limit or skip the first n elements on an ordered stream.

If I make the input source stream unordered by invoking the unordered() method or using an unordered source, can we effectively say that these operations could then be considered stateless, or am I missing something here?

Upvotes: 1

Views: 175

Answers (2)

Holger
Holger

Reputation: 298509

No, limit and skip are still stateful operations as the processing of elements depends on information about the processing of other elements (i.e. whether they have been processed).

These operations are easier to implement for unordered stream, but this doesn’t change their stateful nature.

You can tell this by simply asking youself: “can this operation get implemented for a single element by just looking at the element and nothing else (besides unchanging information known prior to starting the entire stream operation)?”

Upvotes: 3

Tagir Valeev
Tagir Valeev

Reputation: 100299

I assume that your question is regarding parallel streams as in current Stream API implementation sequential streams don't use ordering in any way.

Current parallel unordered skip/limit implementation uses buffer of size 128 for each parallel task, so it's possible that you will read more elements from the source than actually necessary. This is done to reduce the possible contention on a shared atomic variable which might be crucial for low-Q tasks (tasks where per-element processing is quite fast). So to answer your question, no, unordered skip/limit operations are still stateful.

You may easily check this with the following program:

AtomicLong counter = new AtomicLong();
IntStream.range(0, 1_000_000).parallel().unordered().filter(x -> true)
        .peek(x -> counter.incrementAndGet()).limit(1000).toArray();
System.out.println(counter.get());

It prints how many elements were actually taken from the source. On my four-core system this code prints usually 1280 (128*10) or 1408 (128*11), even though 1000 elements were requested.

Upvotes: 2

Related Questions