CODEWITHSUNDEEP
javajava-8java-stream
MoienGK
MoienGK

Reputation: 4654

Java 8 Stream distinct is not working

This is what i am doing: 

List scores = Stream.concat(oldEntries.stream(), newEntries.stream())
                    .sorted()
                    .distinct()
                    .limit(maxSize)
                    .collect(Collectors.toList());

I am expecting a sorted list without any duplicates, but some times there is duplicate in the list.

I have override the hashCode and equals method, I have also observed that these methods are returning the correct value every time. Can any one see what is wrong with my stream?

This is my equals() and hashCode() They are auto generated by IDEA :

..
private int userId;
private int levelId;
private int score;

@Override
public boolean equals(Object o) {

    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;

    Score score = (Score) o;

    if (userId != score.userId) return false;
    return levelId == score.levelId;

}

@Override
public int hashCode() {
    int result = userId;
    result = 31 * result + levelId;
    return result;
}

public int compareTo(Score other) {

    if (other == null) {
        return 1;
    } else {
        return Integer.compare(other.score, this.score);
    }
}

 ..

Upvotes: 10

Views: 12654

Answers (2)

Anonymous
Anonymous

Reputation: 86232

It is a bug.

The documentation of Stream.distinct() simply says:

Returns a stream consisting of the distinct elements (according to Object.equals(Object)) of this stream.

For ordered streams, the selection of distinct elements is stable (for duplicated elements, the element appearing first in the encounter order is preserved.) For unordered streams, no stability guarantees are made.

There is no requirement that for ordered streams the equal objects should come right after each other (consecutively). However the implementation seems to assume they do. What the documentation means is that the first occurrence of user 2, level 3 should be preserved and the second occurrence discarded.

According to the Java bug database the bug exists up to Java 13 and remains unresolved.

Links

Upvotes: 5

Kayaman
Kayaman

Reputation: 73538

Your stream is first ordered according to compareTo, i.e. using score.

It's then "distinctified" using equals(), i.e. using userId and levelId. According to the javadoc:

For ordered streams, the selection of distinct elements is stable (for duplicated elements, the element appearing first in the encounter order is preserved.) For unordered streams, no stability guarantees are made.

Example:

score 1, user 2, level 3
score 3, user 2, level 3
score 1, user 3, level 1

After sorting...

score 1, user 2, level 3
score 1, user 3, level 1
score 3, user 2, level 3

Distinct now does nothing, because the elements are not equal according to user/level. This can result in "duplicate" elements, because you're ordering the stream based on one thing, but determining equality by an entirely different thing.

Upvotes: 10

Related Questions