How to select duplicate values from a list in java?

Question

For example my list contains {4, 6, 6, 7, 7, 8} and I want final result = {6, 6, 7, 7}

One way is to loop through the list and eliminate unique values (4, 8 in this case).

Is there any other efficient way rather than looping through list ? I asked this question because the list that I am working is very large ? My code is

List<Long> duplicate = new ArrayList();
for (int i = 0; i < list.size(); i++) {
     Long item = (Long) list.get(i);
     if (!duplicate.contains(item)) {
          duplicate.add(item);
         }
     }

Answer 1

Try this:

Inspired by this answer: https://stackoverflow.com/a/41262509/11256849

for (String s : yourList){
     if (indexOfNth(yourList, s, 2) != -1){
         Log.d(TAG, s);
      }
   }

Using this method:

public static <T> int indexOfNth(ArrayList list, T find, int nthOccurrence) {
        if (list == null || list.isEmpty()) return -1;
        int hitCount = 0;
        for (int index = 0; index < list.size(); index++) {
            if (list.get(index).equals(find)) {
                hitCount++;
            }
            if (hitCount == nthOccurrence) return index;
        }
        return -1;
    }

Answer 2

The following will work with Eclipse Collections:

IntBag bag = IntLists.mutable.with(4, 6, 6, 7, 7, 8).toBag().selectDuplicates();

If you want boxed values instead of primitives, the following will work:

Bag<Integer> bag = Lists.mutable.with(4, 6, 6, 7, 7, 8).toBag().selectDuplicates();

Note: I am a committer for Eclipse Collections.

Answer 3

again lambda saves the day:

List<Long> duplicates = duplicate.stream()
  .collect( Collectors.collectingAndThen( Collectors.groupingBy( Function.identity() ),
    map -> {
      map.values().removeIf( v -> v.size() < 2 );  // eliminate unique values (4, 8 in this case)
      return( map.values().stream().flatMap( List::stream ).collect( Collectors.toList() ) );
    } ) );  // [6, 6, 7, 7]

the speed-optimized version of the above solution:

List<Long> duplicates = duplicate.stream().collect( Collectors.collectingAndThen(
    Collectors.groupingBy( Function.identity(), Collectors.counting() ),
    map -> {
      map.values().removeIf( v -> v < 2 );  // eliminate unique values (4, 8 in this case)
      return( map.entrySet().stream().collect( Collector.of( ArrayList<Long>::new, (list, e) -> {
        for( long n = 0; n < e.getValue(); n++ )
          list.add( e.getKey() );
      }, (l1, l2) -> null ) ) );
    } ) );  // [6, 6, 7, 7]

the long values of duplicate are not saved but counted — quite certainly the fastest and most space-saving variant

Answer 4

I like answer Java 8, Streams to find the duplicate elements. Solution return only unique duplicates.

 Integer[] numbers = new Integer[] { 1, 2, 1, 3, 4, 4 };
 Set<Integer> allItems = new HashSet<>();
 Set<Integer> duplicates = Arrays.stream(numbers)
    .filter(n -> !allItems.add(n)) //Set.add() returns false if the item was already in the set.
    .collect(Collectors.toSet());
 System.out.println(duplicates); // [1, 4]

Answer 5

With Guava and Java 8, it's trivial and fast:

Multiset<Integer> multiset = HashMultiset.create(list);
return list.stream()
    .filter(i -> multiset.count(i) > 1)
    .collect(Collectors.toList());

The first line computes the counts using a sort of a hash map. The remainder is more than obvious.

Something like this could simulate the multiset:

HashMap<Integer, Integer> multiset = new HashMap<>();
list.stream().forEach(i -> 
    multiset.compute(i, (ignored, old) -> old==null ? 1 : old+1)));

Answer 6

import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class FindDuplicate {

    public static void main(String[] args) {

        // Load all your ArrayList
        List<String> list = new ArrayList<String>();
        list.add("Jhon");
        list.add("Jency");
        list.add("Mike");
        list.add("Dmitri");
        list.add("Mike");

        // Set will not allow duplicates
        Set<String> checkDuplicates = new HashSet<String>();

        System.out.println("Actual list " + list);
        for (int i = 0; i < list.size(); i++) {
            String items = list.get(i);
            if (!checkDuplicates.add(items)) {
                // retain the item from set interface
                System.out.println("Duplicate in that list " + items);
            }
        }

    }
}

Answer 7

Your List should ideally have been a Set which doesn't allow duplicates in the first place. As an alternative to looping, you could either convert and switch to Set or use it intermediately to eliminate duplicates as follows:

List<Long> dupesList = Arrays.asList(4L, 6L, 6L, 7L, 7L, 8L);

Set<Long> noDupesSet = new HashSet<Long>(dupesList);
System.out.println(noDupesSet); // prints: [4, 6, 7, 8]

// To convert back to List
Long[] noDupesArr = noDupesSet.toArray(new Long[noDupesSet.size()]);
List<Long> noDupesList = Arrays.asList(noDupesArr);
System.out.println(noDupesList); // prints: [4, 6, 7, 8]

Answer 8

Here's my version of the solution:

import java.util.ArrayList;

public class Main {

public static void main(String[] args) {

    ArrayList<Integer> randomNumbers = new ArrayList<Integer>();
    ArrayList<Integer> expandingPlace = new ArrayList<Integer>();
    ArrayList<Integer> sequenceOfDuplicates = new ArrayList<Integer>();

    for (int i = 0; i < 100; i++) {
        randomNumbers.add((int) (Math.random() * 10));
        expandingPlace.add(randomNumbers.get(i));
    }

    System.out.println(randomNumbers); // Original list.

    for (int i = 0; i < randomNumbers.size(); i++) {
        if (expandingPlace.get(i) == expandingPlace.get(i + 1)) {
            expandingPlace.add(0);
            sequenceOfDuplicates.add(expandingPlace.get(i)); 
            sequenceOfDuplicates.add(expandingPlace.get(i + 1));
        }
    }

    System.out.println(sequenceOfDuplicates); // What was in duplicate there.

}

}

It adds numbers from 0 to 9 to a list, and it adds to another list what is in "duplicate" (a number followed by the same number). You can use your big list instead of my randomNumbers ArrayList.

Answer 9

Is there any other efficient way rather than looping through list ?

You could hire a magic elf and let it do it for you. How would you ever want to do this without looping through it? If you don't loop through the list, you even won't be able to have a look at the elements. It is like you want to sum a whole bunch of numbers together without looking at those numbers. Summing elements is much easier than searching for duplicates or searching for unique elements. In general, 97% of what code does is looping through lists and data and process and update it.

So, said that, you have to loop. Now you might want to choose the most efficient way. Some methods come to mind:

• Sort all the numbers and then loop only once through it to find duplicates (since they will be next to each other). However, keep in mind that sorting algorithms also loop through the data.
• For each element in the list, check if there is another element with the same value. (This is how you did it. This means you have two loops inside each other. (contains loops through the list of course.))

Answer 10

Some good answers so far but another option just for the fun of it. Loop through the list trying to place each number into a Set e.g. a HashSet. If the add method returns false you know the number is a duplicate and should go into the duplicate list.

EDIT: Something like this should do it

Set<Number> unique = new HashSet<>();
List<Number> duplicates = new ArrayList<>();
for( Number n : inputList ) {
    if( !unique.add( n ) ) {
        duplicates.add( n );
    }
}

Answer 11

List<Number> inputList = Arrays.asList(4, 6, 6, 7, 7, 8);
List<Number> result = new ArrayList<Number>();
for(Number num : inputList) {
   if(Collections.frequency(inputList, num) > 1) {
       result.add(num);
   }
}

I am not sure about the efficiency, but I find the code easy to read (and that should be preferred.

EDIT: changing Lists.newArrayList() to new ArrayList<Number>();

Answer 12

Given that you can do this by looping through the list only once, I wouldn't worry about performance too much. If you search for more performant solutions then you'll probably end up over-complicating the code and the readability and maintainability will suffer. At the end of the day, if you want to check the whole list for duplicates then you have to visit every element.

I'd advise writing the obvious solution and see how it performs. You'll probably be surprised how fast Java can iterate over a list, even if it is particularly large.

Answer 13

Have a

Map<Integer, Integer> numberToOccurance = new HashMap<Integer, Integer>();

maintain count and number, at the end iterate keyset and get values with more than one count