Trying to recreate java.util.ConcurrentModificationException

Question

Code looks like this: The maps being used here are Guava maps

private Map<SomeObject, SomeOtherObject> myMap = Maps.newLinkedHashMap();

public Map<SomeObject, SomeOtherObject> getMap() {
  return Maps.newHashMap(myMap);
}

public void putMap(SomeObject a, SomeOtherObject b) {
  myMap.put(a,b);
}

So, the above threw java.util.ConcurrentModificationException and am trying to recreate the scenario. But no matter what I try, the system seems resilient. Here is what I tried:

 1. Created 'n' threads some of which call getMap() and some call putMap()
 2. Created 'n' threads that only call putMap() and one thread the is in an infinite loop that calls getMap()
 3. Created 2 threads each of which alternates calling getMap() and putMap(). Meaning, thread-1 first calls get then put and thread-2 first calls put and then get.

None of the above works, either keeps running or goes to OOM. Any pointers on how to go about doing this?

EDIT I believe the ConcurrentModificationException is thrown when returning the copy of the map {Maps.newHashMap(myMap);}. During this process the iterator creates a copy and while the iterator is at work, if the contents of the map are modified its not happy.

Answer 1

Often the trick to reproducing concurrency errors is to create a delay in processing.

In this case, the exception is thrown if the put thread adds a member while the get thread is iterating over the map.

I was able to get good result with the following, which artificially slows down the iteration (which also does a put) by adding a delay in hashCode

import java.util.Map;
import java.util.HashMap;

class CME {
    static class K {
        String value;
        K(String value) { this.value = value; }
        @Override public int hashCode() { 
            try {
                Thread.sleep(100);
            } catch (Exception e) {}
            return value.hashCode();
        }
    }

    final static Map<K, String> map = new HashMap<>();
    static {
        for (int i = 0; i < 1000; i++) {
            String s = Integer.toString(i);
            map.put(new K(s), s);
        }
    }

    public static void main(String[] args) {
        Runnable get = new Runnable() {
                @Override public void run() {
                    for (int i = 0; ; i++) {
                        if (i%1000 ==0) { System.out.printf("get: %d\n", i); }
                        Map<K, String> m2 = new HashMap<>(map);
                    }
                }
            };
        new Thread(get).start();
        for (int i = 0; ; i++) {
            if (i%1000 ==0) { System.out.printf("put: %d\n", i); }
            String s = Integer.toString(1000 + i);
            map.put(new K(s), s);
        }
    }
}

Answer 2

ConcurrentModificationException is thrown when iterating over a Collection or Map, using its Iterator, and during the iteration the Collection is modified (not necessarilly on another thread, though).

If the copy operation uses an Iterator, then there is no guarantee other threads will be modifying the Map while the iteration is busy, especially if the Map is small.

To force the issue you'll have to force a thread that is making a copy to wait for a modifying thread from within the copy loop. Using the Guava library may prevent you from doing so, but temporarily replacing it with a manual copy will help you pinpoint the problem.

Below is a basic example using different threads that forces the issue syncing with CountDownLatches :

public static void main(String[] args) {
    final Map<Integer, Integer> map = new HashMap<>();
    final CountDownLatch readLatch = new CountDownLatch(1);
    final CountDownLatch writeLatch = new CountDownLatch(1);

    for (int i = 0; i < 100; i++) {
        map.put(i, i);
    }

    new Thread(new Runnable() {
        @Override
        public void run() {
            try {
                for (Map.Entry<Integer, Integer> entry : map.entrySet()) {
                    if (entry.getKey().equals(Integer.valueOf(10))) {
                        try {
                            writeLatch.countDown();
                            readLatch.await();
                        } catch (InterruptedException e) {
                            Thread.currentThread().interrupt();
                        }
                    }
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }).start();

    new Thread(new Runnable() {
        @Override
        public void run() {
            try {
                writeLatch.await();
                map.put(150, 150);
                readLatch.countDown();
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
    }).start();
}

The reading thread at some point releases the write thread, and then waits, in the mean time the writing thread makes a modification, and then allows the reading thread to resume looping.

If you're debugging, it may be helpful to insert such latches to force the issue. Note that a subsequent fix of the concurrency issue may deadlock with the latches in place. For fixing the issue you may be able to get reliable failures using sleep(), a subsequent fix should then simply work reliably.

Answer 3

The code sample below should print "GOT IT" (i.e. ConcurrentModificationException) within a few milliseconds.

In substance: the code puts to and gets from the map concurrently and inevitably gets a CME. The random part is important: if you keep putting the same key you will probably not get the same effect.

public class CME {

    private static final Test test = new Test();
    private static final Random rand = new Random();

    public static void main(String[] args) throws InterruptedException {
        Runnable getter = new Runnable() {
            @Override public void run() {
                try {
                    while (!Thread.interrupted()) {
                        Map<String, String> tryit = test.getMap();
                    }
                } catch (ConcurrentModificationException e) {
                    System.out.println("GOT IT!");
                }
            }
        };
        Runnable putter = new Runnable() {
            @Override public void run() {
                while (!Thread.interrupted()) {
                    //use a random component to make sure the map
                    //is actually mutated
                    char c = (char) rand.nextInt();
                    test.putMap(String.valueOf(c), "whatever");
                }
            }
        };
        Thread g = new Thread(getter);
        Thread p = new Thread(putter);
        g.start();
        p.start();
        g.join(); //wait until CME
        p.interrupt(); //then interrupt the other thread
                       //to allow program to exit
    }

    static class Test {
        private Map<String, String> myMap = Maps.newLinkedHashMap();
        public Map<String, String> getMap() { return Maps.newHashMap(myMap); }
        public void putMap(String a, String b) { myMap.put(a, b); }
    }
}

Answer 4

Assuming that you indeed use com.google.common.collect.Maps, the implementation of newHashMap is

public static <K, V> HashMap<K, V> newHashMap(Map<? extends K, ? extends V> map) {
    return new HashMap<K, V>(map);
}

If we then look at the implementation of HashMap:

public HashMap(Map<? extends K, ? extends V> m) {
    this(Math.max((int) (m.size() / DEFAULT_LOAD_FACTOR) + 1,
            DEFAULT_INITIAL_CAPACITY), DEFAULT_LOAD_FACTOR);
    putAllForCreate(m);
}

and

private void  [More ...] putAllForCreate(Map<? extends K, ? extends V> m) {
    for (Iterator<? extends Map.Entry<? extends K, ? extends V>> i 
            = m.entrySet().iterator(); i.hasNext(); ) {
        Map.Entry<? extends K, ? extends V> e = i.next();
        putForCreate(e.getKey(), e.getValue());
    }
}

So indeed, the call to newHashMap uses an iterator to traverse the map. As already pointed out by other answers, if putMap is called while iterating the map, this will throw a ConcurrentModificationException.

How can you reproduce this? I would say that two threads are enough: one repetitively calls getMap, the other one putMap.

Answer 5

If you really want to create a ConcurrentModificationException, you need an Iterator.

Create it either explicitly by calling myMap.entrySet().iterator(), myMap.keySet().iterator(), or myMap.values().iterator(), or implicitly by iterating over the entries, keys, or values in a "foreach" loop. While iterating, modify the map.

A ConcurrentModificationException is thrown when a collection is modified when something is iterating over it, which would otherwise lead to undefined behavior in the iterator. From the linked Javadocs:

This exception may be thrown by methods that have detected concurrent modification of an object when such modification is not permissible. For example, it is not generally permissible for one thread to modify a Collection while another thread is iterating over it. In general, the results of the iteration are undefined under these circumstances. Some Iterator implementations (including those of all the general purpose collection implementations provided by the JRE) may choose to throw this exception if this behavior is detected. Iterators that do this are known as fail-fast iterators, as they fail quickly and cleanly, rather that risking arbitrary, non-deterministic behavior at an undetermined time in the future.

Note that this exception does not always indicate that an object has been concurrently modified by a different thread. If a single thread issues a sequence of method invocations that violates the contract of an object, the object may throw this exception. For example, if a thread modifies a collection directly while it is iterating over the collection with a fail-fast iterator, the iterator will throw this exception.

Answer 6

ConcurrentModificationException is thrown when a fail-fast collection is modified while it is iterated. The easiest way to make this happen is for example:

ArrayList<String> list = new ArrayList<String>();
// put items in list
for(String s : list)
   list.remove(0);