What's the difference between ConcurrentHashMap and Collections.synchronizedMap(Map) in term of performance?

Question

I am trying to evaluate those concepts by using code. This is what I ended up with

    public void runCWith3Threads() {

        // mesure add with 3 threads
        for (int i = 0; i < 10; i++) {
            Map<Integer, Person> shm = Collections.synchronizedMap(new HashMap<Integer, Person>());
            Map<Integer, Person> chm = new ConcurrentHashMap<Integer, Person>();

            MapThread sm1 = new MapThread(shm, 0, 20000, "sm1");
            MapThread sm2 = new MapThread(shm, 20000, 30000, "sm2");
            MapThread sm3 = new MapThread(shm, 30000, 50000, "sm3");

            sm1.start();sm2.start();sm3.start();

            while (true) {
                try {
                    sm1.join();
                    sm2.join();
                    sm3.join();
                    break;
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }

            long secondMax = sm1.time > sm2.time ? sm1.time : sm2.time;
            long firstMax = secondMax > sm3.time ? secondMax : sm3.time;

            System.out.println("Millisec of SynchronizedMap cost: " + firstMax);

            MapThread m1 = new MapThread(chm, 0, 20000, "m1");
            MapThread m2 = new MapThread(chm, 20000, 30000, "m2");
            MapThread m3 = new MapThread(chm, 30000, 50000, "m3");

            m1.start();m2.start();m3.start();

            while (true) {
                try {
                    m1.join();
                    m2.join();
                    m3.join();
                    break;
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }

            secondMax = m1.time > m2.time ? m1.time : m2.time;
            firstMax = secondMax > m3.time ? secondMax : m3.time;

            System.out.println("Millisec of ConcurrentHashMap cost: " + firstMax);
            System.out.println();
        }
    }

    public class MapThread extends Thread {
        Map<Integer, Person> map;
        int from;
        int to;
        long time;
        String name;

        public MapThread(Map<Integer, Person> map, int from, int to, String name) {
            this.map = map;
            this.from = from;
            this.to = to;
            this.name = name;
        }

        public void run() {

            long start = System.currentTimeMillis();
            for (int i = from; i < to; i++) {
                map.put(i, new Person());
            }
            long end = System.currentTimeMillis();
            time = end - start;
            return;
        }
    }

What I expect is, after the code runs, the ConcurrentHashMap 's result will be faster, since it allows multiple insertion into the map. For the SynchronizedMap, since each thread is waiting for the previous thread to finish (map is synchronized), the code will acts as the same once running with a single thread environment

However, the result is not completely reflect what I expected

Millisec of SynchronizedMap cost: 250
Millisec of ConcurrentHashMap cost: 203

Millisec of SynchronizedMap cost: 171
Millisec of ConcurrentHashMap cost: 172

Millisec of SynchronizedMap cost: 172
Millisec of ConcurrentHashMap cost: 188

Millisec of SynchronizedMap cost: 171
Millisec of ConcurrentHashMap cost: 172

Millisec of SynchronizedMap cost: 187
Millisec of ConcurrentHashMap cost: 172

Millisec of SynchronizedMap cost: 171
Millisec of ConcurrentHashMap cost: 189

Millisec of SynchronizedMap cost: 187
Millisec of ConcurrentHashMap cost: 171

Millisec of SynchronizedMap cost: 188
Millisec of ConcurrentHashMap cost: 171

Millisec of SynchronizedMap cost: 172
Millisec of ConcurrentHashMap cost: 172

Millisec of SynchronizedMap cost: 171
Millisec of ConcurrentHashMap cost: 188

Why is that ?

Update

1. With

Map<Integer, Person> chm = new ConcurrentHashMap<Integer, Person>(100000, 10, 3);

I have result

Millisec of SynchronizedMap cost: 208
Millisec of ConcurrentHashMap cost: 216

Millisec of SynchronizedMap cost: 255
Millisec of ConcurrentHashMap cost: 196

1. With

Map<Integer, Person> chm = new ConcurrentHashMap<Integer, Person>(100000);

I have result

Millisec of SynchronizedMap cost: 204
Millisec of ConcurrentHashMap cost: 283

Millisec of SynchronizedMap cost: 203
Millisec of ConcurrentHashMap cost: 200

Answer 1

If you are doing benchmarks, you should:

1. doing a warm-up phase (due to JIT compilation and class loading),
2. repeat the test several times (due to garbage collection).

If I create such a benchmark that is similar to your benchmark, I get the following results:

Warmup...
Benchmark...
 4 *  500000: 0.22s / 0.04s
 4 * 1000000: 0.55s / 0.10s
 4 * 1500000: 1.10s / 0.16s
 4 * 2000000: 0.90s / 0.19s
 4 * 2500000: 1.68s / 0.25s

The first number shows the number of threads, the second number the size of the int ranges, the third number is the duration for a synchronized Map and the fourth number the duration for ConcurrentHashMap. As you can see, the ConcurrentHashMap is in all cases significantly faster.

Below you can find the whole Java code. Note that is makes use of features from Java 8. However, that should have no impact on the results:

public static void main(String... args) {
    System.out.println("Warmup...");
    for (int i = 0; i < 10000; ++i) {
        test(Collections.synchronizedMap(new HashMap<>()), 2, 1000);
        test(new ConcurrentHashMap<>(), 2, 1000);
    }
    System.out.println("Benchmark...");
    for (int i = 0; i < 5; ++i) {
        int threads = 4;
        int range = 500000 * (i + 1);
        System.out.printf("%2d * %7d: %s / %s\n",
            threads, range,
            test(Collections.synchronizedMap(new HashMap<>()), threads, range),
            test(new ConcurrentHashMap<>(), threads, range));
    }
}
public static String test(Map<Integer,Object> map, int threads, int range) {
    long duration = IntStream.range(0, 10)
        .mapToLong(i -> execute(
                IntStream.range(0, threads)
                .<Runnable>mapToObj(t -> () -> bulkPut(map, t * range, (t + 1) * range, new Object()))
                .toArray(Runnable[]::new)))
        .min().getAsLong();
    return String.format("%4.2fs",
        duration / 1000.0, threads, range);
}
public static <T> void bulkPut(Map<Integer,T> map, int from, int to, T value) {
    for (int i = from; i < to; ++i) {
        map.put(i, value);
    }
}
public static long execute(Runnable... runnables) {
    List<Thread> threads = new ArrayList<>();
    AtomicLong duration = new AtomicLong();
    for (Runnable runnable : runnables) {
        Thread thread = new Thread(() -> {
                long start = System.currentTimeMillis();
                try {
                    runnable.run();
                } finally {
                    long elapsed = System.currentTimeMillis() - start;
                    duration.accumulateAndGet(elapsed, Math::max);
                }
            });
        thread.start();
        threads.add(thread);
    }
    for (Thread thread : threads) {
        try {
            thread.join();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }
    return duration.get();
}