Why does my Java program's performance drop significantly after startup?

Question

I am writing a small Java application to analyze a large number of image files. For now, it finds the brightest image in a folder by averaging the brightness of every pixel in the image and comparing it to the other images in the folder.

Sometimes, I get a rate of 100+ images/second right after startup, but this almost always drops to < 20 images/second, and I'm not sure why. When it is at 100+ images/sec, the CPU usage is 100%, but then it drops to around 20%, which seems too low.

Here's the main class:

public class ImageAnalysis {

    public static final ConcurrentLinkedQueue<File> queue = new ConcurrentLinkedQueue<>();
    private static final ConcurrentLinkedQueue<ImageResult> results = new ConcurrentLinkedQueue<>();
    private static int size;
    private static AtomicInteger running = new AtomicInteger();
    private static AtomicInteger completed = new AtomicInteger();
    private static long lastPrint = 0;
    private static int completedAtLastPrint;

    public static void main(String[] args){
        File rio = new File(IO.CAPTURES_DIRECTORY.getAbsolutePath() + File.separator + "Rio de Janeiro");

        String month = "12";

        Collections.addAll(queue, rio.listFiles((dir, name) -> {
            return (name.substring(0, 2).equals(month));
        }));

        size = queue.size();

        ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() + 1);

        for (int i = 0; i < 8; i++){
            AnalysisThread t = new AnalysisThread();
            t.setPriority(Thread.MAX_PRIORITY);
            executor.execute(t);
            running.incrementAndGet();
        }
    }

    public synchronized static void finished(){
        if (running.decrementAndGet() <= 0){

            ImageResult max = new ImageResult(null, 0);

            for (ImageResult r : results){
                if (r.averageBrightness > max.averageBrightness){
                    max = r;
                }
            }

            System.out.println("Max Red: " + max.averageBrightness + " File: " + max.file.getAbsolutePath());
        }
    }

    public synchronized static void finishedImage(ImageResult result){
        results.add(result);
        int c = completed.incrementAndGet();

        if (System.currentTimeMillis() - lastPrint > 10000){
            System.out.println("Completed: " + c + " / " + size + " = " + ((double) c / (double) size) * 100 + "%");
            System.out.println("Rate: " + ((double) c - (double) completedAtLastPrint) / 10D + " images / sec");
            completedAtLastPrint = c;

            lastPrint = System.currentTimeMillis();
        }
    }

}

And the thread class:

public class AnalysisThread extends Thread {

    @Override
    public void run() {
        while(!ImageAnalysis.queue.isEmpty()) {
            File f = ImageAnalysis.queue.poll();

            BufferedImage image;
            try {
                image = ImageIO.read(f);

                double color = 0;
                for (int x = 0; x < image.getWidth(); x++) {
                    for (int y = 0; y < image.getHeight(); y++) {
                        //Color c = new Color(image.getRGB(x, y));
                        color += image.getRGB(x,y);
                    }
                }

                color /= (image.getWidth() * image.getHeight());

                ImageAnalysis.finishedImage((new ImageResult(f, color)));

            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        ImageAnalysis.finished();
    }
}

Answer 1

The problem I could see here is the usage of queues in the high-performance concurrency model you are looking for. Using a queue is not optimal while using with a modern CPU design. Queue implementations have write contention on the head, tail and size variables. They are either always close to full or close to empty due to differences in pace between consumers and producers, especially while using in a high I/O situation. This results in high levels of contention. Further, in Java queues are significant source of garbage.

What I suggest is to apply Mechanical Sympathy while designing your code. One of the best solution you can have is the usage of LMAX Disruptor, which is a high performance inter-thread messaging library, which is aimed to solve this concurrency problem

Additional References

http://lmax-exchange.github.io/disruptor/files/Disruptor-1.0.pdf

http://martinfowler.com/articles/lmax.html

https://dzone.com/articles/mechanical-sympathy

http://www.infoq.com/presentations/mechanical-sympathy

Answer 2

I see two reasons why you may experience CPU usage deterioration:

• your tasks are very I/O intensive (reading images - ImageIO.read(f));
• there is thread contention over the synchronized method that your threads access;

Further the sizes of the images may influence execution times.

To exploit parallelism efficiently I would suggest that you redesign your app and implement two kind of tasks that would be submitted to the executor:

• the first tasks (producers) would be I/O intensive and will read the image data and queue it for in-memory processing;
• the other (consumers) will pull and analyze the image information;

Then with some profiling you will be able to determine the correct ratio between producers and consumers.

Answer 3

You appear to have a mixed up both using a thread pool and creating threads of your own. I suggest you use on or the other. In fact I suggest you only use the fixed thread pool

Most likely what is happening is your threads are getting an exception which is being lost but killing the task which kills the thread.

I suggest you just the the thread pool, don't attempt to create your own threads, or queue as this is that the ExecutorService does for you. For each task, submit it to the pool, one per image and if you are not going to check the Error of any task, I suggest you trap all Throwable and log them otherwise you could get a RuntimeExcepion or Error and have no idea this happened.

If you have Java 8, a simpler approach would be to use parallelStream(). You can use this to analyse the images concurrently and collect the results without having to divide up the work and collect the results. e.g

List<ImageResults> results = Stream.of(rio.listFiles())
                                   .parallel()
                                   .filter(f -> checkFile(f))
                                   .map(f -> getResultsFor(f))
                                   .list(Collectors.toList());