CODEWITHSUNDEEP
javamultithreadingconcurrency
kommradHomer
kommradHomer

Reputation: 4210

threadPoolExecutor outOfMemoryException

i have a class with a run method , and the timer in the main method of that class is calling the class with this code: 

Timer timer = new Timer();
timer.scheduleAtFixedRate(new LegacySmsSender(), 0, 2*1000);

in the run method , i declare a ThreadExecutorPool :

ThreadPoolExecutor packetProcessorThreadPool =
        new ThreadPoolExecutor(4,
                                4,
                                Long.MAX_VALUE,
                                TimeUnit.DAYS,
                                new LinkedBlockingQueue<Runnable>(),
                                new MyThreadFactory("packetProcessorThreadPool")
                                );

i create 4 PacketProcessors with new PacketProcessor() and do packetProcessorThreadPool.submit and hold their Future returns in a list. Then I wait for all of them to finish in a loop :

for(Future<?> f:packetProcessorList)
{
   System.out.println("before f.get() "+new Date());
   f.get();
}

And inside those PacketProcessor Class's run methods , they declare a ThreadPoolExecutor and create and submit threads in numbers between 1-5000(its usually creating 6-7 threads) and the threadPoolExecutor code in PacketProcessor is like this:

ThreadPoolExecutor commonThreadPool =
        new ThreadPoolExecutor(
                        20,
                        20,
                        Long.MAX_VALUE,
                        TimeUnit.DAYS,
                        new LinkedBlockingQueue<Runnable>(),
                        new MyThreadFactory("commonThreadPool"));

and in the end , i ran this for some 20 minutes , i check the VisualVM , my memory usage and live thread count rises all the time. What is the problem ?

Note : Please never hesitate to ask me for more info or questions

here is the screenshot with some info :

enter image description here

edit 1 :

i took a heap Dump of 270 MB. i found 160mb of char[] . and i found like 1000-2000 thousand query strings. i build query strings with a StringBuilder. why they dont get GCed ?

Upvotes: 0

Views: 2236

Answers (3)

Tim Bender
Tim Bender

Reputation: 20442

The problem is that you've told all of the ExecutorServices that you create to make threads and then keep them alive forever. This is very plainly obvious in the code:

new ThreadPoolExecutor(
                        20,
                        20,
                        Long.MAX_VALUE,
                        TimeUnit.DAYS,
                        new LinkedBlockingQueue<Runnable>(),
                        new MyThreadFactory("commonThreadPool"));

The third and fourth arguments basically say "keep idle threads alive for (practically) infinity days".

You can solve this in a couple ways:

  1. Call ExecutorService.shutdown on the executor instances when you are done passing them tasks.
  2. The best solution, use a single ScheduledExecutorService and a more reasonable task distribution logic to limit the number of threads and drive up performance by preventing thread context switching.

Upvotes: 0

Chobicus
Chobicus

Reputation: 2094

in the run method , i declare a ThreadExecutorPool

If you declare ThreadExecutorPool in run method that gets executed every 2 seconds you will get many threads in a few minutes.

Upvotes: 1

artbristol
artbristol

Reputation: 32427

Executors are meant to be declared once on application startup and reused. Otherwise you might as well just create individual threads on demand. If you keep creating new executors during application execution, their threads will be left running and so the thread count will keep increasing.

So just create the executors using a DI framework and inject them into your code. Or if it's a small project, put them in a static field.

Upvotes: 2

Related Questions