CODEWITHSUNDEEP
javamultithreadingexecutorservice
avalon
avalon

Reputation: 2291

ThreadPoolExecutor runs only first runnable

I have a thread pool executor

private ExecutorService exec=Executors.newCachedThreadPool();

and I have a class

class MyCallable implements Callable<String>{

    private ReentrantLock lock=new ReentrantLock();

    private Condition cond=lock.newCondition();


    public String call(){

    System.out.println("TEST!!);
    try{
      lock.lock();
      cond.await();
    }finally{
       lock.unlock();
    }
   }
}

and I do:

for(int i=0;i<10000;i++){
   exec.submit(new MyCallable());
}

I want to have 10000 threads with waiting callables, but I can see only one TEST!! in my logs, so it submits my task only once, why it freezes??? My callable has it`s own lock in each object instance, how to have 1000 waiting threads?

Upvotes: 0

Views: 987

Answers (3)

Stephen C
Stephen C

Reputation: 718698

I took your code, filled in the missing bits, fixed the compilation errors and ran it. It prints "TEST!!" lots of times. If I reduce the number of tasks to a reasonable number, it runs to completion. If I leave it at 10,000 the test fails with an OOME saying that it can't allocate any more thread stacks.

I want to have 10000 threads with waiting callables

My testing suggests that that many threads is going to cause you to run out of memory. It is really, really not a good idea.

Here is the exact code I used:

package test;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class ExecutorTest {

    public static void main(String[] args) {
        ExecutorService exec = Executors.newCachedThreadPool();
        for (int i = 0; i < 10000; i++) {
            exec.submit(new MyCallable());
        }
    }
}


package test;

import java.util.concurrent.Callable;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

class MyCallable implements Callable<String> {

    private ReentrantLock lock = new ReentrantLock();

    private Condition cond = lock.newCondition();

    public String call() {

        System.out.println("TEST!!");
        try {
            lock.lock();
            cond.await();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            lock.unlock();
        }
        return "foo";
    }
}

Here's the while(true) {await} test:

[steve@newbox ~]$ cat ./workspace/test/src/test/CondAwait.java 
package test;

import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

public class CondAwait {

    public static void main(String[] args) throws InterruptedException {
         ReentrantLock lock = new ReentrantLock();

         Condition cond = lock.newCondition();
         lock.lock();
         while(true){
             System.out.println("Hi!");
             cond.await();
         }
    }

}
[steve@newbox ~]$ java -cp ./workspace/test/bin test.CondAwait 
Hi!
^C[steve@newbox ~]$

In short, the main thread freezes in the first await call ... as I would have expected.

I agree that it`s not a good idea, but those threads will be like a request cache, user sends a request, it's processor should be put into the thread pool, if we have 10k requests per second, you will see 10k threads in the pool. Please suggest how to do it better.

Use a bounded thread pool, and set the bound to be roughly equal to the number of cores that you have.

Sure, you won't have your 10,000 callables all suspended at the same time, but that is a GOOD THING.

If these callables are meant to simulate requests that wait a few seconds for a something external to respond, then a thread pool of up to 100 might be reasonable. But if you really, really need massive parallelism then you should be looking at something that uses NIO selectors so that a small number of worker threads can run a large number of requests, interleaving them rather than blocking in I/O.

Upvotes: 1

hemant1900
hemant1900

Reputation: 1226

I just tried you program and it works fine for me. I could see multiple tasks running. 

public class MyCallable implements Callable<String> {

    private ReentrantLock lock = new ReentrantLock();
    private Condition cond = lock.newCondition();

    public String call() {
        System.out.println("TEST!!");
        try {
           lock.lock();
           cond.await();
        } catch (InterruptedException e) {
           e.printStackTrace();
        } finally {
           lock.unlock();
        }
        return "done";
    }

    public static void main(String[] args) {
        ExecutorService service = Executors.newCachedThreadPool();
        for (int i = 0; i < 10; i++) {
           service.submit(new MyCallable());
        }
    }
}

And I would suggest 10000 concurrent tasks is not a good idea. How many processors you have? ;)

Upvotes: 0

Steven Pessall
Steven Pessall

Reputation: 993

Have a look at the class java.util.concurrent.CyclicBarrier. As stated in its JavaDoc it is:

A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point.

Each of your threads needs to call the method await of the barrier. This means that the thread is suspended until either the Barrier has reached the expected number of threads or a timeout has occured.

Upvotes: 0

Related Questions