CODEWITHSUNDEEP
javamemoryconcurrency
Allerson
Allerson

Reputation: 109

ExecutorService takes too much memory

Here is the situation: I have to use Xfire to exchange data with a server.The server can not handle to much concurrency.50 is the limit.So I plan to use ExecutorService to limit the amount of concurrent threads.Then Q&A find the usage of memory is nearly 100% when it has 50 concurrency after the program is running 20 mins.

Here is my code:

public class CompletionServiceImpl {

private static Logger logger = Logger.getLogger("BackgroundLog");

private int threadNum;

private ExecutorService executor = null;

private CompletionService<Integer> sc = null;

private static CompletionServiceImpl completionServiceImpl = null;

private CompletionServiceImpl(){
    this.threadNum = getThreadNum();
    this.executor = Executors.newFixedThreadPool(threadNum);
    this.sc = new ExecutorCompletionService<Integer>(executor);
}

/***
* get the size of thread pool
***/
private int getThreadNum(){

    int threadNum = 5;
    Properties props = new Properties();
    try {
        props.load(Thread.currentThread().getContextClassLoader().getResourceAsStream("tlWeixinconfig.properties"));
        threadNum = Integer.parseInt(props.getProperty("THREAD_NUM"));
    } catch (IOException e) {
        logger.error(e.getMessage(), e);
    }
    return threadNum;
}


public static CompletionServiceImpl getInstance(){
    if(completionServiceImpl == null){
        synchronized(CompletionServiceImpl.class){
            if(completionServiceImpl == null){
                logger.info("thread pool is initialized.");
                completionServiceImpl = new CompletionServiceImpl();
            }
        }
    }
    return completionServiceImpl;
}

public ExecutorService getExecutor() {
    return executor;
}

public CompletionService<Integer> getSc() {
    return sc;
}

}

public class MyCallable implements Callable{

private static Logger logger = Logger.getLogger("BackgroundLog");

private String id;

private String usr;

private String type;

private String expireDate;

private String billingURL;

private int timeout;

private int result;

public MyCallable(String id, String usr,String type, String expireDate, String billingURL,int timeout,int result){
    super();
    this.id = id;
    this.usr = usr;
    this.type = type;
    this.expireDate = expireDate;
    this.billingURL = billingURL;
    this.timeout = timeout;
    this.result = result;
}

 private int newinsertdrawcn(int result)throws Throwable {
        try {
            URL url = new URL(billingURL);
            HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection();
            httpConnection.setConnectTimeout(timeout);

            httpConnection.connect();
            Client client = new Client(httpConnection.getInputStream(), null);
            client.setProperty(CommonsHttpMessageSender.HTTP_TIMEOUT, String.valueOf(timeout));
            client.setProperty(CommonsHttpMessageSender.DISABLE_KEEP_ALIVE, "true");
            client.setProperty(CommonsHttpMessageSender.DISABLE_EXPECT_CONTINUE, "true");

            Object[] results = client.invoke("drawcn", new Object[] {id, usr, type, expireDate });
            if (results.length > 0) {
                result = Integer.parseInt(results[0].toString());
            }
        } catch (Throwable t) {
            throw t;
        }
        return result;
    }

@Override
public  Integer call(){
    try{
        result = newinsertdrawcn(result);
    }catch(Throwable t){
        logger.error(t.getMessage(),t);
    }
    return result;
}

}

Can anybody explain why and how to solve this problem?

or is there someone knows how to limit the amount of concurrent threads?

Upvotes: 0

Views: 1672

Answers (1)

Stephen C
Stephen C

Reputation: 719346

There are two possibilities:

  • It is caused by having too many threads in the thread pool. Each thread could have a 2Mb stack, and active threads will most likely have objects on their respective stacks.

  • It is caused by memory leaks that are exacerbated by having lots of threads active. (One possibility is a memory leak due to not using thread-local variables properly.)

You'll need to investigate with a memory profiler.

how to limit the amount of concurrent threads?

Simple. Reduce the executor's thread pool size.

Upvotes: 1

Related Questions