CODEWITHSUNDEEP
javamultithreadingmongodbthreadpoolmongodb-query
sofs1
sofs1

Reputation: 4176

How can I solve MongoWaitQueueFullException?

I run a java program which is a thread executor program that inserts thousands of documents to a table in mongodb. I get the following error

Exception in thread "pool-1-thread-301" com.mongodb.MongoWaitQueueFullException: Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 500 has been exceeded.
    at com.mongodb.PooledConnectionProvider.get(PooledConnectionProvider.java:70)
    at com.mongodb.DefaultServer.getConnection(DefaultServer.java:73)
    at com.mongodb.BaseCluster$WrappedServer.getConnection(BaseCluster.java:221)
    at com.mongodb.DBTCPConnector$MyPort.getConnection(DBTCPConnector.java:508)
    at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:456)
    at com.mongodb.DBTCPConnector.getPrimaryPort(DBTCPConnector.java:414)
    at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:176)
    at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:159)
    at com.mongodb.DBCollection.insert(DBCollection.java:93)
    at com.mongodb.DBCollection.insert(DBCollection.java:78)
    at com.mongodb.DBCollection.insert(DBCollection.java:120)
    at ScrapResults103$MyRunnable.run(MyProgram.java:368)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:695)

How can I resolve this? Please help me.

Upvotes: 7

Views: 20660

Answers (3)

Serhii Povísenko
Serhii Povísenko

Reputation: 3906

waitQueueMultiple is the product of maxConnectionPoolSize and threadsAllowedToBlockForConnectionMultiplier hence you can modify one of these three options to tune your app in MongoClientOptions with corresponding values and consume it to your MongoClient as an argument how it was done above here (marked as an answer) https://stackoverflow.com/a/25347310/2852528

BUT

I strongly recommend analysing your code first (where it communicates to the DB), and if no optimization is available (e.g. caching, using aggregation, paging etc.) then go ahead and change the options

Upvotes: 4

Mojtaba Yeganeh
Mojtaba Yeganeh

Reputation: 2922

you are in maxWaitQueueSize limit , so increase multiplier ;)

 MongoClientOptions options = MongoClientOptions.builder()
                .threadsAllowedToBlockForConnectionMultiplier(10)
                .build();

 MongoClient mongo = new MongoClient("127.0.0.1:27017", options);
 //run 2000 threads and use database ;)

Upvotes: 4

Manjunath
Manjunath

Reputation: 1685

You need to check what is the connections per host value which you have given while setting up connection (looking at the exception I think you would have set it to 500).

MongoClientOptions.Builder builder = new MongoClientOptions.Builder();
builder.connectionsPerHost(200);
MongoClientOptions options = builder.build();
mongoClient = new MongoClient(URI, connectionOptions);

An ideal way of setting the connections per host would be by trial and error but you need to make sure that the value which you set should not exceed the number of connections you can have by opening the mongo shell and executing:

db.serverStatus().connections.available

Upvotes: 7

Related Questions