LDAP VLV throws error "Other sort requests already in progress"

Question

I am trying to implement pagination in LDAP using vlv, using reference from document https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/controls/VirtualListViewRequestControl.html

it is working fine with single thread, but when try with multiple threads concurrently upto 5 threads it works fine, but as number of threads increased only 5 threads can run successfully exceed threads got failed with below error message:

LDAPException(resultCode=51 (busy), numEntries=0, numReferences=0, diagnostiMessage='Other sort requests already in progress', ldapSDKVersion=5.1.1..

I am using OpenLDAP, Unboundid api for connection with Java. About data size it is around 100k.

• Tried with single connection and multiple connections(with multiple concurrent threads) getting same error in both cases.
• Tried to synchronize block for fetching data.
• On exception, make thread to wait and try again. All above things didn't worked, threads cannot fetch data from LDAP. After trying to close and reconnect connection as described in https://www.openldap.org/lists/openldap-technical/201107/msg00006.html failed thread can fetch data but after retry lot of times, in my case thread retried about 2k times then it started fetching data.

Is there any better solution, retrying 2k times and getting result is not a good option.

Answer 1

From my experience in JAVA, it is better to use thread pools which shifts your solution from "how to manage threads" into a more robust and tasks oriented one.

To the point (of your use case): you may want to define a thread pool with a fixed size of thread. The pool will manage all incoming loads by re-using the threads in the pool. This is very efficient because more threads does not equal more performance. You may want to use a mechanism that re-uses threads, rather than just open and close threads and use too much of them.

You may start with something similar to this:

ExecutorService executorService = Executors.newFixedThreadPool(10);

Future<SearchResult> task1 = executorService.submit(() -> {
  // your logic goes here
  return result;
});


SearchResult result = task1.get();

This is an over simplified piece of code but you can clearly see that:

• Tasks may be initiated from a stack (dynamically)
• Results can be fetched by using a listener (you grab results only when they are ready - no polling needed)
• The thread pool manages loads - so you can tweak your configuration and boost performance without changing your code (perfect for various environments that may want to configure your solution to suit their hardware profile)

I think you should give it a try.. after all - retrying 2000 times before success is really not that kind of idle 🙃