gstackoverflow
gstackoverflow

Reputation: 37106

Performance degradation when I use async requests in LdapConnectionPool in comparison with LdapConnection

I am investigating performance. I've added 100'000 entries into my LDAP server.

The aim is to read 10'000 entries from the LDAP server (I use different size batches for that).

  1. Using async API and independent LdapConnection (size is 20)
  2. Using async API and LdapConnection from LdapConnectionPool (size is 20)

Let me provide some of my code:

Case #1:

@ParameterizedTest
@EnumSource(BatchSize::class)
fun getByBatchAsyncBasicListener(batchSize: BatchSize) {
    val areWholeBatches = ENTRY_COUNT % batchSize.size == 0
    val batchCount = if (areWholeBatches) {
        ENTRY_COUNT / batchSize.size
    } else {
        ENTRY_COUNT / batchSize.size + 1
    }
    val entries = HashSet<SearchResultEntry>()
  
        IntRange(0, batchCount - 1).map { batchIndex ->
            IntRange(1, batchSize.size)
                .filter { entryIndex -> entryIndex + (batchIndex * batchSize.size) <= ENTRY_COUNT }
                .map { entryIndex ->
                    getEntryDn(entryIndex + (batchIndex * batchSize.size))
                }.let { idList ->
                    getEntriesBatchAsync(baseSearchDn, idList)
                }
        }.map { (asyncRequestId, listener) ->
            asyncRequestId.get()
            entries.addAll(listener.searchEntries)
        }

}

fun getEntriesBatchAsync(
    baseDn: String,
    dns: List<String>,
    vararg attributes: String,
): Pair<AsyncRequestID, BasicAsyncSearchResultListener> {
    val resultListener = BasicAsyncSearchResultListener()
    val searchRequest = SearchRequest(
        resultListener,
        baseDn,
        SearchScope.ONE,
        DereferencePolicy.NEVER,
        0,
        0,
        false,
        getIdsFilter(dns),
        *attributes,
    )
    val asyncSearch = getConnection().asyncSearch(searchRequest)
    return Pair(asyncSearch, resultListener)
}

Some details about getConnection(): at this case I create up to 20 connections and return them in random manner.

The result is:

image

Case #2:

@ParameterizedTest
@EnumSource(BatchSize::class)
fun getByBatchAsyncBasicListenerConnectionPoolNew(batchSize: BatchSize) {
    require(ENTRY_COUNT > batchSize.size)
    val areWholeBatches = ENTRY_COUNT % batchSize.size == 0
    val batchCount = if (areWholeBatches) {
        ENTRY_COUNT / batchSize.size
    } else {
        ENTRY_COUNT / batchSize.size + 1
    }
    val entries = HashSet<SearchResultEntry>()
    try {
        IntRange(0, batchCount - 1).map { batchIndex ->
            IntRange(1, batchSize.size)
                .filter { entryIndex -> entryIndex + (batchIndex * batchSize.size) <= ENTRY_COUNT }
                .map { entryIndex ->
                    getEntryDn(entryIndex + (batchIndex * batchSize.size))
                }.let { idsBatch ->
                    getEntriesBatchAsyncPoolNew(baseSearchDn, idsBatch)
                }
        }.flatMap { (asyncReq, listener) ->
            asyncReq.get()
            listener.searchEntries
        }.let { result -> entries.addAll(result) }       
}

fun getEntriesBatchAsyncPoolNew(
    baseDn: String,
    dns: List<String>,
    vararg attributes: String,
): Pair<AsyncRequestID, BasicAsyncSearchResultListener> =
    SearchRequest(
        BasicAsyncSearchResultListener(),
        baseDn,
        SearchScope.ONE,
        DereferencePolicy.NEVER,
        0,
        0,
        false,
        getIdsFilter(dns),
        *attributes,
    )
        .let { searchReq ->
            getConnectionPool().processRequestsAsync(listOf(searchReq), -1)
                .map { asyncRequestID ->
                    Pair(asyncRequestID, searchReq.searchResultListener as BasicAsyncSearchResultListener)
                }[0]
        }

In this case, getConnectionPool() returns the same object of type LDAPConnectionPool which is created like this:

    val startTLSPostConnectProcessor = StartTLSPostConnectProcessor(SSLUtil(TrustAllTrustManager()).createSSLContext())
    val ldapConnectionPool = LDAPConnectionPool(exampleConnection, 1, 20, startTLSPostConnectProcessor)
    ldapConnectionPool.setBindRequest(simpleBindRequest)

The result is:

image

I tried to make thread dump during test run and I see a lot of reader threads. Maybe it could help ti understand the root cause.

image

image

As you can see, using the LDAPConnectionPool makes performance much worse.

Do I use API in a wrong way? Is there some way to improve the performance?

Upvotes: 0

Views: 77

Answers (0)

Related Questions