CODEWITHSUNDEEP
javajbossldap
awi
awi

Reputation: 2916

LDAP requests hanging for 15 seconds

I have JBoss application server that is using LDAP for authentication. Lately we have noticed that there are a lot of slow requests (> 15 seconds).

I did some threaddumps of the server and noticed that many threads where waiting on a lock: com.sun.jndi.ldap.LdapRequest@54ceac

java.lang.Object.wait(Native Method)
com.sun.jndi.ldap.Connection.readReply(Connection.java:418)
com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:340)
com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:192)
com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2637)
com.sun.jndi.ldap.LdapCtx.(LdapCtx.java:283)
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:134)
com.sun.jndi.url.ldap.ldapURLContextFactory.getObjectInstance(ldapURLContextFactory.java:35)
javax.naming.spi.NamingManager.getURLObject(NamingManager.java:584)

All of the requests I have seen that have been waiting in this state have used more than 15 seconds to complete. We are monitoring the LDAP-server and all requests from the monitoring tool finish in less than 200 ms. This makes me think this is a problem with the com.sun.jndi.ldap code. Decompiling the com.sun.jndi.ldap.Connection class (jdk1.5.0_12) I see this:

BerDecoder readReply(LdapRequest ldaprequest) throws IOException, NamingException
{
_L2:
    BerDecoder berdecoder;
    if((berdecoder = ldaprequest.getReplyBer()) != null)
    break; /* Loop/switch isn't completed */
    try
    {
label0:
    {
        synchronized(this)
        {
        if(sock == null)
            throw new ServiceUnavailableException((new StringBuilder()).append(host).append(":").append(port).append("; socket closed").toString());
        }
        synchronized(ldaprequest)
        {
        berdecoder = ldaprequest.getReplyBer();
        if(berdecoder == null)
        {
            ldaprequest.wait(15000L);
            break label0;
        }
        }
        break; /* Loop/switch isn't completed */
    }
    }
    ...

There is apparently a hardcoded timeout of 15000 milliseconds.

Do anyone have any ideas for a fix/workaround?

Upvotes: 6

Views: 9631

Answers (4)

loopkin
loopkin

Reputation: 11

You're using an old jdk1.5 (jdk1.5.0_12).

I'm having the same issue with jdk1.5_16 using tomcat 5.5. We have a thread which is waiting for a ldap response and it blocks all other thread because I don't know about JBoss but in tomcat at least all ldap authentication are done sequentially.

If you look in the decompiled code you pasted, after the wait 15 second you have a break label0 which is effectively a loop. So it will loop until ldap answer (no timeout!).

I'm not sure in which version it was fixed, but in 1.5.0_22 the code is now:

BerDecoder readReply(LdapRequest paramLdapRequest)
  throws IOException, NamingException
{
   BerDecoder localBerDecoder;
   int i = 0;

   while (((localBerDecoder = paramLdapRequest.getReplyBer()) == null) && (i == 0)) {
     try
     {
       synchronized (this) {
         if (this.sock == null) {
           throw new ServiceUnavailableException(this.host + ":" + this.port + "; socket closed");
         }
       }

       synchronized (paramLdapRequest)
       {
         localBerDecoder = paramLdapRequest.getReplyBer();
         if (localBerDecoder == null)
           if (this.readTimeout > 0)
           {
             paramLdapRequest.wait(this.readTimeout);
             i = 1;
           } else {
             paramLdapRequest.wait(15000L);
           }
         else
           break label163:
       }
     }
     catch (InterruptedException localInterruptedException) {
       throw new InterruptedNamingException("Interrupted during LDAP operation");
     }

 }

So now, if you provide a timeout value it will wait for that time, and then exit the loop. An this should unlock the authentication queue.

Upvotes: 1

Zoran Regvart
Zoran Regvart

Reputation: 4690

Sounds similar to this bug, have you tried inspecting the network traffic with packet sniffer to check for this condition?

Upvotes: 1

bkdougan
bkdougan

Reputation:

I've seen something like this before when using LDAP to connect to an ActiveDirectory box (in a network where there's more than one server). It ended up being a DNS issue and we just needed to flush our DNS cache ("ipconfig /flushdns" on a windows box). This may or may not be your issue, just thought it'd be worth a try.

Upvotes: 0

phatmanace
phatmanace

Reputation: 5021

looks to me as though it only waits if the reply is null - wondering if there is some kind of version mismatch that is causing your app not be able to parse the reply from your server.

Have you tried attaching source, and seeing you can set a breakpoint in eclipse.

-ace

Upvotes: 0

Related Questions