java.net.UnknownHostException and java.net.SocketException: Too many open files

Question

I am running a web application on Tomcat. My application make connection to other web services to fulfill request from client. Sometime i get java.net.UnknownHostException while opening URL connection and then after sometimes I start getting java.net.SocketException: Too many open files. And my server stops accepting connection further. Please guide.

String  response;
    HttpURLConnection conn = null;
BufferedReader rd = null;
InputStream in = null;
try
{
    // Send data
    String urlStr = URL;
    URL url = new URL(urlStr);
    conn = (HttpURLConnection)url.openConnection();
    conn.setRequestProperty("content-type", "application/x-www-form-urlencoded");
    int contentLength = conn.getContentLength();
    //   System.out.println("content length 1" +  contentLength);
    if (contentLength <= 0)
    {
        InputStream in1 = (conn).getErrorStream();
        if (in1 != null)
        {
            in1.close();
        }
        conn.getInputStream().close();
        in = null;
        conn.disconnect();
        conn = null;
        return null;
    }
    in = conn.getInputStream();
    if (conn.getResponseCode() != 200)
    {
        InputStream in1 = (conn).getErrorStream();
        if (in1 != null)
        {
            in1.close();
        }
        conn.getInputStream().close();
        in = null;
        conn.disconnect();
        conn = null;
        in.close();
        in = null;
        return null;
    }
    // Get the response
    rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
    StringBuffer stringBuffer = new StringBuffer();
    String line = null;
    while ((line = rd.readLine()) != null)
    {
        stringBuffer.append(line);
    }
    response = stringBuffer.toString();
    System.out.println("full ads response = " + response);
}
catch (Exception ex)
{
    //   ex.printStackTrace();
}
finally
{
    try
    {
        if (conn != null)
        {
            InputStream in1 = (conn).getErrorStream();
            if (in1 != null)
            {
                in1.close();
            }
            conn.getInputStream().close();
            in = null;
            conn.disconnect();
            conn = null;
        }
        if (rd != null)
        {
            rd.close();
            rd = null;
        }
        if (in != null)
        {
            in.close();
            in = null;
        }
    }
    catch (Exception e)
    {
    }
}
return null;        

Answer 1

I recently came across this issue when attempting to make lots of connections to an unreachable URL. The connection would fail with UnknownHostException, but eventually fail with java.net.SocketException: Too many open files.

The issue is, an underlying native socket handle was created by HttpURLConnection. It appears that this handle isn't released until after the object is garbage collected. In my case, there wasn't enough memory being used to trigger a garbage collection. If the GC never runs, the native handles aren't released.

My solution was to periodically call System.gc() when getting the UnknownHostExceptions. I know that seems like a kludge but it worked, the underlying native handles were released and the issue disappeared.

Answer 2

UnknownHostException just means the URL you used refers to a non-existent hostname, or possibly that the hostname's DNS record or system was in transition at that instant.

The socket leak may be caused by your excessive cleanup code below, see comments.

When you suppress all exceptions you don't have a hope in Hades of finding out what's going wrong. So that's the first thing to fix. Never ignore exceptions. Log them. Print their stack traces. Do something.

However your code has numerous other problems as well:

HttpURLConnection conn = null;
BufferedReader rd = null;
InputStream in = null;
try
{
    // Send data
    String urlStr = URL;

This variable is redundant. Just use URL below.

    URL url = new URL(urlStr);
    conn = (HttpURLConnection)url.openConnection();
    conn.setRequestProperty("content-type", "application/x-www-form-urlencoded");

OK so far.

    int contentLength = conn.getContentLength();
    //   System.out.println("content length 1" +  contentLength);
    if (contentLength <= 0)

It's very unusual to check for zero content length. Is this a proxy for checking for errors? I would simply remove this test.

    {
        InputStream in1 = (conn).getErrorStream();
        if (in1 != null)
        {
            in1.close();
        }
        conn.getInputStream().close();
        in = null;
        conn.disconnect();
        conn = null;
        return null;
    }

All the above is redundant. You already have it in the finally block. Remove it.

    in = conn.getInputStream();
    if (conn.getResponseCode() != 200)

Now that is how to check for errors. However you must call getResponseCode() before you call getInputStream(), otherwise the latter may throw FileNotFoundException when you might be more interested in the response code being 404.

    {
        InputStream in1 = (conn).getErrorStream();
        if (in1 != null)
        {
            in1.close();
        }
        conn.getInputStream().close();
        in = null;
        conn.disconnect();
        conn = null;
        in.close();
        in = null;
        return null;
    }

Again, all the above is redundant. You already have it in the finally block. Remove it.

    // Get the response
    rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
    StringBuffer stringBuffer = new StringBuffer();
    String line = null;

Initialization of line is redundant. The variable is assigned in the next line. Remove it.

    while ((line = rd.readLine()) != null)
    {
        stringBuffer.append(line);
    }
    response = stringBuffer.toString();
    System.out.println("full ads response = " + response);
}
catch (Exception ex)

You should only catch IOException here.

{
    //   ex.printStackTrace();

Never ignore an exception.

}
finally
{
    try
    {
        if (conn != null)
        {
            InputStream in1 = (conn).getErrorStream();
            if (in1 != null)
            {
                in1.close();
            }

You don't need to get or close the error stream. Remove all this.

            conn.getInputStream().close();
            in = null;

Setting this variable to null is pointless. It's a local variable so it's about to fall out of scope anyway.

            conn.disconnect();
            conn = null;

Ditto.

        }
        if (rd != null)
        {
            rd.close();
            rd = null;
        }

You've already closed the input stream. This is redundant. Remove this block.

        if (in != null)
        {
            in.close();
            in = null;
        }

You've already closed the input stream of the connection. This is all redundant. Remove it.

    }
    catch (Exception e)
    {
    }

Again you should only be catching IOException here, and you shouldn't be ignoring the error. Personally I would have the method throw IOException and not have any catch blocks at all. Returning null isn't very useful to the caller. He's generally better off knowing exactly what went wrong so he can make a useful decision as to what to do about it.

}
return null;        

Answer 3

Try increasing your linux server open files limit to fix java.net.SocketException: Too many open files. The limit of max files is stored in this file in linux:

/proc/sys/fs/file-max

to check the open files at a current time, you can use:

lsof

to check the no. of open files, use:

lsof | wc -l

also, in addition to conn.disconnect(), close your connection conn.close(). Make sure that you close inputreader as well. All such close should be in finally block, so that connection is also closed in case of any exception.

Answer 4

Are you running on windows or linux ? Check this article, hope it helps : http://edmund.haselwanter.com/en/blog/2009/03/13/tomcat-too-many-open-files/