CODEWITHSUNDEEP
javaioexception
James Raitsev
James Raitsev

Reputation: 96391

Debugging the "Too many files open" issue

The application i am working on suddenly crashed with 

java.io.IOException: ... Too many open files

As i understand the issue it means that files are opened but not closed.

Stacktrace of course happens after the fact and can only help understand before what event error occurred.

What would be an intelligent way to search your code base to find this issue which only seems to occur when app is under high stress load.

Upvotes: 16

Views: 13346

Answers (4)

BlackJoker
BlackJoker

Reputation: 3191

  1. use lsof -p pid to check what cause leak of file references;

  2. use ulimit -n to see the limit of opened file references of a single process;

  3. check any IO resources in your project,are they released in time?,Note that,File,Process,Socket(and Http connections) are all IO resources.

  4. sometimes, too many threads will cause this problem too.

Upvotes: 15

Alex Chacha
Alex Chacha

Reputation: 423

Have you tried attaching to the running process using jvisualvm (Java 5.0 and later in the JDK bin directory). You can open the running process and do a heap dump (which if you have an older JDK you will need to analyze using eclipse or intellij or netbeans et. al.).

In JDK 7 the heap dump button is under the "Monitor" tab. It will create a heap dump tab, "Classes" sub-tab that you can check and see if any classes that open files exist in high quantity. Another very useful feature is heap dump compare, so you can take a reference heap dump, let your app run a bit and then take another heap dump and compare the two (the link to compare is on the "[heapdump]" tab you get when you take one. There is also a flag in java for taking a heapdump on crash or OOM exception, you can go down that route if comparing heap dumps does not give you an obvious class that is causing the problem. Also, "Instances" subtab in the heap dump diff will show you what has been allocated in the time between the two heap dumps which may also help.

jvisualvm is an awesome tool that does not get enough mentions.

Upvotes: 3

Rob I
Rob I

Reputation: 5737

What OS? If it's linux/mac, there is information under /proc that should help. On Windows, use the Process Explorer.

As far as searching the code base, perhaps look for code that catches or raises IOException - I think I/O methods that already catch/raise this have a high likelihood of needing a close() call.

Upvotes: 4

NPE
NPE

Reputation: 500357

I think the best way to use a tool specifically designed for the purpose, such as this one:

This little Java agent is a tool that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or handle leaks, and dump the list of currently open files and where/when/who opened them.

In addition, upon "too many open files" exception, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.

I seem to remember YourKit also having some facilities around this, but can't find any specific information at the moment.

Upvotes: 14

Related Questions