CODEWITHSUNDEEP
springcloud-foundry
Dean Peterson
Dean Peterson

Reputation: 467

"Too Many Open Files" error on Cloudfoundry

I receive the dreaded "Too Many Open Files" error after about 5 minutes running my application. This is a showstopper for me. I do know there is a 256 open file limit. I ran lsof to track down if I have a leak. I found that many of the open handles are simply connections Tomcat and other processes must make. The "nginx" process seems to be the only one that fluctuates but it still only goes to a maximum around 81. My application does not seem to leak file descriptors. I absolutely love Cloud Foundry. It is the first PAAS that hasn't required I refactor my application to make things work. When is the file limit going to be raised? I use Micro Cloud Foundry for testing but I want to run on the Hosted Cloud Foundry as soon as possible. I get this error in both versions. Is there a way around this? I tried modifying the limit on the Micro Cloud instance but I get errors saying I do not have the rights to make that kind of change. Any help or suggestions on this?

Upvotes: 0

Views: 1135

Answers (2)

Matt Reider
Matt Reider

Reputation: 36

The new file descriptor quota is in the following database migration:

https://github.com/cloudfoundry/cloud_controller_ng/blob/master/db/migrations/20130131184954_new_initial_schema.rb

Line 185.

This particular setting will not take effect on our http://cloudfoundry.com until the April time frame when we emerge from our beta status and our "Next Generation" components are in production.

If you run your own version of Cloud Foundry, you could run this migration, assuming you are using cloud_controller_ng.

The Micro Cloud Foundry we are using internally for development does have the new cloud controller in it. You can read how we get that running for our own purposes here:

We are in a bit of a transition period as we deprecate the legacy bits and move towards these NG components. Apologies for the hitches, but they will be worth the cost. Thanks for your patience.

Best,

Matt Reider Product Manager Cloud Foundry

Upvotes: 2

Dean Peterson
Dean Peterson

Reputation: 467

I logged in as the root user. I modified both the /var/vcap/packages/dea/dea/lib/dea/agent.rb file and the /etc/security/limits.conf file.

For the agent.rb file I followed the instructions here: http://mdahlman.wordpress.com/2012/04/20/micro_cloud_foundry/

For the limits.conf file I followed the instructions here: http://myadventuresincoding.wordpress.com/2010/10/09/ubuntu-increasing-the-maximum-number-of-open-files/

I am not sure which fix helped or if it required modifying both files. The application seems to be working now so I am moving on. If someone has a better solution I would be happy to hear it.

This only allows me to run my application on a self hosted micro cloud foundry. It would be much better if someone had a solution for the hosted version. Unfortunately I would have to find some way to limit my app to use under 256 file descriptors and that is not likely to happen.

Upvotes: 0

Related Questions