Rails app (mod_rails) hangs every few hours

Question

I'm running a Rails app through Phusion Passenger (mod_rails) which will run smoothly for a while, then suddenly slow to a crawl (one or two requests per hour) and become unresponsive. CPU usage is low throughout the whole ordeal, although I'm not sure about memory.

Does anyone know where I should start to diagnose/fix the problem?

Update: restarting the app every now and then does fix the problem, although I'm looking for a more long-term solution. Memory usage gradually increases (initially ~30mb per instance, becomes 40mb after an hour, gets to 60 or 70mb by the time it crashes).

Answer 1

Sounds like you have a memory leak of some sort. If you'd like to bandaid the issue you can try setting the PassengerMaxRequests to something a bit lower until you figure out what's going on.

http://www.modrails.com/documentation/Users%20guide%20Apache.html#PassengerMaxRequests

This will restart your instances, individually, after they've served a set number of requests. You may have to fiddle with it to find the sweet spot where they are restarting automatically before they lock up.

Other tips are: -Go through your plugins/gems and make sure they are up to date -Check for heavy actions and requests where there is a lot of memory consumption (NewRelic is great for this) -You may want to consider switching to REE as it has better garbage collection

Finally, you may want to set a cron job that looks at your currently running passenger instances and kills them if they are over a certain memory threshold. Passenger will handle restarting them.

Answer 2

New Relic can show you combined memory usage. Engine Yard recommends tools like Rack::Bug, MemoryLogic or Oink. Here's a nice article on something similar that you might find useful.

Answer 3

If restarting the app cures the problem, looking at its resource usage would be a good place to start.