CODEWITHSUNDEEP
rubymemory-leakspassengerruby-on-rails-3.2
Christian Szardenings
Christian Szardenings

Reputation: 301

Huge memory leak / bloat after Upgrading from Rails 3.0 to Rails 3.2 (Ruby 1.9.3 + Passenger)

Last week we upgraded a small project which was running on Rails 3.0.1 to Rails 3.2.2.

Shortly after the upgrade we recognized that occasionally, 2-3 times a day, we're seening a huge Phusion Passenger process (1-5 GB).

We're running Phusion Passenger 3.0.11 and Ruby 1.9.3-p0. We already tried different Ruby version (1.9.2-p290 and 1.9.3-p125) without a success.

Afterwards, we tried to track our memory usage with Oink. Unfortunately Oink doesn't show the reason for the memory bloat - The large processes seem to not write to the logfiles anymore.

When we downgraded back to Rails 3.0.1 the problem is gone. Does anyone have similar problems? We really checked our code for possible memory leaks, such as too many ActiveRecord instances, but didn't find any.

Do you think it's worth it to try Unicorn instead of Passenger? Or is it likely that we'll run into the same problem?

Any suggestions about how to trace the memory leak are welcome. We already set up newrelic, but it doesn't show detailed information about the memory leak.

Upvotes: 8

Views: 1433

Answers (2)

lzap
lzap

Reputation: 17174

If this is Red Hat compatible Linux, you can use SystemTap. I am not sure if SystemTap is available on Debian/Ubuntu systems, if not there is alternative called DTrace. Here are some articles - I have been pretty successful with tracking down several regressions, although none of these were memory issues (maybe you can find a STP script that could do the trick for you). Read here:

http://lukas.zapletalovi.com/2012/02/peek-into-your-ruby-app-with-systemtap.html http://lukas.zapletalovi.com/2012/01/probing-ruby-apps-with-systemtap-in.html http://sourceware.org/systemtap/wiki/RubyMarker

The last link shows probes you can hoop into in SystemTaps in Ruby. There are thinks like gc runs or memory allocations, could help you. Good luck!

Upvotes: 1

Michael
Michael

Reputation: 10464

I would be interested to know what passenger-memory-stats shows and what type of Memory you have passenger set for PassengerMaxPoolSize PassengerPoolIdleTime and any other passenger settings.

How did you upgrade passenger?

What is your apache setup like? prefork or worker?

I suspect that you are seeing poor gc performance, attempt to tune them by putting this in a wrapper around the ruby that passenger uses:

#!/bin/sh
export RUBY_HEAP_MIN_SLOTS=600000
export RUBY_GC_MALLOC_LIMIT=59000000
export RUBY_FREE_MIN=200000
exec "/usr/bin/ruby" "$@"

It should fix some performance issues you are seeing in general due to poor ruby gc defaults.

Upvotes: 0

Related Questions