CODEWITHSUNDEEP
ruby-on-rails-3heroku
sauronnikko
sauronnikko

Reputation: 5435

Rails 3 + Heroku: Find out dyno's memory usage

Is there any way to find out how much memory a heroku web dyno is using? Let's say I wanna write a rake task that runs periodically and checks every dyno's memory usage. How could I do this? Thanks!

Upvotes: 13

Views: 4741

Answers (3)

Wolfram Arnold
Wolfram Arnold

Reputation: 7273

Since Heroku runs on an Amazon instance with Linux, you can use the proc filesystem to get runtime system data. The /proc/<pid>/smaps file contains memory info on the running process and all the libraries that it loads. For the resident process size, do this:

f = File.open("/proc/#{Process.pid}/smaps")
f.gets # Throw away first line
l = f.gets # Contains a string like "Size:               2148 kB\n"
l =~ /(\d+) kB/  # match the size in kB 
f.close
$1.to_i  # returns matched size as integer

Update: The proc file system has an even better source, /proc/<pid>/status. There's an entry VmRSS: for the total resident memory portion and VmHWM: for the highest resident memory used (high water mark). For more details of these and other fields, see The Linux Kernel docs for the proc file system.

Upvotes: 4

Wolfram Arnold
Wolfram Arnold

Reputation: 7273

There's also a native Heroku way to do this now. Heroku rolled out a labs feature log runtime metrics which will inject CPU load and memory usage information into the logging stream. These logs come with a source ID (e.g. "web.1") and a dyno unique ID, so that you can tell the dynos apart. It looks something like this:

source=web.1 dyno=heroku.2808254.d97d0ea7-cf3d-411b-b453-d2943a50b456 sample#load_avg_1m=2.46 sample#load_avg_5m=1.06 sample#load_avg_15m=0.99
source=web.1 dyno=heroku.2808254.d97d0ea7-cf3d-411b-b453-d2943a50b456 sample#memory_total=21.00MB sample#memory_rss=21.22MB sample#memory_cache=0.00MB sample#memory_swap=0.00MB sample#memory_pgpgin=348836pages sample#memory_pgpgout=343403pages

To turn this on, simply run: 

heroku labs:enable log-runtime-metrics
heroku restart

Upvotes: 8

Sim
Sim

Reputation: 13528

I took the suggestion from the accepted answer and implemented a /proc filesystem parser with aggregation & threshold-based diffing. I found it quite useful in debugging some Ruby 2.0 memory problems on Heroku. Get the code, also included here for convenience.

# Memory snapshot analyzer which parses the /proc file system on *nix
#
# Example (run in Heroku console):
#
#    ms = MemorySnapshot.new
#    1.upto(10000).map { |i| Array.new(i) }; nil
#    ms.snapshot!; nil
#    ms.diff 10
#    => {"lib/ld-2.11.1.so"=>156, "heap"=>2068, "all"=>2224}
#
class MemorySnapshot

  attr_reader :previous
  attr_reader :current

  def initialize
    snapshot!
    @previous = @current
  end

  # Generates a Hash of memory elements mapped to sizes of the elements in Kb
  def snapshot!
    @previous = @current
    @current = reduce(names_with_sizes)
  end

  # Calculates the difference between the previous and the current snapshot
  # Threshold is a minimum delta in kilobytes required to include an entry
  def diff(threshold = 0)
    self.class.diff_between previous, current, threshold
  end

  # Calculates the difference between two memory snapshots
  # Threshold is a minimum delta in kilobytes required to include an entry
  def self.diff_between(before, after, threshold)
    names = (before.keys + after.keys).uniq
    names.reduce({}) do |memo, name|
      delta = after.fetch(name) { 0 } - before.fetch(name) { 0 }
      memo[name] = delta if delta.abs >= threshold
      memo
    end
  end

  private

  def reduce(matches)
    total = 0
    current_name = nil
    matches.reduce(Hash.new { 0 }) do |memo, match|
      current_name = match[:name] || current_name
      size = match[:size].to_i
      total += size
      memo[current_name] += size
      memo
    end.tap { |snapshot| snapshot['all'] = total }
  end

  def names_with_sizes
    smap_entries.map do |line|
      /((^(\/|\[)(?<name>[^ \]]+)\]?\s+)|(^))(?<size>\d+)\s/.match(line)
    end
  end

  def smap_entries
    smaps.
        gsub(/^(([^Sa-f0-9])|(S[^i]))[^\n]+\n/m, '').
        gsub(/\nSize:/m, '').
        gsub(/[0-9a-f]+-[0-9a-f]+.{6}[0-9a-f]+ [0-9a-f]+:[0-9a-f]+ [0-9a-f]+\s+/i, '').
        split("\n")
  end

  def smaps
    File.read("/proc/#{Process.pid}/smaps")
  end
end

Upvotes: 6

Related Questions