CODEWITHSUNDEEP
ruby-on-railsrubyruby-on-rails-3passenger
robink
robink

Reputation: 13

Rails, passenger, communicating between threads

I want to write in simple logger for my that puts messages in memory and, in background, every X seconds write it to a database.

Here is the buffered logger code:

module BufferedLogger
  def buffer
    @buffer ||= []
  end

  def log( message )
    buffer << message
  end

  def write_buffer  
    while message = buffer.shift do 
      # save the message in nosql
    end
  end

  def repeat_every( interval )
    Thread.new do
      loop do
        start_time = Time.now
        yield
        elapsed = Time.now - start_time
        sleep([interval - elapsed, 0].max)
      end
    end
  end

  extend self

  thread = repeat_every(10) do
    write_buffer
  end 

end

In development, this works fine, buffer() access to the same @buffer var in both log and write_buffer method. But as soon as I go to production or staging env, i.e. as soon as I'm behind passenger, this @buffer don't seem to be shared anymore.

Any pointer?

Upvotes: 0

Views: 298

Answers (2)

the Tin Man
the Tin Man

Reputation: 160553

If you're trying to save database I/O it's usually not an issue, so you might be prematurely optimizing.

If I needed to buffer database writes, I'd probably turn off autocommit, then write the records as they occur, with a commit after n records, or after n seconds, whichever comes first.

Databases can handle a lot of traffic and have all sorts of internal buffering also, so your disk and system impact is minimized. If your database is on the same machine as your Rails host, then you should split them for performance reasons.

Otherwise, unless you have gathered metrics and can point to database I/O as a problem, I'd say write the code and don't worry about buffering until that problem surfaces.

Upvotes: 0

shigeya
shigeya

Reputation: 4912

Since passenger create separate process, and how these process will persists depends on passenger's algorithm, I guess it will not work well as you expect. (btw, I had bad experience in this regard using global variables/class variables.)

My suggestion to buffer the log is, use logger like fluentd as intermediate processor. Fluentd can monitor and gather the log. You can write a plugin to write the collected log to DB. I think this will suit your needs.

Upvotes: 1

Related Questions