CODEWITHSUNDEEP
rubyasynchronouseventmachine
Matt Green
Matt Green

Reputation: 2092

Blocking on an EventMachine::Multi request

I'm working on a class responsible for aggregating data from several sites at once. It should issue requests to all sites in parallel, collect the results, and then return them to the caller. All of the calls need to have finished by the time the aggregate method returns.

Additionally, the method will be running in the context of a Rails controller, and it may or may not be running under an evented web server.

The pure EM-based version of the code follows:

class DataAggregator
  def aggregate(artist)
    multi = EventMachine::MultiRequest.new

    EventMachine.run do
      multi.add :a, EventMachine::HttpRequest.new("http://www.postrank.com").get
      multi.add :b, EventMachine::HttpRequest.new("http://www.google.com").get
      multi.add :c, EventMachine::HttpRequest.new("http://www.github.com").get
      multi.add :d, EventMachine::HttpRequest.new("http://www.yahoo.com").get
      multi.add :e, EventMachine::HttpRequest.new("http://www.facebook.com").get

      multi.callback do
        puts "Done!"
        EventMachine.stop unless defined?(Thin)
      end
    end

    # Not optimal, but I don't see any way to do this otherwise.
    unless multi.finished?
      sleep 0.1
    end
  end
end

I have two issues with the code as is:

  1. Is there any way to get it to behave under Thin? Because Thin already has an event loop, blocking on completion of the multi request will never finish as I've written it above. Yet, I still want to block on completion of the multi request before returning from aggregate. In other words, I want an EventMachine within an EventMachine.

  2. Polling is never the answer. Is there any other way to write that?

Upvotes: 1

Views: 426

Answers (1)

Matt Green
Matt Green

Reputation: 2092

I stopped being stubborn and tried out Typhoeus. It Just Works for this scenario. As another thread on here mentioned, blocking on an async call is the worst of both worlds.

Upvotes: 2

Related Questions