CODEWITHSUNDEEP
ruby-on-railsheroku
acadavid
acadavid

Reputation: 506

How to do parallel HTTP requests in Heroku?

I'm building a Ruby on Rails app that access about 6-7 APIs, grabs information from them based on user's input, compares and display results to the users (the information is not saved in the database). I will be using Heroku to deploy the app. I would like those HTTP requests to access the APIs to be done in parallel so the answer time is better instead of doing it sequential. What do you think is the best way to achieve this in Heroku?

Thank you very much for any suggestions!

Upvotes: 8

Views: 2380

Answers (4)

maxpenguin
maxpenguin

Reputation: 5149

If you want to actually do the requests on the server side (tfe's javascript solution is a good idea), your best bet would be using EventMachine. Using EventMachine gives a simple way to do non-blocking IO.

Also check out EM-Synchrony for a set of Ruby 1.9 fiber aware clients (including HTTP).

All you need to do for a non-blocking HTTP request is something like:

require "em-synchrony"
require "em-synchrony/em-http"
EM.synchrony do
    concurrency = 2
    urls = ['http://url.1.com', 'http://url2.com']

    # iterator will execute async blocks until completion, .each, .inject also work!
    results = EM::Synchrony::Iterator.new(urls, concurrency).map do |url, iter|

        # fire async requests, on completion advance the iterator
        http = EventMachine::HttpRequest.new(url).aget
        http.callback { iter.return(http) }
        http.errback { iter.return(http) }
    end

    p results # all completed requests
    EventMachine.stop
end

Goodluck!

Upvotes: 6

oma
oma

Reputation: 40760

I haven't tried parallelizing requests like that. But I've tried parallel on heroku, works like a charm! This is my simple blog post about it.

http://olemortenamundsen.wordpress.com/2010/10/17/spawning-multiple-threads-at-heroku-using-parallel/

Upvotes: 1

tfe
tfe

Reputation: 34932

You could always make the requests client-side using Javascript. Then not only can you run them in parallel, but you won't even need the round-trip to your own server.

Upvotes: 1

gjb
gjb

Reputation: 6317

Have a look at creating each request as a background job: http://blog.heroku.com/archives/2009/7/15/background_jobs_with_dj_on_heroku/

The more 'Workers' you buy from Heroku, the more background jobs can be processed concurrently, leaving your 'Dynos' to serve your users.

Upvotes: 0

Related Questions