CODEWITHSUNDEEP
rubymultithreading
Andrew Bullock
Andrew Bullock

Reputation: 37378

Parallelism in Ruby

I've got a loop in my Ruby build script that iterates over each project and calls msbuild and does various other bits like minify CSS/JS.

Each loop iteration is independent of the others so I'd like to parallelise it.

How do I do this?

I've tried:

myarray.each{|item|
    Thread.start {

        # do stuff

     }
}
puts "foo"

but Ruby just seems to exit straight away (prints "foo"). That is, it runs over the loop, starts a load of threads, but because there's nothing after the each, Ruby exits killing the other threads :(

I know I can do thread.join, but if I do this inside the loop then it's no longer parallel.

What am I missing?

I'm aware of http://peach.rubyforge.org/ but using that I get all kinds of weird behaviour that look like variable scoping issues that I don't know how to solve.

Edit

It would be useful if I could wait for all child-threads to execute before putting "foo", or at least the main ruby thread exiting. Is this possible?

Upvotes: 2

Views: 787

Answers (3)

fl00r
fl00r

Reputation: 83680

Use em-synchrony here :). Fibers are cute.

require "em-synchrony"
require "em-synchrony/fiber_iterator"

# if you realy need to get a Fiber per each item
# in real life you could set concurrency to, for example, 10 and it could even improve performance
# it depends on amount of IO in your job
concurrency = myarray.size
EM.synchrony do
  EM::Synchrony::FiberIterator.new(myarray, concurrency).each do |url|
    # do some job here
  end
  EM.stop
end

Upvotes: 2

Russell
Russell

Reputation: 12439

Store all your threads in an array and loop through the array calling join:

threads = myarray.map do |item|
  Thread.start do
    # do stuff
  end
end
threads.each { |thread| thread.join }
puts "foo"

Upvotes: 6

Fernando Diaz Garrido
Fernando Diaz Garrido

Reputation: 3995

Take into account that ruby threads are green threads, so you dont have natively true parallelism. I f this is what you want I would recommend you to take a look to JRuby and Rubinius:

http://www.engineyard.com/blog/2011/concurrency-in-jruby/

Upvotes: 1

Related Questions