CODEWITHSUNDEEP
rubyprocessparallel-processing
Semyon Pupkov
Semyon Pupkov

Reputation: 83

Ruby parallel process in map

Help me plz

How i can implement method pmap for Array like map but in two process

I have code

class Array

  def pmap
    out = []
    each do |e|
      out << yield(e)
    end
    out
  end

end

require 'benchmark'

seconds = Benchmark.realtime do
  [1, 2, 3].pmap do |x|
    sleep x
    puts x**x
  end
end

puts "work #{seconds} seconds"

In result i must get 3 second for benchmark

Upvotes: 2

Views: 4006

Answers (2)

dbenhur
dbenhur

Reputation: 20408

Try the parallel gem.

require 'parallel'

class Array
  def pmap(&blk)
    Parallel.map(self, {:in_processes: 3}, &blk)
  end
end

Upvotes: 3

gmalette
gmalette

Reputation: 2459

To get absolutely 2 forks

You don't absolutely need RPC. Marshal + Pipe should usually work.

class Array

  def pmap
    first, last = self[0..(self.length/2)], self[(self.length/2+1)..-1]

    pipes = [first, last].map do |array|
      read, write = IO.pipe
      fork do
        read.close
        message = []
        array.each do |item|
          message << yield(item)
        end
        write.write(Marshal.dump message)
        write.close
      end
      write.close
      read
    end

    Process.waitall

    first_out, last_out = pipes.map do |read|
      Marshal.load(read.read)
    end

    first_out + last_out
  end

end

Edit

Now using fork

Upvotes: 3

Related Questions