CODEWITHSUNDEEP
rparallel-processingtidyversepurrr
mathlover1235
mathlover1235

Reputation: 13

Parallel version of `map()`

I have a function that takes multiple arguments.

my_function <- function(list_of_vectors, list_of_scalars){
 ....
return(list)
}

I want to use map() over my function but using a parallel call that would use multicores. my_function() is very computationally expensive, and I need to call it to create output for over 1000 points. ( list_of_vectors is a list of 1000 vectors and list_of_scalars is a list of 1000 scalars)

Is there a mcmap() equivalent or any other formulation ? I had a look at other threads but none solved my issue.

Upvotes: 1

Views: 1126

Answers (2)

Matheus Batistela
Matheus Batistela

Reputation: 54

You can use map() function from Parallel library, like this:

const p = new Parallel([0, 1, 2, 3, 4, 5, 6]);
const log = function () { console.log(arguments); };

// One gotcha: anonymous functions cannot be serialzed
// If you want to do recursion, make sure the function
// is named appropriately
function fib(n) {
  return n < 2 ? 1 : fib(n - 1) + fib(n - 2);
};
    
p.map(fib).then(log)

// Logs the first 7 Fibonnaci numbers, woot!

Upvotes: -1

Ben Bolker
Ben Bolker

Reputation: 226192

You can use future_map() from the furrr package as a drop-in replacement.

This is a very flexible function; how it distributes your computation will depend on a previous call to future::plan() (furrr is built on top of the future package), e.g.

future::plan(multicore, workers = 4)
future_map(...)

to run your job on 4 cores on the same machine.

Upvotes: 4

Related Questions