E_1996
E_1996

Reputation: 89

Parallel processing with Rcpp function - would I get less overhead using foreach or using RcppParallel?

I have an Rcpp function which I am calling from R. I have made this parallel by using the 'foreach' package.

However, based on some brief experiments, the parallel computation is only worth it when the Rcpp function takes at least 0.1 seconds to evaluate.

For my project my function usually takes longer than 0.1 seconds to evaluate, and therefore the parallelisation is "worth it" in most cases. However there are some datasets where the function will take less than 0.5 seconds to evaluate, and it doesn't benefit from parallelisation due to the overhead of the 'foreach' package.

However, it would still be useful if there was a way that it could benefit from being run in parallel even when it takes < 0.1 seconds to evaluate, since the function needs to be evaluated many times.

So - my question is, in general, will making an Rcpp function parallel via C++ (using e.g. RcppParallel package) instead of via R (through foreach) create less overhead? or should I expect the same amount of overhead?

Upvotes: 0

Views: 173

Answers (0)

Related Questions