CODEWITHSUNDEEP
rfunctionparallel-processing
R18
R18

Reputation: 1560

Parallelized code dependent from other functions

Here is the structure of a function I have created inside which a parallelization has been implemented.

    parallelized.function <- function(...){

          # Function used in the parallelization
            used.in.par <- function(...)
          # Function needed by used.in.par (auxiliars)
            aux1<-function(...)
            aux2<-function(...)
          #---------------------------------------------------#
          # PARALLELIZATION PROCESS
            suppressMessages(library(foreach))
            suppressMessages(library(doParallel))

                   ..................................

               %dopar%{ used.in.par(...) }

          #---------------------------------------------------#
         return(something)   
         }

The code works, but it supposes to define aux1 and aux2 inside parallelized.function to work well (which needs a lot of lines of code).

Is there any way to call aux1 and aux2 functions instead of writing all the code inside parallelized.function?

I tried creating new scripts with aux functions and writing source(".../aux1.R") and source(".../aux2.R") inside parallelized.function without success.

Thank you,

Upvotes: 0

Views: 44

Answers (1)

G. Becker
G. Becker

Reputation: 46

foreach package would do that for you. In order to acccess function that was not defined in the current environment, you should use .export; and loading required packages on each of the workers would be possible by using .packages option.

foreach(
  ...,
  .export = c('aux1', 'aux2'),
  .packages = c(...)
) %dopar% {
  ...
}

Note that loading foreach and doParallel package inside foreach loop is not required. But you missed the part when you register clusters.

Upvotes: 3

Related Questions