CODEWITHSUNDEEP
rparallel-processing
hendra
hendra

Reputation: 710

R could not find function "%dopar%"

In RStudio in Windows 10 I wrote a function that performs calculations in parallel, like the following:

doSomething = function(a, b, c) {

    # Inner function that does the actual work when parallelised
    work = function (a, b, c) {
        # Do something
        e = func1(a, b)
        f = func2(c)
        result = e + f

        return(result)
    }

    # Carry out work in parallel
    cl = makeCluster(6)
    registerDoParallel(cl)
    output = foreach(i = 1:10, .packages=c("foo", "bar")) %dopar%
        work(a, b, c)
    stopCluster(cl)

    return(output)
}

This works fine and dandy if I load the function into memory from an R script; however, I want to include it in a package that I am writing. As such, in the package file, I am careful to identify the namespace of the external functions and refer to their packages in the DESCRIPTION file. For example:

doSomething = function(a, b, c) {

    # Inner function that does the actual work when parallelised
    work = function (a, b, c) {
        # Do something
        e = foo::func1(a, b)
        f = bar::func2(c)
        result = e + f

        return(result)
    }

    # Carry out work in parallel
    cl = parallel::makeCluster(6)
    doParallel::registerDoParallel(cl)
    output = foreach::foreach(i = 1:10, .packages=c("foo", "bar")) %dopar%
        work(a, b, c)
    parallel::stopCluster(cl)

    return(output)
}

and in the DESCRIPTION file:

...
Imports:
    foo,
    bar,
    doParallel,
    foreach,
    parallel

(Edit) the NAMESPACE file contains the following:

# Generated by roxygen2 (4.1.1): do not edit by hand

export(doSomething)

The problem is, when I build my package and run the function out of the package, I get the following error and execution stops:

Error in doSomething(a, b, c):
    could not find function "%dopar%"

Since %dopar% is an operator, not a function, I can't prepend it with foreach:: in my package function.

I am not sure what I need to do to get it to work properly. Furthermore 95% of similar problems I've read about are caused by an omission from .packages() causing the error, not the %dopar% operator itself not being recognised. That would not appear to be the cause here.

Please help!

Upvotes: 10

Views: 12549

Answers (2)

Taylor
Taylor

Reputation: 2085

I found myself in a similar situation, and I was having difficulty doing something like foreach::%dopar%. So, using Mr. Flick's comments I just updated my function documentation to include a line like @importFrom foreach %dopar% In other words, your function can look like this so you cna use %dopar% without :: operator:

#' Do a thing
#'
#' @param first
#' @param second
#' @returns stuff
#' @export
#' @importFrom foreach %dopar%
foo <- function(first, second){

  # I can just use %dopar% without :: operator
  cl <- parallel::makeCluster(parallel::detectCores() - 1)
  doParallel::registerDoParallel(cl)
  # foreach::`%dopar%`(
  foreach::foreach(i = 1:nrow(everyCombo), .combine = 'c') %dopar% {
    .
    .
    .
  }
  parallel::stopCluster(cl)

}

Then call devtools::document(), and it will take care of the rest.

Upvotes: 0

M_D
M_D

Reputation: 297

Quick fix for problem with foreach %dopar% is to reinstall these packages:

install.packages("doSNOW")

install.packages("doParallel") 

install.packages("doMPI")

Above packages are responsible for parallelism in R. Bug which existed in old versions of these packages is now removed. I should mention that it will most likely help even though you are not using these packages in your project/package directly.

Upvotes: 1

Related Questions