CODEWITHSUNDEEP
rforeachdoparallel
Brigadeiro
Brigadeiro

Reputation: 2945

Is it possible to create a text log within a foreach() %dopar% call in R?

I want to log certain events to a .txt file in my R script. Part of the script is parallelized. Is it possible to log events with cat() or similar inside a parallelized function?

This does not work.


chunks <- list(1:3, 4:6)

foreach(i = 1:cores) %:%
  foreach(x = chunks[[i]]) %dopar% {
    cat("Working on chunk, ", i, "number ", x, "...\n\n")
  }

This works, but it is system specific (not ideal)

foreach(i = 1:cores, .packages = "glue") %:%
  foreach(x = chunks[[i]]) %dopar% {
    system(glue("echo 'Working on chunk {i}, number {x}' >> output.txt"))
  }

Example output of the second (working) code block:

Working on chunk 1, number 1
Working on chunk 1, number 2
Working on chunk 1, number 3
Working on chunk 2, number 4
Working on chunk 2, number 6
Working on chunk 2, number 5

Upvotes: 1

Views: 815

Answers (1)

Akshit
Akshit

Reputation: 434

You can use doSNOW package in R. You can specify the file where you want the output logs too. Hope this works for you.

library(doSNOW)

cl <- makeCluster(2, outfile = "abc.out")
registerDoSNOW(cl)

chunks <- list(1:3, 4:6)

foreach(i = 1:2) %:%
  foreach(x = chunks[[i]]) %dopar% {
    cat("Working on chunk, ", i, "number ", x, "...\n\n")
    print("ongoing")
    # your statements
  }

stopCluster(cl)

Upvotes: 2

Related Questions