CODEWITHSUNDEEP
rparallel-processingopenmpidoparalleldompi
Ben Carlson
Ben Carlson

Reputation: 1265

Write to log file using doMPI

I am running doMPI on an HPC and I would like to log output from workers. Using doParallel, I was able to use makeCluster(outfile='myfile.log'). With doMPI, there does not seem to be an outfile argument in any of the methods. I tried using sinkWorkerOutput(). This works, but only wrote the log for one of the workers. I suspect that each worker is overwriting the other. Is there an analog for outfile for doMPI?

A related question - inside of a worker, can I find the worker number?

EDIT: here is a link to an answer discussing how to use outfile: How can I print when using %dopar%

Thank you for your help,

Ben

Upvotes: 1

Views: 172

Answers (1)

Steve Weston
Steve Weston

Reputation: 19677

To send worker output to a file in the doMPI package, set the startMPIcluster "verbose" option to TRUE:

cl <- startMPIcluster(verbose=TRUE)

This creates one file per worker with names of the form "MPI_1_steve_41747.log". The MPI rank, user name, and process ID is used to make the file names unique. You can also specify the log directory via the "logdir" option.

To get a worker number, you can simply call the mpi.comm.rank function.

Upvotes: 0

Related Questions