CODEWITHSUNDEEP
rmpicluster-computingparallel.foreachsnow
uPhone
uPhone

Reputation: 313

How to specify both the number of cores and nodes for MPI running in R

I want to run R in parallel on HPC clusters using MPI. I understand the makeCluster function from package snow can be used to specify the number of nodes. For example: makeCluster(2,type="MPI") means 2 nodes. Without specifying the type to "MPI", makeCluster(2) would mean 2 cores on a single node.

Is there a way to specify both the number of nodes and cores?

How many cores on each node at default would be used if specifying the type to MPI?

Upvotes: 1

Views: 537

Answers (1)

Dirk is no longer here
Dirk is no longer here

Reputation: 368439

The snow and parallel packages do not to the best of my knowledge expose this -- as it gets hardware-dependent very quickly.

No, if your MPI implementation is, say, OpenMPI then you can specify this in your server's MPI configuration. There is a pretty rich grammar for this, and the hwloc library should give you hardware locality.

But R at the very end of this only knows 'number of worker nodes' and passes the how and which down to the particular implementation you pick.

Upvotes: 3

Related Questions