CODEWITHSUNDEEP
mpimpich
gpuguy
gpuguy

Reputation: 4585

MPI: Why does my MPICH program fails for large no. of processes?

/* C Example */
#include <mpi.h>
#include <stdio.h>
#include <stddef.h>
#include <stdlib.h>

int main (int argc, char* argv[])
{
  int rank, size;
  int buffer_length = MPI_MAX_PROCESSOR_NAME;
  char hostname[buffer_length];

  MPI_Init (&argc, &argv);      /* starts MPI */
  MPI_Comm_rank (MPI_COMM_WORLD, &rank);        /* get current process id */
  MPI_Comm_size (MPI_COMM_WORLD, &size);        /* get number of processes */

  MPI_Get_processor_name(hostname, &buffer_length); /* get hostname */

  printf( "Hello world from process %d running on %s of %d\n", rank, hostname, size );
  MPI_Finalize();
  return 0;
}

The above program compiles and run successfully on ubuntu 12.04 for smaller no. of processes. But it fails when I try to execute with 1000s of processes. Why it is so? I am expecting that scheduler would keep the threads in queue and can dispatch one by one (I am running this code on a single core machine)

Why the following error is coming for large no. of processes and how to resolve this issue?

root@ubuntu:/home# mpiexec -n 1000 ./hello
[proxy:0:0@ubuntu] HYDU_create_process (./utils/launch/launch.c:26): pipe error (Too many open files)
[proxy:0:0@ubuntu] launch_procs (./pm/pmiserv/pmip_cb.c:751): create process returned error
[proxy:0:0@ubuntu] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:935): launch_procs returned error
[proxy:0:0@ubuntu] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[proxy:0:0@ubuntu] main (./pm/pmiserv/pmip.c:226): demux engine error waiting for event
Killed

Upvotes: 2

Views: 2198

Answers (1)

kraffenetti
kraffenetti

Reputation: 365

You are running into the open file limit on your system. Default in Ubuntu is 1024. You can try raising the limit in your session with the ulimit command.

ulimit -n 2048

Upvotes: 3

Related Questions