CODEWITHSUNDEEP
cmpilarge-datahpc
user8883441
user8883441

Reputation: 3

mpi in c how to extends the use of MPI_Type_create_subarray?

How can I use the MPI_Type_create_subarray to pass a arrays of characters but of very large size of type unsigned long long int as the overall rows (total_nrows) = 31,613,582,882 and the number of rows to be written by each rank (real_rows3) around 2,580,689,006, in case using 14 ranks. 

unsigned long long int globalsizes[2] = {total_nrows, 365};
unsigned long long int localsizes [2] = {real_rows3, 365};
unsigned long long int starts[2]      = {startrow, 0};
MPI_Datatype localarray;

 MPI_Type_create_subarray(2, globalsizes, localsizes, starts, MPI_ORDER_C, MPI_CHAR, &localarray);

Upvotes: 0

Views: 125

Answers (1)

Gilles Gouaillardet
Gilles Gouaillardet

Reputation: 8380

Long story short, you cannot.

From the MPI standard :

int MPI_Type_create_subarray(int ndims,
                           const int array_of_sizes[],
                           const int array_of_subsizes[],
                           const int array_of_starts[],
                           int order,
                           MPI_Datatype oldtype,
                           MPI_Datatype *newtype)

Arguments are int so there is an intrinsic limit of 2^31-1.

That being said, you might be able to achieve a similar result by using an intermediate (and larger) datatype. For example, an array of 2^32 MPI_CHAR can be seen an array of 2^22 vectors of 2^10 MPI_CHAR. That obviously does not work with any number, but that might help you.

An other option is to use BigMPI.

Basically, int (32 bits) are replaced with MPI_Count (64 bits). Keep in mind BigMPI is not part of the MPI 3.1 standard, so this is not an option if you have to write portable code.

[edit]

There is a chance BigMPI will land into the MPI standard, and possibly with MPI 4.0.

Upvotes: 2

Related Questions