Scatter a grid into equally-sized blocks

Question

Given a grid of 10x10 size like this:

        0,1,2,3,4,          5,6,7,8,9,
        10,11,12,13,14,     15,16,17,18,19,
        20,21,22,23,24,     25,26,27,28,29,
        30,31,32,33,34,     35,36,37,38,39,
        40,41,42,43,44,     45,46,47,48,49,

        50,51,52,53,54,     55,56,57,58,59,
        60,61,62,63,64,     65,66,67,68,69,
        70,71,72,73,74,     75,76,77,78,79,
        80,81,82,83,84,     85,86,87,88,89,
        90,91,92,93,94,     95,96,97,98,99

how to scatter the visually separated blocks to four processes?
I have tried the way described here (or even here) with Scatterv and it worked, but I remember having a project with the exact same problem and not using either resize or Scatterv to solve it.

Here's a minimum code sample with what I have:

#include <stdio.h>
#include <mpi.h>
#include <stdlib.h>
#include <assert.h>
#include <memory.h>
#include <unistd.h>

void print_array(int* arr, int width, int height)
{
    int i;
    for (i = 0; i< width * height;i++)
    {
        if((i != 0) && (i % width == 0))
            printf("\n");
        printf("%4d ", arr[i]);
    }
    putchar('\n');
}

int main() {
    int board[100] = {
            0,1,2,3,4,          5,6,7,8,9,
            10,11,12,13,14,     15,16,17,18,19,
            20,21,22,23,24,     25,26,27,28,29,
            30,31,32,33,34,     35,36,37,38,39,
            40,41,42,43,44,     45,46,47,48,49,

            50,51,52,53,54,     55,56,57,58,59,
            60,61,62,63,64,     65,66,67,68,69,
            70,71,72,73,74,     75,76,77,78,79,
            80,81,82,83,84,     85,86,87,88,89,
            90,91,92,93,94,     95,96,97,98,99
    };

    int numprocs, rank;
    MPI_Init(NULL, NULL);
    MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    int* block = calloc(25, sizeof(int));
    assert(block != NULL);


    MPI_Datatype sent_block_t, resized_sent_block_t;
    MPI_Type_vector(5, 5, 10, MPI_INT, &sent_block_t);
    MPI_Type_create_resized(sent_block_t, 0, 5*sizeof(int), &resized_sent_block_t);
    MPI_Type_commit(&sent_block_t);
    MPI_Type_commit(&resized_sent_block_t);

    if (rank == 0) {
        print_array(board, 10, 10);
        MPI_Scatter(&(board[0]), 1, resized_sent_block_t,
                    &(block[0]), 25, MPI_INT,
                    0, MPI_COMM_WORLD);
    }
    else {
        MPI_Scatter(NULL, 0, resized_sent_block_t,
                    &(block[0]), 25, MPI_INT,
                    0, MPI_COMM_WORLD);
    }

    for (int i = 0; i < numprocs; i++) {
        MPI_Barrier(MPI_COMM_WORLD);
        sleep(1);
        if (i == rank) {
            printf("\nRank: %d\n", rank);
            print_array(block, 5, 5);
        }
    }

    MPI_Finalize();
    free(block);
}

Running it with 4 processes I get this:

  0    1    2    3    4    5    6    7    8    9 
  10   11   12   13   14   15   16   17   18   19 
  20   21   22   23   24   25   26   27   28   29 
  30   31   32   33   34   35   36   37   38   39 
  40   41   42   43   44   45   46   47   48   49 
  50   51   52   53   54   55   56   57   58   59 
  60   61   62   63   64   65   66   67   68   69 
  70   71   72   73   74   75   76   77   78   79 
  80   81   82   83   84   85   86   87   88   89 
  90   91   92   93   94   95   96   97   98   99 

Rank: 0
   0    1    2    3    4 
  10   11   12   13   14 
  20   21   22   23   24 
  30   31   32   33   34 
  40   41   42   43   44 

Rank: 1
   5    6    7    8    9 
  15   16   17   18   19 
  25   26   27   28   29 
  35   36   37   38   39 
  45   46   47   48   49 

Rank: 2
  10   11   12   13   14 
  20   21   22   23   24 
  30   31   32   33   34 
  40   41   42   43   44 
  50   51   52   53   54 

Rank: 3
  15   16   17   18   19 
  25   26   27   28   29 
  35   36   37   38   39 
  45   46   47   48   49 
  55   56   57   58   59 

This scattering is wrong, notice the contents of rank 2 and 3.
The correct ones would be:

Rank 0:             Rank 1:
0,1,2,3,4,          5,6,7,8,9,
10,11,12,13,14,     15,16,17,18,19,
20,21,22,23,24,     25,26,27,28,29,
30,31,32,33,34,     35,36,37,38,39,
40,41,42,43,44,     45,46,47,48,49,

Rank 2:             Rank 3:
50,51,52,53,54,     55,56,57,58,59,
60,61,62,63,64,     65,66,67,68,69,
70,71,72,73,74,     75,76,77,78,79,
80,81,82,83,84,     85,86,87,88,89,
90,91,92,93,94,     95,96,97,98,99

Question
Is there any way to scatter equal-sized blocks of a grid without using Scatterv?

Answer 1

I don't think it is possible to do this with MPI_Scatter since the displacements of the datatype are not constant - within a row the displacement is 5 integers (or a count of 1 with your resized type), but skipping to the next row is a displacement of 50 integers (or count=10).

On 4 processes, using:

  int counts[4] = {1, 1, 1, 1};
  int disps[4] = {0, 1, 10, 11};
  ...
  if (rank == 0) print_array(board, 10, 10);

  MPI_Scatterv(&(board[0]), counts, disps, resized_sent_block_t,
              &(block[0]), 25, MPI_INT,
              0, MPI_COMM_WORLD);

seems to work OK. Note that you don't need separate calls to scatter - MPI guarantees that parameters that are only valid at the root are only ever referenced at the root.