CODEWITHSUNDEEP
mpi
user1472972
user1472972

Reputation: 105

MPI_Exscan- How to use for parallel File write

I am working on some MPI code for Parallel IO. Say I have an array [2 ,5,6,9,0,4,3,1,8,7 ] written in a File, and I have 2 processes.

I define File views to read part of this file. Process 0 sees : 2 ,5,6,9,0 (5 elements) Process 1 sees : 4,3,1,8,7 (5 elements) Both processes call a sort function . As a result of sort function, Process 0 has : 0,1,2,3 (4 elements) Process 1 has : 4,5,6,7,8,9 (6 elements)

Now i need to write this output to a new output file. Process 0 would write it correctly as it writes at offset 0. But how would Process 1 know the offset where to write the file ? I know I need to define a File view to write , BUT what would be the new displacement. I kind of think MPI_Exscan can do it.. But I am not sure how... Can someone help please ??

Thanks in Advance

Upvotes: 1

Views: 552

Answers (2)

Jonathan Dursi
Jonathan Dursi

Reputation: 50927

Sure; you use MPI_Exscan with MPI_SUM on the local number of elements to get the total number to the "left" of you, and use that in your view (either as an offset, or in the type you create to define your view).

Here's a little Fortran program where each rank generates a "random" (well, 2*rank+1) sized array of characters ('0' for rank 0, etc), uses MPI_Exscan to find out what offset it should use when writing, and then writes:

program testexscan
    use mpi
    implicit none

    integer :: nelements, nleft, total
    character, allocatable, dimension(:) :: array

    integer :: rank, nprocs, ierr, fh
    integer(kind=MPI_OFFSET_KIND) :: disp

    call MPI_Init(ierr)

    call MPI_Comm_rank(MPI_COMM_WORLD, rank,  ierr)
    call MPI_Comm_rank(MPI_COMM_WORLD, nprocs, ierr)

    call generatedata(rank, array)

    nelements = size(array)
    call  MPI_Exscan   (nelements, nleft,  1, MPI_INTEGER, MPI_SUM, MPI_COMM_WORLD, ierr)

    print *, 'rank = ', rank, ' nelements = ', nelements, ' nleft= ', nleft

    call MPI_File_open(MPI_COMM_WORLD, 'output.txt', ior(MPI_MODE_WRONLY,MPI_MODE_CREATE),  &
                       MPI_INFO_NULL, fh, ierr)

    disp = nleft * 1    ! offset in bytes
    call MPI_File_set_view(fh, disp, MPI_CHARACTER, MPI_CHARACTER, "native", MPI_INFO_NULL, ierr )
    call MPI_File_write_all(fh, array, nelements, MPI_CHARACTER, MPI_STATUS_IGNORE, ierr)
    call MPI_File_close(fh, ierr)

    deallocate(array)

    call MPI_Finalize(ierr)

    contains

        subroutine generatedata(rank, arr)
        character, dimension(:), allocatable, intent(inout) :: arr
        integer, intent(in) :: rank

        nelements = rank * 2 + 1

        allocate(array(nelements))
        array = char(ichar("0")+rank)

        end subroutine generatedata

end program testexscan

Running this gives:

$ mpif90 -o testexscan testexscan.f90 
$ mpirun -np 4 ./testexscan
 rank =            0  nelements =            1  nleft=            0
 rank =            1  nelements =            3  nleft=            1
 rank =            2  nelements =            5  nleft=            4
 rank =            3  nelements =            7  nleft=            9
$ cat output.txt 
0111222223333333

Upvotes: 4

suszterpatt
suszterpatt

Reputation: 8273

Your description is a bit vague, but in any case, process 1 can just send a message to process 2 that contains the offset to be used.

Upvotes: 0

Related Questions