CODEWITHSUNDEEP
c++c++11mpi
Teodor Nikolov
Teodor Nikolov

Reputation: 783

Copying MPI_Win?

I'm writing a MPI_Win wrapper in C++11 which looks as follows. 

#include <utility>
#include <mpi.h>

class WinWrapper {
    MPI_Win win;

public:
    template<typename Scalar>
    WinWrapper(Scalar *data, std::size_t dataSize, MPI_Comm comm) {
        MPI_Win_create(data, dataSize * sizeof(Scalar), sizeof(Scalar),
                       MPI_INFO_NULL, comm, &win);
    }

    WinWrapper(WinWrapper const &wrapper) {
        win = wrapper.win;
    }

    WinWrapper(WinWrapper &&wrapper) : win(wrapper.win) {
        wrapper.win = MPI_WIN_NULL;
    }

    WinWrapper &operator=(WinWrapper const &wrapper) {
        win = wrapper.win;
        return *this;
    }

    WinWrapper &operator=(WinWrapper &&wrapper) {
        if (&wrapper != this) {
            win = wrapper.win;
            wrapper.win = MPI_WIN_NULL;
        }
        return *this;
    }

    ~WinWrapper() {
        if (win != MPI_WIN_NULL) {
            MPI_Win_free(&win);
        }
    }

};  // end class WinWrapper

int main(int argc, char **argv) {
    MPI_Init(&argc, &argv);

    {
        std::size_t dataSize{100};
        int* data;
        MPI_Alloc_mem(sizeof(int) * dataSize, MPI_INFO_NULL, &data);
        WinWrapper w1{data, dataSize, MPI_COMM_WORLD};
        // WinWrapper w2 = w1;
        WinWrapper w3 = std::move(w1);
        WinWrapper w4{std::move(w3)};
        //WinWrapper w5{w4};
    }

    MPI_Finalize();
}

I pass a Scalar * which can be int *, double *, etc ... pointers allocated using MPI_Alloc_mem. When I uncomment the copy constructor or the copy assignment operator in main, I get a segmentation fault

[latitude:21878] *** Process received signal ***
[latitude:21878] Signal: Segmentation fault (11)
[latitude:21878] Signal code: Address not mapped (1)
[latitude:21878] Failing at address: 0x18
[latitude:21878] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)    [0x7fbc12c1c390]
[latitude:21878] [ 1] /home/teodor/soft/lib    /libmpi.so.20(ompi_win_free+0xe)[0x7fbc12e70f7e]
[latitude:21878] [ 2] /home/teodor/soft/lib/libmpi.so.20(MPI_Win_free+0x9b)[0x7fbc12ea3ecb]
[latitude:21878] [ 3] ./example/winwrapper(main+0xbd)[0x400c0d]
[latitude:21878] [ 4] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fbc11fb7830]
[latitude:21878] [ 5] ./example/winwrapper(_start+0x29)[0x400a79]
[latitude:21878] *** End of error message ***
[2]    21878 segmentation fault (core dumped)  ./example/winwrapper

Is there a similar routine to MPI_Comm_dup for MPI_Win? If not, how do I make the copy operations work?

Upvotes: 1

Views: 149

Answers (2)

Jeff Hammond
Jeff Hammond

Reputation: 5642

There is no good definition of copying a window. MPI_Comm_dup copies the state of the MPI_Comm object, which is all hidden. An MPI window consists of hidden state plus the memory attached to the window. Copying the state would copy the pointers but not the data to which they pointed. That isn’t very useful (there are reasons to do this but they are very nuanced).

You can write your own copy constructor easily enough, but first you need to decide what you mean by it. If you define your requirements, I’ll tell you how to implement them.

Upvotes: 0

Zulan
Zulan

Reputation: 22670

Making a copy of the MPI_Win handle and then freeing it once the first wrapper goes out of scope, is a bad idea. You won't be able to use any copies any longer and a destruction of a copy will likely crash (although I can't reproduce the segfault on my system).

I am not aware of a way to copy an existing MPI_Win. The most straight forward way is to actually delete the copy operations, and treat the wrapper type as move-only. When you use the type, you should have a clear idea about ownership - WinWrapper on the stack or in a std::unique_ptr<WinWrapper> and others hold references or raw pointers. You have to guarantee that the lifetime of the refs/raw pointers cannot exceed the pointed to wrapper. If you must copy around and have no idea bout lifetime, put it into a std::shared_ptr<WinWrapper>. Now you could also hide the std::shared_ptr in another wrapper, but that can lead to intransparent performance issues.

For proper RAII (resource acquisition is initialization), you should also take care of the memory within the wrapper. Otherwise you must manually take care Scalar* is not freed before the wrapper is destructed. I would recommend using MPI_Win_allocate.

Upvotes: 2

Related Questions