CODEWITHSUNDEEP
pythonshared-memory
danben
danben

Reputation: 83250

Segfault accessing shared memory in Python

I've been getting a segfault when trying to access shared memory, which I've boiled down to the following example. f1 and f2 are identical save for the fact that I'm storing the shared memory object in an extra variable in f2, but f2 works and f1 segfaults. Why would this happen?

(Calling the two functions in the opposite order does not change the behavior.)

from multiprocessing import shared_memory
import numpy as np

segment_name = "segment"
data_shape = (5,5,5)


def get_view(shm):
    shape = (1,) + data_shape
    arr_one = np.ndarray(shape, dtype=np.float64, buffer=shm.buf)
    arr_two = np.ndarray(data_shape, dtype=np.float64, buffer=arr_one[0])
    return arr_two


def f1():
    arr = get_view(shared_memory.SharedMemory(name=segment_name))
    print(arr[0][0][0])


def f2():
    shm = shared_memory.SharedMemory(name=segment_name)
    arr = get_view(shm)
    print(arr[0][0][0])


if __name__ == '__main__':
    shape = (1,) + data_shape
    dummy = np.ndarray(shape, dtype=np.float64)
    shared_memory.SharedMemory(name=segment_name, create=True, size=dummy.nbytes)
    f2()
    f1()

Upvotes: 3

Views: 226

Answers (1)

danben
danben

Reputation: 83250

Turns out that the extra reference was necessary. Without arr pointing to the shared memory object, its del method would be invoked immediately and unlink the underlying buffer whose reference was held by the numpy array.

Upvotes: 1

Related Questions