CODEWITHSUNDEEP
pythonnumpy
David Brochart
David Brochart

Reputation: 885

Numpy load() memory overflow results in Segmentation fault

With the following code that leads to a memory overflow:

import numpy as np

a = np.arange(10000000)
np.save('a', a)

l = []
while True:
    l.append(np.load('a.npy'))

Python errors out a Segmentation fault, instead of a MemoryError. It seems to be the case for machines like AWS EC2 which don't have swap memory (I've tried with a machine with more RAM and a swap memory and I get a MemoryError).

Upvotes: 1

Views: 897

Answers (1)

Ani Menon
Ani Menon

Reputation: 28219

I am giving you multiple resources to go through below, but generally this is very much OS/system dependent. SegFlts

Simple answer: You are probably getting it because you are trying to access memory address which is not present in the memory. Its also possible to get segFaults when your memory module has issues or because of any of the following(these should clear your doubts):

Look at : Segfault instead of MemoryError when bytearray too big

Also, why segmentation fault? Possible answer

Tracing a SegFault(If you still feel something is fishy!) : python tracing a segmentation fault

Refer: Similar question

The question asked is actually broad, as the answer to it may take pages of explanation(hence added relevant links).

Upvotes: 1

Related Questions