CODEWITHSUNDEEP
arrayspython-3.xpandasmemoryamazon-sagemaker
bls
bls

Reputation: 361

Sagemaker: MemoryError: Unable to allocate ___for an array with shape ___ and data type float64

I am running a notebook in sagemaker and it seems like one of the arrays produced after vectorizing text is causing issues.

Reading other answers it seems like it is an issue with overcommit. And one of the solutions proposed is to set it to always overcommit with this:

$ echo 1 > /proc/sys/vm/overcommit_memory

Is there any documentation or do you have any suggestion on how to do the same thing in sagemaker?

Thank you very much.

Upvotes: 9

Views: 3520

Answers (2)

Ugochi
Ugochi

Reputation: 9

Changing the default kernel in the SageMaker Notebook to conda_python3 resolved this issue for me.

Upvotes: -2

Kisa
Kisa

Reputation: 31

Open a root shell by sudo -i and then run the echo 1 > /proc/sys/vm/overcommit_memory

Upvotes: 3

Related Questions