CODEWITHSUNDEEP
numpydeep-learninggoogle-colaboratoryram
Yesh
Yesh

Reputation: 41

Session crash in Colab due to excess usage of RAM

I'm getting the following runtime error when I tried to create a huge 3D numpy array. Your session crashed after using all available RAM.

This is the code that's causing the error,

decoder_output_one_hot = np.zeros((30000, 23, 20000), dtype='float32').

Why is this occurring when I can access over 25GB of RAM?

Upvotes: 3

Views: 1179

Answers (1)

jakevdp
jakevdp

Reputation: 86310

Your are attempting to create an array with 30000 x 23 x 20000 = 13,800,000,000 entries. Each entry is a 32-bit floating point number, so the total number of bytes is 13,800,000,000 x (32 / 8) = 55,200,000,000: in other words, your array would occupy over 50GB in RAM, twice what you have available.

Upvotes: 3

Related Questions