CODEWITHSUNDEEP
pythonarraysnumpy
XBB
XBB

Reputation: 109

Memory Error Python Numpy Array

I'm running into a memory error when trying to evaluate a function at a number of points and store in an array. I've read a few other posts about similar issues and I don't think that I am exceeding the memory limits of my system:

              total        used        free      shared  buff/cache   available
Mem:      131898384     8964068   122074628       15128      859688   121905868
Swap:      14648316     1001760    13646556

and also

MemTotal:       131898384 kB
MemFree:        75056308 kB
MemAvailable:   74889296 kB

The error message is:

PDF= np.zeros([T,Y])
MemoryError

My code is:

Y=40000
T = 200000
tmin = 0
timestep = 10
tmax = timestep*T
TD_Psi = np.zeros([T,Y],'complex')
t = np.linspace(tmin, tmax, T)

for j in range(T):
    for i in range(M):
        TD_Psi[j] = TD_Psi[j] + c[i]*MO_basis[i]*np.exp(-evals[i]*t[j]*1j)

PDF= np.zeros([T,Y])
for time in range(T):
    PDF[time] = np.real(np.conjugate(TD_Psi[time])*TD_Psi[time])

I'm not sure if the problem is in my code or my system and what I can do to resolve this issue. It runs up until T = 150000 before giving me the error.

Thank you very much for your help.

Upvotes: 0

Views: 1523

Answers (2)

Asif Mehmood
Asif Mehmood

Reputation: 103

Since you're running out of memory and the data cannot be stored in RAM, for a quick solution try increasing your virtual memory.

  1. Open My Computer
  2. Right click and select Properties
  3. Go to Advanced System Settings
  4. Click on Advance Tab
  5. Click on settings under Performance
  6. Click on Change under Advance Tab
  7. Increase the memory size, that will increase virtual memory size.

Make sure to select as much as you need, it will convert your hard disk space to virtual memory, and your issue will be resolved now if your data fits into allocated memory.

Upvotes: 0

user2357112
user2357112

Reputation: 281968

np.zeros([T,Y],'complex') is 128 GB. np.zeros([T,Y]) is another 64 GB. You don't have 192 GB of RAM. You're out of memory.

Upvotes: 2

Related Questions