CODEWITHSUNDEEP
matlab
Bazman
Bazman

Reputation: 2150

Increase Matlab memory

Why does the code below cause Matlab to keep increasing the amount of memory used? Nothing is being stored from one iteration to the next? Yet Matlab keeps using up my system memory until the whole machine grinds to a halt. I am running on windows 7 professional with about 16GB of physical ram. y is just a 100*1 vector and x a 100*7 matrix of data. The system start to struggle as Matlab gets over 15GB which in understandable what is not so understandable is why Matlab needs so much memory for the program below.

clc;
iter=100000000;
b_OIRE=[1,0,1,1,1,1,1];
nsims=2;

for t=1:nsims
    y=y_store(:,t);
    [b_GIREII]=GIREII(y,x,b_OIRE,iter);
end


function [b_GIREII MSE]=GIREII(y,x,b_OIRE,iter)  % [## "iter" denotes the iteration number]
   [n, p]=size(x);
   dim=1;
   b=x\y; 
   b_GIREII=b;

   sigma_sq=((y-x*b)'*(y-x*b))/(n-p);

   econFlag=0;
   [U,sigma,V] = svd(x,econFlag);

   U1=U(:,1:p);
   d=zeros(p,1);
   k=zeros(p,1);

   alpha=V'*b_GIREII;
   Delta=sigma.^1;         % [Error! not sigma.^2 but sigma.^1]
   Delta=diag(Delta);
   f=Delta.*alpha;
   F=diag(f);
   Theta=sum(f);
   c=p^2*sigma_sq+p*Theta^2;
   g=Theta*sum(alpha);
   one=ones(p,1);

   b=F*alpha;
   I=eye(p);
   A=sigma_sq*I+F.^2;
   G=sigma_sq*I+Theta*F;
   H=sigma_sq*I+f*f';

   q=(p-1)/p;

   k1=0;
   k2=0;
   d1=0;
   d2=0;

   for ii=1:p
       k1=k1+alpha(ii)/(q*sigma_sq+Delta(ii)^2*alpha(ii)^2);
       k2=k2+1/(q*sigma_sq+Delta(ii)^2*alpha(ii)^2);
       d1=d1+alpha(ii);
       d2=d2+Delta(ii)^2*alpha(ii)^2;
   end

   for ii=1:p
       k(ii)=(Delta(ii)*alpha(ii)^2)/((q*sigma_sq+Delta(ii)^2*alpha(ii)^2))-(k1/k2)*(Delta(ii)*alpha(ii)/(q*sigma_sq+Delta(ii)^2*alpha(ii)^2));
       d(ii)=(1/p)*(d1/(sigma_sq+d2)*Delta(ii)*alpha(ii)-k(ii));   
   end

   K=diag(k);
   D=one*d';
   b_GIREII= V*(K+D)*U1'*y;
   MSE=(k'*A*k)+(2*k'*H*d)-(2*b'*k)+(p*d'*H*d)-((2*g/Theta)*f'*d)+alpha'*alpha;

   best_GIREII_MSE=MSE;
   best_b_GIREII=b_GIREII;
   best_index=1;

   for jj=1:iter           % [## "iter" denotes the iteration number]
       alpha=V'*b_GIREII;
       Alpha_store(:,jj)=alpha;
       f=Delta.*alpha;
       f_store(:,jj)=f;
       F=diag(f);
       Theta=sum(f);
       c=p^2*sigma_sq+p*Theta^2;
       g=Theta*sum(alpha);
       b=F*alpha;
       A=sigma_sq*I+F.^2;
       A_store(:,:,jj)=A;
       G=sigma_sq*I+Theta*F;
       H=sigma_sq*I+f*f';

       k1=0;
       k2=0;
       d1=0;
       d2=0;

       for ii=1:p
            k1=k1+alpha(ii)/(q*sigma_sq+Delta(ii)^2*alpha(ii)^2);
            k2=k2+1/(q*sigma_sq+Delta(ii)^2*alpha(ii)^2);
            d1=d1+alpha(ii);
            d2=d2+Delta(ii)^2*alpha(ii)^2;
       end

       for ii=1:p
            k(ii)=(Delta(ii)*alpha(ii)^2)/((q*sigma_sq+Delta(ii)^2*alpha(ii)^2))-(k1/k2)*(Delta(ii)*alpha(ii)/(q*sigma_sq+Delta(ii)^2*alpha(ii)^2));
            d(ii)=(1/p)*(d1/(sigma_sq+d2)*Delta(ii)*alpha(ii)-k(ii));   
       end

       D=one*d';
       K=diag(k);
       b_GIREII= V*(K+D)*U1'*y;
       MSE=(k'*A*k)+(2*k'*H*d)-(2*b'*k)+(p*d'*H*d)-((2*g/Theta)*f'*d)+alpha'*alpha;

       if(MSE<best_GIREII_MSE)
           best_b_GIREII=b_GIREII;
           best_GIREII_MSE=MSE;
           best_index=jj+1;
       end

       if rem(jj,10000000)==0
           disp('10000000 interations complete')
       end
   end
end

Upvotes: 1

Views: 347

Answers (2)

Dennis Jaheruddin
Dennis Jaheruddin

Reputation: 21563

It seems like @Rody already found the problem with your code, but in fact it is not hard to track this kind of problem down in general.

Here are the three steps to get there:

  1. Turn on dbstop if error
  2. Wait till the code breaks down, or hit Ctrl+C once you see the memory usage growing beyond what you expected, to investigate in the even more general case you could also set a breakpoint.
  3. Use whos and see whether you either have a huge number of variables, or some huge variables in your workspace.

Upvotes: 3

Autonomous
Autonomous

Reputation: 9075

In the function GIREII.m, you are putting following three variables in a loop which runs for 100000000 times.

1. Alpha_store(:,jj)=alpha; %%jj goes from 1:10^8
2. f_store(:,jj)=f;
3. A_store(:,:,jj)=A;

Interestingly, you are not using either of them. I guess you can just remove them. Please pay attention to the warnings in the MATLAB editor, which are underlined in orange color.

Upvotes: 3

Related Questions