CODEWITHSUNDEEP
matlabgradient-descent
Jeff Rohlman
Jeff Rohlman

Reputation: 43

Gradient descent always going to infinity

I've tried everything and can't figure out why my gradient descent isn't working. I've looked at numerous examples and have changed the gradient descent code multiple times. When I run the program I get a response of NaN. I then printed every iteration and saw that before I got to NaN the value was going higher and higher (or lower and lower to negative infinity). I've tried different alpha values, starting betas values, and the number of iterations and every time it doesn't work. What's going on?

Here is my code:

A = load('A2-datasets/data-build-stories.mat');
X = [ones(60,1) A.data_build_stories(:,1)];
y = A.data_build_stories(:,2);
b = gradDes(X, y);

function beta = gradDes(X,y)
    alpha = 0.01;
    beta = [0;0];
    m = length(y);
    for i = 1:1000
        beta = beta - (alpha/m) * (X' * (X * beta - y));
    end
end

And here is data-build-stories.mat: 

770 54  
677 47  
428 28  
410 38  
371 29  
504 38  
1136 80  
695 52  
551 45  
550 40  
568 49  
504 33  
560 50  
512 40  
448 31  
538 40  
410 27  
409 31  
504 35  
777 57  
496 31  
386 26  
530 39  
360 25  
355 23  
1250 102  
802 72  
741 57  
739 54  
650 56  
592 45  
577 42  
500 36  
469 30  
320 22  
441 31  
845 52  
435 29  
435 34  
375 20  
364 33  
340 18  
375 23  
450 30  
529 38  
412 31  
722 62  
574 48  
498 29  
493 40  
379 30  
579 42  
458 36  
454 33  
952 72  
784 57  
476 34  
453 46  
440 30  
428 21

Upvotes: 3

Views: 1624

Answers (1)

Matteo Peluso
Matteo Peluso

Reputation: 452

you are iterating through the gradient descent with a too big alpha for your data.

try and change it:

    A = load('tmp.txt');
    X = [ones(60,1) A(:,1)];
    y = A(:,2);
    b = gradDes(X, y);

    function beta = gradDes(X,y)
        alpha = 0.00000001;
        beta = [0;0];
        m = length(y);
        for i = 1:1000
            beta = beta - (alpha/m) * (X' * (X * beta - y));
        end
    end

    b =[  0.0001 0.0719]

Upvotes: 3

Related Questions