CODEWITHSUNDEEP
machine-learningneural-networkoctavegradient-descent
D buiss
D buiss

Reputation: 173

My gradient descent is not giving the exact value

I have written gradient descent algorithm in Octave but it is not giving me the exact answer. The answer differs from one to two digits.

Here is my code:

function theta = gradientDescent(X, y, theta, alpha, num_iters)

m = length(y); % number of training examples
s = 0;
temp = theta;
for iter = 1:num_iters
  for j = 1:size(theta, 1)
    for i = 1:m
      h = theta' * X(i, :)';
      s = s + (h - y(i))*X(i, j);
    end
    s = s/m;
    temp(j) = temp(j) - alpha * s;
  end 
  theta = temp; 
end

end

For:

theta = gradientDescent([1 5; 1 2; 1 4; 1 5],[1 6 4 2]',[0 0]',0.01,1000);

My gradient descent gives this:

 4.93708
-0.50549

But it is expected to give this:

 5.2148
-0.5733

Upvotes: 0

Views: 109

Answers (1)

Kaushal Pahwani
Kaushal Pahwani

Reputation: 474

Minor fixes :

  1. Your variable s probably the delta is initialised incorrectly.
  2. So it the temp variable probably the new theta
  3. Incorrectly calculating the delta

Try with below changes.

function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)

m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
temp = theta;
for iter = 1:num_iters
    temp = zeros(length(theta), 1);
    for j = 1:size(theta)
        s = 0
        for i = 1:m
            s = s + (X(i, :)*theta - y(i)) * X(i, j);
        end
    end
    s = s/m;
    temp(j) = temp(j) - alpha * s;
end 
    theta = temp; 
    J_history(iter) = computeCost(X, y, theta);
end
end

Upvotes: 1

Related Questions