CODEWITHSUNDEEP
machine-learningoctavevectorizationgradient-descent
user3762027
user3762027

Reputation: 1

Vectorized gradient descent basics

I'm implementing simple gradient descent in octave but its not working. Here is the data I'm using:

X = [1 2 3
     1 4 5
     1 6 7]

y = [10
     11
     12]

theta = [0
         0
         0]

alpha = 0.001 and itr = 50

This is my gradient descent implementation:

function theta = Gradient(X,y,theta,alpha,itr)
m= length(y)

for i = 1:itr,
    th1 = theta(1) - alpha * (1/m) * sum((X * theta - y) .* X(:, 1));
    th2 = theta(2) - alpha * (1/m) * sum((X * theta - y) .* X(:, 2));
    th3 = theta(3) - alpha * (1/m) * sum((X * theta - y) .* X(:, 3));

    theta(1) = th1;
    theta(2) = th2;
    theta(3) = th3;
end

Questions are:

I've been struggling with these things for a long time now. Unable to resolve it myself.

Sorry for bad formatting.

Upvotes: 0

Views: 988

Answers (1)

zizouraj
zizouraj

Reputation: 493

Your Batch gradient descent implementation seems perfectly fine to me. Can you be more specific on what is the error you are facing. Having said that, for your question Is that the correct way to test the hypothesis? [h(x) = theta' * x]. Based on the dimensions of your test set here, you should test it as h(x) = X*theta. For your second question, the number of iterations depends on the data set provided. To decide on the optimized number of iterations, you need to plot your cost function with the number of iterations. And as iterations increase, values of cost function should decrease. By this you can decide upon how many iterations you need. You might also consider, increasing the value of alpha in steps of 0.001, 0.003, 0.01, 0.03, 0.1 ... to consider best possible alpha value. For your third question, I guess you are directly trying to model the data which you have in this question. This data is very small, it just contains 3 training examples. You might be trying to implement linear regression algorithm. For that, you need to that proper training set which contains sufficient data to train your model. Then you can test your model with your test data. Refer to Andrew Ng course of Machine Learning in www.coursera.org. You will find more information in that course.

Upvotes: 0

Related Questions