CODEWITHSUNDEEP
matlabmachine-learningneural-networkgradient-descent
blue-sky
blue-sky

Reputation: 53876

Gradient descent not updating theta values

Using the vectorized version of gradient as described at : gradient descent seems to fail

theta = theta - (alpha/m *  (X * theta-y)' * X)';

The theta values are not being updated, so whatever initial theta value this is the values that is set after running gradient descent :

example1 : 

m = 1
X = [1]
y = [0]
theta = 2
theta = theta - (alpha/m .* (X .* theta-y)' * X)'

theta =

    2.0000

example2 : 

m = 1
X = [1;1;1]
y = [1;0;1]
theta = [1;2;3]
theta = theta - (alpha/m .* (X .* theta-y)' * X)'

theta =

    1.0000
    2.0000
    3.0000

Is theta = theta - (alpha/m * (X * theta-y)' * X)'; a correct vectorised implementation of gradient descent ?

Upvotes: 0

Views: 329

Answers (1)

Nickil Maveli
Nickil Maveli

Reputation: 29721

theta = theta - (alpha/m * (X * theta-y)' * X)'; is indeed the correct vectorized implementation of gradient-descent.

You totally forgot to set the learning rate, alpha.

After setting alpha = 0.01, your code becomes:

m = 1                # number of training examples
X = [1;1;1]
y = [1;0;1]
theta = [1;2;3]
alpha = 0.01
theta = theta - (alpha/m .* (X .* theta-y)' * X)'
theta =

   0.96000
   1.96000
   2.96000

Upvotes: 0

Related Questions