CODEWITHSUNDEEP
pythonmachine-learninglinear-regressiongradient-descent
Yash Sharma
Yash Sharma

Reputation: 1

Batch Gradient Descent converges but with huge error

I am creating a function using Batch Gradient Descent for a linear regression model.
theta0 is intercept, theta1 is slope.
The algorithm is converging but with huge error.

myplot -- red line is sklearn linregress() and blue line is my plot.

def gradient_descent(alpha, x, y):

    m = len(x)

    theta0 = x.iloc[5]
    theta1 = x.iloc[6]

    J = sum([(theta0 + theta1*x.iloc[i] - y.iloc[i])**2 for i in range(m-1)])

    for j in range(1, m):

        grad0 = 1/m * sum([(theta0 + theta1*x.iloc[i] - y.iloc[i]) for i in range(1, m)])
        grad1 = 1/m * sum([(theta0 + theta1*x.iloc[k] - y.iloc[k])*x.iloc[k] for k in range(1, m)])

        # print(sum([(theta0 + theta1*x.iloc[j] - y.iloc[j]) for j in range(1, m)]))
        # print(grad1)

        theta0 -= alpha * grad0
        theta1 -= alpha * grad1

        # e = sum([(theta0 + theta1*x.iloc[i] - y.iloc[i])**2 for i in range(m)])

        # print((e))

    return theta0, theta1

Upvotes: 0

Views: 62

Answers (1)

Yash Sharma
Yash Sharma

Reputation: 1

solved it. we need to take different alphas(learning rates) for different theta(weights).

def gradient_descent(alpha1, alpha2, x, y):
    
    m = len(x)

    theta0 = 0
    theta1 = 0
  
    J = sum([(theta0 + theta1*x.iloc[i] - y.iloc[i])**2 for i in range(1, m)])

    for j in range(1, m):

        grad0 = sum([(theta0 + theta1*(x.iloc[i]) - y.iloc[i]) for i in range(1, m)])
        grad1 = sum([(theta0 + theta1*(x.iloc[k]) - y.iloc[k])*x.iloc[k] for k in range(1, m)])

        # print(sum([(theta0 + theta1*x.iloc[j] - y.iloc[j]) for j in range(1, m)]))
        # print(grad0, grad1)

        theta0 -= alpha1*grad0
        theta1 -= alpha2*grad1

        # e = sum([(theta0 + theta1*x.iloc[i] - y.iloc[i])**2 for i in range(1,m)])

        # print(abs(J - e))
        # J = e
    return theta0, theta1

Upvotes: 0

Related Questions