Pure Python Implementation of gradient descent

Question

I have tried to implement gradient descent myself using Python. I know there are similar topics on this, but for my attempt, my guess slope can always get really close to the real slope, but the guess intercept never matched or even come close to the real intercept. Does anyone know why is that happening?

Also, I read a lot of gradient descent post and formula, it says for each iteration, I need to multiply the gradient by the negative learning rate and repeat until it converges. As you can see in my implementation below, my gradient descent only works when I multiply the learning rate to the gradient and not by -1. Why is that? Did I understand the gradient descent wrong or is my implementation wrong? (The exam_m and exam_b will quickly go overflow if I multiply the learning rate and gradient by -1)

intercept = -5
slope = -4


x = []
y = []
for i in range(0, 100):
    x.append(i/300)
    y.append((i * slope + intercept)/300)

learning_rate = 0.005
# y = mx + b 
# m is slope, b is y-intercept

exam_m = 100
exam_b = 100

#iteration
#My error function is sum all (y - guess) ^2
for _ in range(20000):
    gradient_m = 0
    gradient_b = 0
    for i in range(len(x)):
        gradient_m += (y[i] - exam_m * x[i] - exam_b) * x[i]
        gradient_b += (y[i] - exam_m * x[i] - exam_b)
        #why not gradient_m -= (y[i] - exam_m * x[i] - exam_b) * x[i] like what it said in the gradient descent formula

    exam_m += learning_rate * gradient_m
    exam_b += learning_rate * gradient_b
    print(exam_m, exam_b)

Answer 1

The reason for overflow is the missing factor (2/n). I have broadly shown the use of negative signs for more clarification.

import numpy as np
import matplotlib.pyplot as plt

intercept = -5
slope = -4
# y = mx + b

x = []
y = []

for i in range(0, 100):
    x.append(i/300)
    y.append((i * slope + intercept)/300)

n = len(x)
x = np.array(x)
y = np.array(y)

learning_rate = 0.05
exam_m = 0
exam_b = 0
epochs = 1000

for _ in range(epochs):
    gradient_m = 0
    gradient_b = 0
    for i in range(n):
        gradient_m -= (y[i] - exam_m * x[i] - exam_b) * x[i]
        gradient_b -= (y[i] - exam_m * x[i] - exam_b)

    exam_m = exam_m - (2/n)*learning_rate * gradient_m
    exam_b = exam_b - (2/n)*learning_rate * gradient_b

print('Slope, Intercept: ', exam_m, exam_b)

y_pred = exam_m*x + exam_b
plt.xlabel('x')
plt.ylabel('y')
plt.plot(x, y_pred, '--', color='black', label='predicted_line')
plt.plot(x, y, '--', color='blue', label='orginal_line')
plt.legend()
plt.show()

Output: Slope, Intercept: -2.421033215481844 -0.2795651072061604