My Python code for logistic regression gradient for Andrew Ng course does not work

Question

I have written some code for the non-regularized logistic regression cost function and for finding a gradient but no matter what I try, my code keeps returning the same TypeError.

I have already tried a vectorized and for-loop implementation of my code and nothing has been working. I would also like to point out that the grader always grades my cost function with full marks but not the code for finding the partial derivatives. My results always match the expected cost, but nothing is returned for the gradient part.

It says that this is the Cost: J(𝜃)=1𝑚∑𝑖=1𝑚[−𝑦(𝑖)log(ℎ𝜃(𝑥(𝑖)))−(1−𝑦(𝑖))log(1−ℎ𝜃(𝑥(𝑖)))]

And this is the Partial Derivative: ∂𝐽(𝜃)∂𝜃𝑗=1𝑚∑𝑖=1𝑚(ℎ𝜃(𝑥(𝑖))−𝑦(𝑖))𝑥(𝑖)𝑗

(From doing the course, I can verify that this is correct)

def costFunction(theta, X, y):
    # Initialize some useful values
    m = y.size  # number of training examples

    # You need to return the following variables correctly 
    J = 0
    grad = np.zeros(theta.shape)

    # ====================== YOUR CODE HERE ============
    for i in range(m):
        hypothesis = sigmoid(np.dot(theta.T, X[i, :]))
        J += y[i] * math.log(hypothesis) + (1 - y[i]) * math.log(1 - hypothesis)
        for j in range(n):
            grad = (hypothesis - y[i]) * X[i, j]

    J = (-1 / m) * J
    grad = (1 / m) * grad
    # =============================================================
    return J, grad



# Initialize fitting parameters
initial_theta = np.zeros(n+1)

cost, grad = costFunction(initial_theta, X, y)

print('Cost at initial theta (zeros): {:.3f}'.format(cost))
print('Expected cost (approx): 0.693\n')

print('Gradient at initial theta (zeros):')
#print('\t[{:.4f}, {:.4f}, {:.4f}]'.format(*grad))
print('Expected gradients (approx):\n\t[-0.1000, -12.0092, -11.2628]\n')

# Compute and display cost and gradient with non-zero theta
test_theta = np.array([-24, 0.2, 0.2])
cost, grad = costFunction(test_theta, X, y)

print('Cost at test theta: {:.3f}'.format(*cost))
print('Expected cost (approx): 0.218\n')

print('Gradient at test theta:')
print('\t[{:.3f}, {:.3f}, {:.3f}]'.format(*grad))
print('Expected gradients (approx):\n\t[0.043, 2.566, 2.647]')

I expect the output to be:

Cost at initial theta (zeros): 0.693
Expected cost (approx): 0.693

Gradient at initial theta (zeros):
    [-0.1000, -12.0092, -11.2628]
Expected gradients (approx):
    [-0.1000, -12.0092, -11.2628]

but instead I get the following:

Cost at initial theta (zeros): 0.693
Expected cost (approx): 0.693

Gradient at initial theta (zeros):
Expected gradients (approx):
    [-0.1000, -12.0092, -11.2628]

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-35-ab2a7b541269> in <module>()
     15 cost, grad = costFunction(test_theta, X, y)
     16 
---> 17 print('Cost at test theta: {:.3f}'.format(*cost))
     18 print('Expected cost (approx): 0.218\n')
     19 

TypeError: format() argument after * must be an iterable, not numpy.float64

Answer 1

When looking at the function costFunction(), the return value J (which you are assigning to cost) is a scalar. Therefore, it cannot be unpacked with a star * and should be passed directly to the string formatting method:

print('Cost at test theta: {:.3f}'.format(cost)) # passing 'cost' without the star