CODEWITHSUNDEEP
pythonnumpyleast-squares
Behrooz Ns
Behrooz Ns

Reputation: 63

numpy.linalg.lstsq too slow

I am trying to compare the performance of numpy.linalg.lstsq against solving the least-squares problem manually. I wrote the following code.

import numpy as np
import timeit

m,n = 400,10

A = np.random.rand(m,n)
b = np.random.rand(m)

t1 = timeit.timeit(lambda : np.linalg.inv(A.T@A) @ A.T @ b, number=100)
t2 = timeit.timeit(lambda : np.linalg.solve(A.T@A, A.T@b), number=100)
t3 = timeit.timeit(lambda : np.linalg.lstsq(A,b)[0], number=100)

print(t1)
print(t2)
print(t3)

To my surprise, the output is

0.0040054810015135445
0.002654149997397326
0.010454912000568584

lstsq is about 5 times slower. Why does this happen?

Upvotes: 0

Views: 1273

Answers (1)

Eric Johnson
Eric Johnson

Reputation: 714

From lstsq you may see it has a feature which the others doesn't:

If there are multiple minimizing solutions, the one with the smallest 2-norm is returned.

In order to do so the problem needs to be examined and sometimes to be solved using the Pseudo Inverse while other methods can do other things like QR Decomposition.

Upvotes: 2

Related Questions