CODEWITHSUNDEEP
pythonnumpylinear-algebra
P. Camilleri
P. Camilleri

Reputation: 13218

Numpy largest singular value larger than greatest eigenvalue

Let 

import numpy as np

M = np.array([[ 1., -0.5301332 , 0.80512845],
              [ 0., 0., 0.],
              [ 0., 0., 0.]])

M is rank one, its only non zero eigenvalue is 1 (its trace). However np.linalg.norm(M, ord=2) returns 1.39 which is strictly greater than 1. Why?

The eigenvalues of M, returned by np.linalg.eigvals are 1, 0, 0, but the singular values of M are 1.39, 0, 0, which is a surprise to me. What did I miss?

Upvotes: 1

Views: 995

Answers (3)

FBruzzesi
FBruzzesi

Reputation: 6475

In this particular case the 2-norm of M coincides with the Frobenius norm, which is given by the formula (np.sum(np.abs(M**2)))**(1/2), therefore we can see that:

import numpy as np

M = np.array([[ 1., -0.5301332 , 0.80512845],
              [ 0., 0., 0.],
              [ 0., 0., 0.]])

np.sqrt(np.sum(np.abs(M**2)))
1.388982732341062

np.sqrt(np.sum(np.abs(M**2))) == np.linalg.norm(M,ord=2) == np.linalg.norm(M, ord='fro')
True

In particular one can prove that the 2-norm is the square root of the largest eigenvalue of M.T@M i.e.

np.sqrt(np.linalg.eigvals(M.T@M)[0])
1.388982732341062

And this is its relation with eigenvalues of a matrix. Now recall that the singular values are the square root of the eigenvalues of M.T@M and we unpack the mistery.

Using a characterization of the Frobenius norm (square root of the sum of the trace of M.T@M):

np.sqrt(np.sum(np.diag(M.T@M)))
1.388982732341062

Confronting the results:

np.sqrt(np.linalg.eigvals(M.T@M)[0]) == np.sqrt(np.sum(np.diag(M.T@M))) == np.linalg.svd(M)[1][0]
True

Upvotes: 1

R Liab
R Liab

Reputation: 156

This is perfectly normal. In the general case, the singular values are not equals to the eigen values. This is true only for positive Hermitian matrices.

For squared matrices, you have the following relationship: 

M = np.matrix([[ 1., -0.5301332 , 0.80512845],
              [ 0., 0., 0.],
              [ 0., 0., 0.]])
u, v= np.linalg.eig(M.H @ M) # M.H @ M is Hermitian
print(np.sqrt(u)) # [1.38898273 0.         0.        ]
u,s,v = lin.svd(M)
print(s) # [1.38898273 0.         0.        ]

Upvotes: 0

Aly Hosny
Aly Hosny

Reputation: 827

second norm of a matrix the square root of the sum of all elements squared

norm(M, ord=2) = (1.**2  + 0.5301332**2 + 0.80512845**2)**0.5 = 1.39

to get the relation between the eigen values and singular values you need to calculate the eigen values of M^H.M and square root it

eigV = np.linalg.eigvals(M.T.dot(M))
array([1.92927303, 0.        , 0.        ])
eigV**0.5
array([1.38898273, 0.        , 0.        ])

Upvotes: 0

Related Questions