CODEWITHSUNDEEP
matlabsvd
user3624526
user3624526

Reputation: 69

compute SVD using Matlab function

I have a doubt about SVD. in the literature that i had read, it's written that we have to convert our input matrix into covariance matrix first, and then SVD function from matlab (SVD) is used.

But, in Mathworks website we can use SVD function directly to the input matrix (no need to convert it into covariance matrix)..

   [U,S,V]=svd(inImageD);

  1. Which one is the true??

  2. And if we want to do dimensionality reduction, we have to project our data into eigen vector.. But where is the eigen vector generated by SVD function.. I know that S is the eigen value.. But what is U and S??

  3. To reduce our data dimensional, do we need to substract the input matrix with its mean and then multiply it with eigen vector?? or we can just multiply our input matrix with the eigen vector (no need to substract it first with its mean)..

EDIT

Suppose if I want to do classification using SIFT as the features and SVM as the classifier.

I have 10 images for training and I arrange them in a different row..

So 1st row for 1st images, 2nd row for second images and so on...

Feat=[1 2 5 6 7   >> Images1
      2 9 0 6 5   >> Images2
      3 4 7 8 2   >> Images3
      2 3 6 3 1   >> Images4
      ..
      .
      so on. . ]

To do dimensionality reduction (from my 10x5 matrix), we have yo do A*EigenVector

And from what U had explained (@Sam Roberts), I can compute it by using EIGS function from the covariance matrix (instead of using SVD function).

And as I arrange the feat of images in different row, so I need to do A'*A So it becomes:

 Matrix=A'*A
 MAT_Cov=Cov(Matrix)
 [EigVector EigValue] = eigs (MAT_Cov);

is that right??

Upvotes: 2

Views: 4914

Answers (1)

Sam Roberts
Sam Roberts

Reputation: 24137

Eigenvector decomposition (EVD) and singular value decomposition (SVD) are closely related.

Let's say you have some data a = rand(3,4);. Note that this not a square matrix - it represents a dataset of observations (rows) and variables (columns).

Do the following:

[u1,s1,v1] = svd(a);
[u2,s2,v2] = svd(a');
[e1,d1] = eig(a*a');
[e2,d2] = eig(a'*a);

Now note a few things.

  1. Up to the sign (+/-), which is arbitrary, u1 is the same as v2. Up to a sign and an ordering of the columns, they are also equal to e1. (Note that there may be some very very tiny numerical differences as well, due to slight differences in the svd and eig algorithms).
  2. Similarly, u2 is the same as v1 and e2.
  3. s1 equals s2, and apart from some extra columns and rows of zeros, both also equal sqrt(d1) and sqrt(d2). Again, there may be some very tiny numerical differences as well just due to algorithmic issues (they'll be on the order of something to the -10 or so).

Note also that a*a' is basically the covariances of the rows, and a'*a is basically the covariances of the columns (that's not quite true - a would need to be centred first by subtracting the column or row mean for them to be equal, and there might be a multiplicative constant difference as well, but it's basically pretty similar).

Now to answer your questions, I assume that what you're really trying to do is PCA. You can do PCA either by taking the original data matrix and applying SVD, or by taking its covariance matrix and applying EVD. Note that Statistics Toolbox has two functions for PCA - pca (in older versions princomp) and pcacov.

Both do essentially the same thing, but from different starting points, because of the above equivalences between SVD and EVD.

Strictly speaking, u1, v1, u2 and v2 above are not eigenvectors, they are singular vectors - and s1 and s2 are singular values. They are singular vectors/values of the matrix a. e1 and d1 are the eigenvectors and eigenvalues of a*a' (not a), and e2 and d2 are the eigenvectors and eigenvalues of a'*a (not a). a does not have any eigenvectors - only square matrices have eigenvectors.

Centring by subtracting the mean is a separate issue - you would typically do that prior to PCA, but there are situations where you wouldn't want to. You might also want to normalise by dividing by the standard deviation but again, you wouldn't always want to - it depends what the data represents and what question you're trying to answer.

Upvotes: 4

Related Questions