CODEWITHSUNDEEP
matlabpca
user765195
user765195

Reputation: 423

Doing a PCA using an optimization in Matlab

I'd like to find the principal components of a data matrix X in Matlab by solving the optimization problem min||X-XBB'||, where the norm is the Frobenius norm, and B is an orthonormal matrix. I'm wondering if anyone could tell me how to do that. Ideally, I'd like to be able to do this using the optimization toolbox. I know how to find the principal components using other methods. My goal is to understand how to set up and solve an optimization problem which has a matrix as the answer. I'd very much appreciate any suggestions or comments.

Thanks! MJ

Upvotes: 1

Views: 1111

Answers (3)

Dan
Dan

Reputation: 45752

Do you have the optimization toolbox? The documentation is really good, just try one of their examples: http://www.mathworks.com/help/toolbox/optim/ug/brg0p3g-1.html.

But in general the optimization function look like this:

[OptimizedMatrix, OptimizedObjectiveFunction] = optimize( (@MatrixToOptimize) MyObjectiveFunction(MatrixToOptimize), InitialConditionsMatrix, ...optional constraints and options... );

You must create MyObjectiveFunction() yourself, it must take the Matrix you want to optimize as an input and output a scalar value indicating the cost of the current input Matrix. Most of the optimizers will try to minimise this cost. Note that the cost must be a scalar.

fmincon() is a good place to start, once you are used to the toolbox you and if you can you should choose a more specific optimization algorithm for your problem.

To optimize a matrix rather than a vector, reshape the matrix to a vector, pass this vector to your objective function, and then reshape it back to the matrix within your objective function.

For example say you are trying to optimize the 3 x 3 matrix M. You have defined objective function MyObjectiveFunction(InputVector). Pass M as a vector:

MyObjectiveFunction(M(:));

And within the MyObjectiveFunction you must reshape M (if necessary) to be a matrix again:

  function cost = MyObjectiveFunction(InputVector)
      InputMatrix = reshape(InputVector, [3 3]);

      %Code that performs matrix operations on InputMatrix to produce a scalar cost

      cost = %some scalar value
  end

Upvotes: 1

Rasman
Rasman

Reputation: 5359

The thing about Optimization is that there are different methods to solve a problem, some of which can require extensive computation.

Your solution, given the constraints for B, is to use fmincon. Start by creating a file for the non-linear constraints:

function [c,ceq] = nonLinCon(x)
c = 0;
ceq = norm((x'*x - eye (size(x))),'fro'); %this checks to see if B is orthonormal.

then call the routine:

B = fmincon(@(B) norm(X - X*B*B','fro'),B0,[],[],[],[],[],[],@nonLinCon)

with B0 being a good guess on what the answer will be.

Also, you need to understand that this algorithms tries to find a local minimum, which may not be the solution you ultimately want. For instance:

X = randn(1,2)
fmincon(@(B) norm(X - X*B*B','fro'),rand(2),[],[],[],[],[],[],@nonLinCon)
ans =
    0.4904    0.8719
    0.8708   -0.4909
fmincon(@(B) norm(X - X*B*B','fro'),rand(2),[],[],[],[],[],[],@nonLinCon)
ans =
    0.9864   -0.1646
    0.1646    0.9864

So be careful, when using these methods, and try to select a good starting point

Upvotes: 2

kitchenette
kitchenette

Reputation: 1623

The Statistics toolbox has a built-in function 'princomp' that does PCA. If you want to learn (in general, without the optimization toolbox) how to create your own code to do PCA, this site is a good resource.

Since you've specifically mentioned wanting to use the Optimization Toolbox and to set this up as an optimization problem, there is a very well-trusted 3rd-party package known as CVX from Stanford University that can solve the optimization problem you are referring to at this site.

Upvotes: 2

Related Questions