Draw the vector w as well as the projection of another vector onto w

Question

How can I plot the vector w with the projected data onto this vector? Here is the code - and my trials to plot the weight vector with y1 and y2.

x1=[1 2;2 3;3 3;4 5;5 5]  % the first class 5 observations
x2=[1 0;2 1;3 1;3 2;5 3;6 5]
m1 = mean(x1); 
m2 = mean(x2); 
m = m1 + m2; 
d1=x1-repmat(m1,5,1);
d2=x2-repmat(m2,6,1);
c = 0.5.*m; 
Sw1 = d1'*d1;
Sw2 = d2'*d2;
Sw = Sw1 + Sw2; 
invSw = inv(Sw);
w= invSw*(m1-m2)' %this is my vector projected
scatter(x1(:,1), x1(:,2), 10, 'ro');
hold on;
scatter(x2(:,1), x2(:,2), 10,'bo');
%this is how i plot the decision boundary, but it doesn't seems correct. 
quiver(c(1,1), c(1,2), 1, -w(1,1)/w(2,1));
quiver(c(1,1), c(1,2), -1, w(1,1)/w(2,1));

auxw= w/norm(w);
plot([0 auxw(1)], [0 auxw(2)]) 
hold off;
figure; 
y1 = x1*w; 
y2 = x2*w; 
hist([y1' y2'])

Answer 1

You are very close. You've only calculated (or tried to calculate) the scalar projection or the amount of scale you apply to each vector in order to project each vector in x1 and x2 onto w though what you have is incomplete. If you recall from linear algebra, to determine the scalar projection between two vectors a and b, or the scalar projection of b onto a, the formula is:

^{Source: Oregon State Mathematics: Calculus for Undergraduates}

In our case, a would be w and b would be each of the vectors seen in x1 and x2. I'm assuming each row of these matrices is a vector. The scalar projections are seen in y1 and y2. You need to compute the vector projection, which is defined as taking the scalar projections and multiplying by the unit vectors of a, or simply:

^{Source: Oregon State Mathematics: Calculus for Undergraduates}

Therefore, the calculation of the scalar projections in y1 and y2 are incorrect. You have to multiply by the normalized vector w, then when you find these scalar projection values, you multiply each of these scalar values with the corresponding normalized vector w. However, plotting these all simultaneously on a graph will be confusing. You will have many lines that will overlap onto the original vector w so what I did was I looped through plotting w, a vector in either x1 or x2 and the corresponding projected vector. Each time we loop, we pause and show the data then clear the figure and start again.

As such, I've added and changed the following to your code.

%// Your data
w = [-0.7936; 0.8899];
x1 = [1 2; 2 3; 3 3; 4 5; 5 5];
x2 = [1 0; 2 1; 3 1; 3 2; 5 3; 6 5];

%// Compute scalar projection
auxw = w/norm(w);
s1 = x1*auxw;
s2 = x2*auxw; %// Change for correctness

%// Compute the vector projection
y1 = bsxfun(@times, s1, auxw.');
y2 = bsxfun(@times, s2, auxw.');

%// Place the original vectors and corresponding projections
%// in one matrix
y = [y1; y2];
x = [x1; x2];

%// Loop through and plot w, a point in either x1 or x2
%// and the corresponding projection
for ii = 1 : size(y,1)
    plot([0 w(1)], [0 w(2)]);
    hold on;
    plot([0 y(ii,1)], [0 y(ii,2)], 'r');
    plot([0 x(ii,1)], [0 x(ii,2)], 'g');
    pause(0.5);
    clf;
end

The function bsxfun allows us to multiply each vector in x1 and x2 by their corresponding scalar values. Specifically, it will take the vectors s1 and s2 and when we transpose auxw to be a 1 x 2 vector, we will create new matrices y1 and y2 where each row of either will compute the vector projections of x1 and x2 and place them into the rows of y1 and y2.

The loop at the end cycles through w, a vector in either x1 or x2 and the corresponding projected vector one at a time and we pause for 0.5 seconds each time to see what the results look like. The vector w is in blue, the projected vector is in green and the original vector from either x1 or x2 is in red.

We get these series of figures:

We can see that the red line, which is the projected vector from either x1 or x2 onto w. The green line is the original vector from either x1 or x2.