CODEWITHSUNDEEP
machine-learningneural-networkxorperceptron
smatthewenglish
smatthewenglish

Reputation: 2889

basic intuition for perceptron algorithm / linear regression

I'm studying about neural networks and I came across the following xorcise (that was a joke!) question.

I asked my friend if I need to implement the perceptron algorithm to solve it and he said "no- just think about it". Well, I thought about it, but my small monkey brain was only able to come up with the following:

My friends words make me think it's a trick question, and the only trick we've discussed thus far is the inability of perceptron to do an XOR function.

Is that was this question is getting at?

How to solve this problem?...

A simple Perzeptron with two inputs x₁, x₂,  
BIAS, and transfer function y = f(z) = sgn(z), 
separates the two dimensional input space into 
two parts with help of a line g. 

Calculate for this Perzeptron the weights 
w1, w2, wb, so that the line separates 
the given 6 patterns (pX1, pX2; Py) into 
two classes:

1X = (0,  0;  -1), 
2X = (2, -2; +1), 
3X = (7 + ε, 3 - ε; +1), 
4X = (7  -  ε, 3 + ε; -1), 
5X = (0, -2 - ε; +1), 
6X = (0 - ε, -2; -1), 


Remark: 0 < ε << 1.

Upvotes: 1

Views: 420

Answers (1)

ergonaut
ergonaut

Reputation: 7057

If you graph the points, you'll see that all the -1s are on the top left side, and all the +1s are on the bottom right. You can draw a line intersecting (0, -2) and (7,3) which gives you the expression:

y = 5x/7 - 2

which is enough to skip running through any algorithm.

The equation of the line to predict +1 occurences is given by:

y < 5x/7 - 2

The line above splits the 2 dimensional space in two. The shaded area is BELOW the line, and the line goes up and to the right. So for any arbitrary point, you just have to figure out if it's in the shaded area (positive prediction = +1).

Say, (pX1, pX2) = (35, 100),

1) one way is to plug pX1 back in the formula (for x' = pX1) to find the closest part on the line (where y=5x/7-2):

y' = 5(35)/7 - 2
y' = 23

Since the point on the line is (35, 23) and the point we are interested is (35, 100), it is above the line. In other words, pX2 is NOT < 23, the prediction returns -1.

2) plot y'=100, so 

100 = 5x/7-2 
x = 142.8

Line point=(142.8, 100), your point (35, 100), our point is on the left of the line point, it still falls outside the shaded area.

3) You can even graph it and visually check if it's in the shaded area

The point is some calculation has to be done to check if it's IN or OUT. That's the point of linear regression. It should be really simple for the machine to figure out because you're just calculating one thing. Predicting once you have your formulas should be quick. The hardest part is determining the formula for the line, which we've already done by graphing the points and seeing an obvious solution. If you use machine learning, it will take longer in this case.

Upvotes: 2

Related Questions