CODEWITHSUNDEEP
c++perceptron
NIkola Bozin
NIkola Bozin

Reputation: 33

Perceptron implemented in C++ does not train as expected. (AND Logic Gate example)

Below is my Perceptron implementation.

The last iteration of FOR loop gives the result:

Input: 0 Input: 0

Output: 0.761594

Error: -0.761594

Which is obviously wrong after so many training samples.

The last few lines of code give the

Input: 1 Input: 1

Output: 0.379652

Error: 0.620348

Which is wrong again and way off...

(All with respect to random weight values in constructor.)

But, if I iterate only for example values (1,1,1), the result will go closer to 1 with each iteration, and that's how it should work.

So I would like to know what could be a cause of this? Because Perceptron should be able to learn AND Gate because outputs are linearly separable.

#include <iostream>
#include <time.h>
#include <stdlib.h>
#include <Windows.h>
#include <math.h>


#define println(x) std::cout<<x<<std::endl;
#define print(x) std::cout<<x;
#define END system("PAUSE"); return 0
#define delay(x) Sleep(x*1000);

typedef unsigned int uint;



class perceptron
{
public:
    perceptron() :learningRate(0.15),biasValue(1),outputVal(0)
    {
        srand((uint)time(0));
        weights = new double[2];
        weights[0] = rand() / double(RAND_MAX);
        weights[1] = rand() / double(RAND_MAX); 
    }
    ~perceptron()
    {
        delete[] weights;
    }
    void train(double x0, double x1, double target)
    {
        backProp(x0, x1, target);
    }
private:
    double biasValue;
    double  outputVal;
    double* weights;
    double learningRate;
private:
    double activationFunction(double sum)
    {
        return tanh(sum);
    }
    void backProp(double x0, double x1, double target)
    {
        println("");

        guess(x0, x1); //Setting outputVal to activationFunction value

        //Calculating Error;

        auto error = target - outputVal;

        //Recalculating weights;

        weights[0] = weights[0] + error * x0 * learningRate;   
        weights[1] = weights[1] + error * x1 * learningRate;

        //Printing values;

        std::cout << "Input:  " << x0 << "   Input:   " << x1 << std::endl;
        std::cout << " Output:   " << outputVal << std::endl;
        std::cout << "Error:   " << error << std::endl;
    }
    double guess(double x0, double x1)
    {
        //Calculating outputValue

        outputVal = activationFunction(x0 * weights[0] + x1 * weights[1]+biasValue);

        return outputVal;
    }
};

int main()
{
    perceptron* p = new perceptron();
    for (auto i = 0; i < 1800; i++)
    {
        p->train(1, 1, 1);
        p->train(0, 1, 0);
        p->train(1, 0, 0);
        p->train(0, 0, 0);
    }
    println("-------------------------------------------------------");
    delay(2);
    p->train(1, 1, 1);
    END;
}

Upvotes: 2

Views: 291

Answers (1)

Nebiyou Yismaw
Nebiyou Yismaw

Reputation: 800

I see a few issues:

  1. A perception activation shouldn't be tanh(). If you use that, you will have to make sure to compute the gradients appropriately. But you can replace the activation by
double activationFunction(double sum)
{
    return sum > 0;
}

This will return 1 if sum > 0, otherwise it returns 0.

  1. When updating weights, biasValue should also be updated,as the perceptron needs to learn its value from your training data. You can update it using
biasValue += error * learningRate;

These changes will allow the perceptron to learn the AND gate.

Upvotes: 2

Related Questions