CODEWITHSUNDEEP
phpneural-networkperceptron
Alexander
Alexander

Reputation: 650

Basic perceptron for AND gate in PHP, am I doing it right? Weird results

I'd like to learn about neural nets starting with the very basic perceptron algorithm. So I've implemented one in PHP and I'm getting weird results after training it. All the 4 possible input combinations return either wrong or correct results (more often the wrong ones).

1) Is there something wrong with my implementation or the results I'm getting are normal?

2) Can this kind of implementation work with more than 2 inputs?

3) What would be the next (easiest) step in learning neural nets after this? Maybe adding more neurons, changing the activation function, or ...?

P.S. I'm pretty bad at math and don't necessarily understand the math behind perceptron 100%, at least not the training part.

Perceptron Class

<?php

namespace Perceptron;

class Perceptron
{
    // Number of inputs
    protected $n;

    protected $weights = [];

    protected $bias;

    public function __construct(int $n)
    {
        $this->n = $n;

        // Generate random weights for each input
        for ($i = 0; $i < $n; $i++) {
            $w = mt_rand(-100, 100) / 100;

            array_push($this->weights, $w);
        }

        // Generate a random bias
        $this->bias = mt_rand(-100, 100) / 100;
    }

    public function sum(array $inputs)
    {
        $sum = 0;

        for ($i = 0; $i < $this->n; $i++) {
            $sum += ($inputs[$i] * $this->weights[$i]);
        }

        return $sum + $this->bias;
    }

    public function activationFunction(float $sum)
    {
        return $sum < 0.0 ? 0 : 1;
    }

    public function predict(array $inputs)
    {
        $sum = $this->sum($inputs);

        return $this->activationFunction($sum);
    }

    public function train(array $trainingSet, float $learningRate)
    {
        foreach ($trainingSet as $row) {
            $inputs = array_slice($row, 0, $this->n);
            $correctOutput = $row[$this->n];

            $output = $this->predict($inputs);
            $error = $correctOutput - $output;

            // Adjusting the weights
            $this->weights[0] = $this->weights[0] + ($learningRate * $error);
            for ($i = 0; $i < $this->n - 1; $i++) {
                $this->weights[$i + 1] =
                    $this->weights[$i] + ($learningRate * $inputs[$i] * $error);
            }
        }

        // Adjusting the bias
        $this->bias += ($learningRate * $error);
    }
}

Main File

<?php

require_once 'vendor/autoload.php';

use Perceptron\Perceptron;

// Create a new perceptron with 2 inputs
$perceptron = new Perceptron(2);

// Test the perceptron
echo "Before training:\n";

$output = $perceptron->predict([0, 0]);
echo "{$output} - " . ($output == 0 ? 'correct' : 'nope') . "\n";

$output = $perceptron->predict([0, 1]);
echo "{$output} - " . ($output == 0 ? 'correct' : 'nope') . "\n";

$output = $perceptron->predict([1, 0]);
echo "{$output} - " . ($output == 0 ? 'correct' : 'nope') . "\n";

$output = $perceptron->predict([1, 1]);
echo "{$output} - " . ($output == 1 ? 'correct' : 'nope') . "\n";

// Train the perceptron
$trainingSet = [
    // The 3rd column is the correct output
    [0, 0, 0],
    [0, 1, 0],
    [1, 0, 0],
    [1, 1, 1],
];

for ($i = 0; $i < 1000; $i++) {
    $perceptron->train($trainingSet, 0.1);
}

// Test the perceptron again - now the results should be correct
echo "\nAfter training:\n";

$output = $perceptron->predict([0, 0]);
echo "{$output} - " . ($output == 0 ? 'correct' : 'nope') . "\n";

$output = $perceptron->predict([0, 1]);
echo "{$output} - " . ($output == 0 ? 'correct' : 'nope') . "\n";

$output = $perceptron->predict([1, 0]);
echo "{$output} - " . ($output == 0 ? 'correct' : 'nope') . "\n";

$output = $perceptron->predict([1, 1]);
echo "{$output} - " . ($output == 1 ? 'correct' : 'nope') . "\n";

Upvotes: 0

Views: 330

Answers (2)

Alexander
Alexander

Reputation: 650

Found my silly mistake, I wasn't adjusting the bias for each row of a training set as I accidentally put it outside the foreach loop. This is what the train() method should look like:

public function train(array $trainingSet, float $learningRate)
{
    foreach ($trainingSet as $row) {
        $inputs = array_slice($row, 0, $this->n);
        $correctOutput = $row[$this->n];

        $output = $this->predict($inputs);
        $error = $correctOutput - $output;

        // Adjusting the weights
        for ($i = 0; $i < $this->n; $i++) {
            $this->weights[$i] += ($learningRate * $inputs[$i] * $error);
        }

        // Adjusting the bias
        $this->bias += ($learningRate * $error);
    }
}

Now I get the correct results after training each time I run the script. Just 100 epochs of training is enough.

Upvotes: 0

WNP
WNP

Reputation: 161

I must thank you for posting this question, I have wanted a chance to dive a little deeper into neural networks. Anyway, down to business. After tinkering around and verbose logging what all is happening, it ended up only requiring 1 character change to work as intended:

public function sum(array $inputs)
{
    ...
    //instead of multiplying the input by the weight, we should be adding the weight
    $sum += ($inputs[$i] + $this->weights[$i]);
    ...
}

With that change, 1000 iterations of training ends up being overkill. One bit of the code was confusing, different setting of weights:

public function train(array $trainingSet, float $learningRate)
{
    foreach ($trainingSet as $row) {
        ...
        $this->weights[0] = $this->weights[0] + ($learningRate * $error);
        for ($i = 0; $i < $this->n - 1; $i++) {
            $this->weights[$i + 1] =
                $this->weights[$i] + ($learningRate * $inputs[$i] * $error);
        }
}

I don't necessarily understand why you chose to do it this way. My unexperienced eye would think that the following would work as well.

for ($i = 0; $i < $this->n; $i++) { 
    $this->weight[$i] += $learningRate * $error;
}

Upvotes: 0

Related Questions