CODEWITHSUNDEEP
neural-networkdata-analysis
edtech
edtech

Reputation: 1754

Neural network not learning

I try to create a neural network with 1 hidden layer (let's assume that data vector contains 4 values, there are 3 neurons on the input layer, 3 neurons on the hidden layer and 1 neuron on the output level). I have two vectors of data with two known results.

I teach the network using first set of data, then I apply the second set. The weights are corrected by using back propagation method. The issue that if I try to predict the values of the first set after weights correction, I get a result which is very close to the second result. So, neural network "forgets" the first training.

A full code of my program is here https://gist.github.com/edtechd/63aace5d88dee1ab6835

Weights values during and after teaching are here https://gist.github.com/edtechd/7f19f0759bb808a31a3f

Here is the NN training function

    public void Train(double[] data, double expectedResult)
    {
        double result = Predict(data);
        double delta = Perceptron.ActivationFunction(expectedResult) - Perceptron.ActivationFunction(result);
        double eta = 20;

        // Calculate layer 2 deltas
        for (int i = 0; i < size2; i++)
        {
            deltas2[i] = delta * weights3[i];
        }

        // Calculate layer 1 deltas
        for (int i = 0; i < size1; i++)
        {
            deltas1[i] = 0;

            for(int j=0; j < size2; j++) {
                deltas1[i] += deltas2[j] * weights2[j * size1 + i];
            }
        }

        // Correct layer 1 weights
        for (int i = 0; i < data.Length; i++)
        {
            for (int j = 0; j < size1; j++) 
            {
                weights1[j * data.Length + i] += eta * deltas1[j] * values1[j] * (1 - values1[j]) * data[i];
            }
        }

        // Correct layer 2 weights
        for (int i = 0; i < size1; i++)
        {
            for (int j = 0; j < size2; j++)
            {
                weights2[j * size1 + i] += eta * deltas2[j] * values2[j] * (1 - values2[j]) * values1[i];
            }
        }

        double resultA = Perceptron.ActivationFunction(result);
        for (int i = 0; i < size2; i++)
        {
            weights3[i] += eta * delta * resultA * (1 - resultA) * values2[i];
        }

    }

Am I missed something?

Upvotes: 0

Views: 145

Answers (1)

edtech
edtech

Reputation: 1754

I have figured out with the problem.

On the teaching step, I was repeatedly showing a first example to the network until the result is close to expected, then I was showing a second example.

  A A A A A B B B B B B

The neural network converges and recognizes examples correctly if I repeatedly show both examples in turn.

 A B A B A B A B A B A B

Upvotes: 1

Related Questions