CODEWITHSUNDEEP
c#neural-network
Remmie
Remmie

Reputation: 47

Why does my neural network Unlearn what it learned?

I copied some of this source code in C#, from another web site and tested the neural network by itself with only a few inputs and two outputs and a hidden layer consisting of 4.

When the neural network gets trained on the expected outputs of a letter, be it A or B or C (Which are formatted as a monotone image made up of 16 by 16 pixels) --Which makes up 126 bytes. I convert the image's literal bytes into a byte array, then convert those into a double and I feed the neural network the entire double array as the input, and I give it the expected output. It trains, on just one letter.

When I train it through other letters, it seems to adjust it's weights in a way which make it "forget" other letters that it learned.

How can I have it so that the weights don't get over written? Am I just giving it the wrong expected values?

Here is the code, keep note it's from another source but i have heavily modified it. One thing I did was write the train function as well as the test function so that I would have easier access to use that part of the code.

The letter A is supposed to output a 0.99 value roughly on the first neuron in the outputs. The rest of the outputs are supposed to be 0.25

The letter B respectively is supposed to output a 0.99 roughly value on the SECOND neuron in the outputs. The rest of the outputs are supposed to be 0.25

Here is the full source code: http://pastebin.com/0E3WM42W It wouldn't let me put code into this because the message body I had was over 33, 000 characters. Anyways Basically the parts I added were the Image loading functions, the image to byte array functions and all that fancy stuff. I was modifying (heavily) the already existing neural network code example. Luckily the examples given were so modular I didn't have to change the classes at all that support the network I'm just having issues adding my own structure as a formality of the network. So any help is appreciated I would really love this network to be able to preserve DIFFERENT information that it learns.

I don't know if it's something I'm doing wrong or if the neural network that I am using simply doesn't support learning multiple things at once (it doesn't seem to be very good at giving a variety of output !! That's for sure )

Upvotes: 3

Views: 466

Answers (1)

Mykhaylo Kopytonenko
Mykhaylo Kopytonenko

Reputation: 953

The most probable reason is that you don't train the network simultaneously with all data patterns (letters, in your case). Usually, the whole set of the training data is fed to the network, and this process is repeated many times until the answer error for the whole data set reaches needed level.

When you present only one letter many times - the network learns only it and nothing else. When you start to train the network with another letter - it modifies the weights to learn the new letter. But the previous data is gradually being forgotten.

Your hidden layer of 4 neurons is more than enough for the 3 letter patterns. So, it is not the reason for your failure.

Upvotes: 2

Related Questions