How a neural network can recognize more patterns at once - how does it work?

Question

I am a beginner when it comes to NN. I understand the basics and I am not sure abou the following - lets consider handwriting recognition network. I understand you can train a network to recognize a pattern, i.e the weights are set appropriately. But if the network will be trained to recognize "A", how could it recognize "B" then, which would certainly require the weights to be set differently? Or does the network only searches for one letter it is currently trained? I hope I made myselft clear - I basically try to understand how a trained network can recognize various characters if the weights will be mixed when training for all.

Answer 1

When a neural network is being trained, what is happening is that the network is searching for a set of weights which when combined with the test inputs, will yield the expected output.

One of the key features in neural networks is the setting up and assignment of the Learning Rate. What this means is essentially how much of the previous acquired information is kept.

It is important that this value be neither too high (if memory serves, setting it to 1 would mean that the weight will be changed by taking into consideration only the current test case) nor too low (setting it up to zero will mean that no weight change will be made). In either case, the neural network would never converge.

When training for hand writing, as far as I know, the training set involves various letters written in various forms. That being said, although neural networks tend to fair better than other AI approaches when there are variations in its input, there are always limitations.

EDIT: As per your question, assuming that you are dealing with a back propagation neural network, what you do is that at each layer, you apply an activation function and pass the result of the current layer to the next.

The extra bit comes during testing, where you compare the result you have with the result you want. This is where you apply the back propagation algorithm to amend the weights, and in this section is where the learning rate comes in.

As you have mentioned in your comment, the weights will be changed, however, the value of the learning rate will determine how much will the weights change. Usually, you want them to change relatively slowly so that they converge, hence why you want to keep the value of the learning rate relatively low. However, if you have a very high learning rate, the current data set will, as you are saying, affect any improvements made by the next.

The way you can look at it is that while training, the neural network is searching for a set of weights which can given its test inputs, it will yield the expected results. So basically, you are looking for weights which satisfy all your test cases.