CODEWITHSUNDEEP
neural-networkencog
Ray Lionfang
Ray Lionfang

Reputation: 687

Encog Neural Network error not changing

I am creating a neural network that works on colored images. but when i train it, the error will never change after some point. even after a thousand iterations. what causes it? or what should i do? here is the structure:

BasicNetwork network = new BasicNetwork();
        network.addLayer(new BasicLayer(null,true,16875));
        network.addLayer(new BasicLayer(new ActivationSigmoid(),true,(50)));
        network.addLayer(new BasicLayer(new ActivationSigmoid(),true,setUniqueNumbers.size()));
        network.getStructure().finalizeStructure();
        network.reset();

the input layer is actually 75 * 75 (75x75 pixels) *3 (red, green, blue), so i came up with 16875.

Upvotes: 2

Views: 876

Answers (1)

Josh T
Josh T

Reputation: 564

When the error stops changing, you have hit a minimum, probably a local minimum.

This means that it has found what it thinks is the best solution so far, and to move away from that point will cause more error (even if it has to go up hill a bit before it can go down hill to an even lower error rate). This happens early when there isn't a strong pattern/correlation in the data. It can also happen when the structure is out of wack.

That looks like it may definitely be one of your problems. ~17,000 input neurons is a ton. And then 50 hidden neurons doesn't seem to match up well with so many inputs. Instead of feeding in so much data, find a way to extract features to reduce the input size and make it more meaningful to the network.

Examples that may help it run better:

  • Down sample the image from 75*75 to, say, 10*10. This depends on what the picture is.
  • Convert from RGB to gray scale
  • Learn about feature extraction - if you can extract features from the image, such as lines, edges, etc., the net will have a lot more to learn from. Seriously, feature extraction is your golden ticket to making ANNs work.

Good luck!

Upvotes: 4

Related Questions