Neural Network returning same output for every input

Question

I have written a simple Artificial Neural Network in Java as part of a project. When I begin training the data (Using a training set i gathered) The error count in each epoch quickly stabilizes (to around 30% accuracy) and then stops. When testing the ANN all outputs for any given input are EXACTLY the same.

I am trying to output a number between 0 and 1 (0 to classify a stock as a faller and 1 to classify a riser - 0.4-0.6 should indicate stability)

When adding the same training data into RapidMiner Studios a proper ANN with much greater (70+%) Accuracy is created, therefore I know that the dataset is fine. There must be some problem in the ANN logic.

Below is the code for running and adjusting the weights. Any and all help appreciated!

    public double[] Run(double[] inputs) {
    //INPUTS
    for (int i = 0; i < inputNeurons.length; i++) {
        inputNeurons[i] = inputs[i];
    }

    for (int i = 0; i < hiddenNeurons.length; i++) {
        hiddenNeurons[i] = 0;
    } //RESET THE HIDDEN NEURONS

    for (int e = 0; e < inputNeurons.length; e++) {
        for (int i = 0; i < hiddenNeurons.length; i++) {
            //Looping through each input neuron connected to each hidden neuron

            hiddenNeurons[i] += inputNeurons[e] * inputWeights[(e * hiddenNeurons.length) + i];
            //Summation (with the adding of neurons)  - Done by taking the sum of each (input * connection weight)
            //The more weighting a neuron has the more "important" it is in decision making
        }
    }

    for (int j = 0; j < hiddenNeurons.length; j++) {
        hiddenNeurons[j] = 1 / (1 + Math.exp(-hiddenNeurons[j]));
        //sigmoid function transforms the output into a real number between 0 and 1
    }

    //HIDDEN
    for (int i = 0; i < outputNeurons.length; i++) {
        outputNeurons[i] = 0;
    } //RESET THE OUTPUT NEURONS

    for (int e = 0; e < hiddenNeurons.length; e++) {
        for (int i = 0; i < outputNeurons.length; i++) {
            //Looping through each hidden neuron connected to each output neuron

            outputNeurons[i] += hiddenNeurons[e] * hiddenWeights[(e * outputNeurons.length) + i];
            //Summation (with the adding of neurons) as above
        }
    }

    for (int j = 0; j < outputNeurons.length; j++) {
        outputNeurons[j] = 1 / (1 + Math.exp(-outputNeurons[j])); //sigmoid function as above
    }

    double[] outputs = new double[outputNeurons.length];
    for (int j = 0; j < outputNeurons.length; j++) {
        //Places all output neuron values into an array
        outputs[j] = outputNeurons[j];
    }
    return outputs;
}

public double[] CalculateErrors(double[] targetValues) {
    //Compares the given values to the actual values
    for (int k = 0; k < outputErrors.length; k++) {
        outputErrors[k] = targetValues[k] - outputNeurons[k];
    }
    return outputErrors;
}

    public void tuneWeights() //Back Propagation
{
    // Start from the end - From output to hidden
    for (int p = 0; p < this.hiddenNeurons.length; p++)     //For all Hidden Neurons
    {
        for (int q = 0; q < this.outputNeurons.length; q++)  //For all Output Neurons
        {
            double delta = this.outputNeurons[q] * (1 - this.outputNeurons[q]) * this.outputErrors[q];
            //DELTA is the error for the output neuron q
            this.hiddenWeights[(p * outputNeurons.length) + q] += this.learningRate * delta * this.hiddenNeurons[p];
            /*Adjust the particular weight relative to the error
             *If the error is large, the weighting will be decreased
             *If the error is small, the weighting will be increased
             */
        }
    }

    // From hidden to inps -- Same as above
    for (int i = 0; i < this.inputNeurons.length; i++)       //For all Input Neurons
    {
        for (int j = 0; j < this.hiddenNeurons.length; j++)  //For all Hidden Neurons
        {
            double delta = this.hiddenNeurons[j] * (1 - this.hiddenNeurons[j]);
            double x = 0;       //We do not have output errors here so we must use extra data from Output Neurons
            for (int k = 0; k < this.outputNeurons.length; k++) {
                double outputDelta = this.outputNeurons[k] * (1 - this.outputNeurons[k]) * this.outputErrors[k];
                //We calculate the output delta again
                x = x + outputDelta * this.hiddenWeights[(j * outputNeurons.length) + k];
                //We then calculate the error based on the hidden weights (x is used to add the error values of all weights)
                delta = delta * x;
            }
            this.inputWeights[(i * hiddenNeurons.length) + j] += this.learningRate * delta * this.inputNeurons[i];
            //Adjust weight like above
        }
    }
}

Neural Network returning same output for every input

Answers (1)

Related Questions