FANN doesn't train

Question

I am using FANN for function approximation. My code is here:

/*
 * File:   main.cpp
 * Author: johannsebastian
 *
 * Created on November 26, 2013, 8:50 PM
 */

#include "../FANN-2.2.0-Source/src/include/doublefann.h"
#include "../FANN-2.2.0-Source/src/include/fann_cpp.h"
//#include <doublefann>
//#include <fann/fann_cpp>
#include <cstdlib>
#include <iostream>


using namespace std;
using namespace FANN;

//Remember: fann_type is double!

int main(int argc, char** argv) {
    //create a test network: [1,2,1] MLP
    neural_net * net = new neural_net;
    const unsigned int layers[3] = {1, 2, 1};
    net->create_standard_array(3, layers);

    //net->create_standard(num_layers, num_input, num_hidden, num_output);

    //net->set_learning_rate(0.7f);

    //net->set_activation_steepness_hidden(0.7);
    //net->set_activation_steepness_output(0.7);

    net->set_activation_function_hidden(SIGMOID);
    net->set_activation_function_output(SIGMOID);
    net->set_training_algorithm(TRAIN_RPROP);

    //cout<<net->get_train_error_function()
    //exit(0);
    //test the number 2
    fann_type * testinput = new fann_type;
    *testinput = 2;
    fann_type * testoutput = new fann_type;
    *testoutput = *(net->run(testinput));
    double outputasdouble = (double) *testoutput;
    cout << "Test output: " << outputasdouble << endl;

    //make a training set of x->x^2
    training_data * squaredata = new training_data;
    squaredata->read_train_from_file("trainingdata.txt");
    //cout<<testinput[0]<<endl;
    //cout<<testoutput[0]<<endl;
    cout<<*(squaredata->get_input())[9]<<endl;
    cout<<*(squaredata->get_output())[9]<<endl;
    cout<<squaredata->length_train_data();

    //scale data
    fann_type * scaledinput = new fann_type[squaredata->length_train_data()];
    fann_type * scaledoutput = new fann_type[squaredata->length_train_data()];
    for (unsigned int i = 0; i < squaredata->length_train_data(); i++) {
            scaledinput[i] = *squaredata->get_input()[i]/200;///100;
            scaledoutput[i] = *squaredata->get_output()[i]/200;///100;
            cout<<"In:\t"<<scaledinput[i]<<"\t Out:\t"<<scaledoutput[i]<<endl;
    }

    net->train_on_data(*squaredata, 1000000, 100000, 0.001);

    *testoutput = *(net->run(testinput));
    outputasdouble = (double) *testoutput;
    cout << "Test output: " << outputasdouble << endl;

    cout << endl << "Easy!";
    return 0;
}

Here's trainingdata.txt:

10 1 1
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
10 100

When I run I get this:

Test output: 0.491454
10
100
10In:   0.005    Out:   0.005
In:     0.01     Out:   0.02
In:     0.015    Out:   0.045
In:     0.02     Out:   0.08
In:     0.025    Out:   0.125
In:     0.03     Out:   0.18
In:     0.035    Out:   0.245
In:     0.04     Out:   0.32
In:     0.045    Out:   0.405
In:     0.05     Out:   0.5
Max epochs  1000000. Desired error: 0.0010000000.
Epochs            1. Current error: 2493.7961425781. Bit fail 10.
Epochs       100000. Current error: 2457.3000488281. Bit fail 9.
Epochs       200000. Current error: 2457.3000488281. Bit fail 9.
Epochs       300000. Current error: 2457.3000488281. Bit fail 9.
Epochs       400000. Current error: 2457.3000488281. Bit fail 9.
Epochs       500000. Current error: 2457.3000488281. Bit fail 9.
Epochs       600000. Current error: 2457.3000488281. Bit fail 9.
Epochs       700000. Current error: 2457.3000488281. Bit fail 9.
Epochs       800000. Current error: 2457.3000488281. Bit fail 9.
Epochs       900000. Current error: 2457.3000488281. Bit fail 9.
Epochs      1000000. Current error: 2457.3000488281. Bit fail 9.
Test output: 1

Easy!
RUN FINISHED; exit value 0; real time: 9s; user: 10ms; system: 4s

Why is the training not working? After I asked a similar question, I was told to scale the NN's input and output. I have done so. Am I getting some parameter(s) wrong, or do I simply have to train longer?

Answer 1

Maybe a bit late, but maybe new FANN beginner will see this answer, I hope this helps !

I think your problem comes from the data format in your trainingdata.txt:

See : FANN data format

You have to do a newline after each input and each output.

In your case, you have 10 examples with 1 input and 1 output. Then, you have to format your file like this :

10 1 1
1 
1
2 
4
3 
9
4 
16
5 
25
6 
36
...

Note : I notice when the data format is wrong, the error computed by training method is very (very) high. Could be an hint to look at your file format when you see huge error value.

Answer 2

The node number in your hidden layer is too few to fit a quadratic function. I would try 10. Besides, I would like to recommend you a fun applet in which you can simulate the training process by parameter setting. I tried with 10 hidden layer nodes and unipolar sigmoid as both hidden layer and output layer activation function, the fitting is not bad (but randomize the weights may lead to the failure of converge, so more nodes in hidden layer are highly recommended, you can try to play this applet yourself and observe some interesting points):