CODEWITHSUNDEEP
machine-learningneural-network
MagoNick
MagoNick

Reputation: 361

Neural Network System Identification

I am trying to implement a Neural Network to identify a Nonlinear System. I have implemented a very simple system in simulink and on the basis of examples of its input and output I would like to have the NN to mimic its behaviour. The system is the followingenter image description here

The schema on the top (the one with the random number as inputs) generates the training set, while the second one (with the ramp as input) is used to check whether the NN behaves properly. The network is a two layer network with a hidden layer of tanh nodes and a linear output layer. I have implemented two versions of the network, one in which stochastic gradient descent is used (update at each step) and another one in which the update is performed in batch. At the moment I am focusing on the iterative solution. I have also implemented the train-validate-test procedure described in this post.

Here the parameters of the network:

I use the 80% of the example set to train the NN, 10% to compute the error and the remaining 10% to produce the output you can see below. However, I cannot make the network generalize the system for different types of inputs.

output during the last training cycle

test against remaining samples and different input

The first two plots show the training phase. The one on the left is the output at the last iteration of the training procedure, the one on the right the plot of the mean value of the error at the end of each training epoch. The last two plots shows the performance of the network on unseen data. The plot on the right is obtained using as input data from the examples set not used in any way for training. The one on the right is generated using a ramp as input and comparing it with the output of the system for the same input.

What am I doing wrong? The full code can be found here

Regards

Andrea

Upvotes: 3

Views: 535

Answers (1)

MagoNick
MagoNick

Reputation: 361

It turned out that the problem was a normalization problem. In fact, I trained the Neural Network on a reference signal that was preprocessed and an output signal obtained with an unnormalized signal. This causes problems in case of nonlinear systems. Now I generate training outputs using a normalized inputs and everything works fine.

If anyone need more details, feel free to contact me

regards Andrea

Upvotes: 2

Related Questions