CODEWITHSUNDEEP
neural-networkclassificationmatlab
Peter
Peter

Reputation: 806

Matlab Neural Network gives unexpected results

I was toying around with the matlab neural network toolbox and I encountered some things I did not expect. My problem is especially with a classification network with no hidden layers, only 1 input and the tansig transfer function. So I expect this classifier to divide a 1D dataset at some point that is defined by the learned input weight and bias.

First of all, I thought that the formula for computing the output y for a given input x is: y = f(x*w + b) with w the input weight and b the bias. What is the correct formula for calculating the output of the network?

I also expected that translating the whole dataset by a certain value (+77) would have a big effect on the bias and/or weight. But this doesn't seem to be the case. Why does the translation of the dataset has not much effect on the bias and weight?

This is my code:

% Generate data
p1 = randn(1, 1000);
t1(1:1000) = 1;
p2 = 3 + randn(1, 1000);
t2(1:1000) = -1;
% view data
figure, hist([p1', p2'], 100)

P = [p1 p2];
T = [t1 t2];

% train network without hidden layer
net1 = newff(P, T, [], {'tansig'}, 'trainlm');
[net1,tr] = train(net1, P, T);

% display weight and bias
w = net1.IW{1,1};
b = net1.b{1,1};
disp(w) % -6.8971
disp(b) % -0.2280

% make label decision
class_correct = 0;
outputs = net1(P);
for i = 1:length(outputs)
    % choose between -1 and 1
    if outputs(i) > 0
       outputs(i) = 1;
    else
       outputs(i) = -1;
    end
    % compare
    if T(i) == outputs(i)
        class_correct = class_correct + 1;
    end
end
% Calculate the correct classification rate (CCR)
CCR = (class_correct * 100) / length(outputs);
fprintf('CCR: %f \n', CCR);
% plot the errors
errors = gsubtract(T, outputs);
figure, plot(errors)

% I expect these to be equal and near 1
net1(0)              % 0.9521
tansig(0*w + b)     % -0.4680

% I expect these to be equal and near -1
net1(4)              % -0.9991
tansig(4*w + b)     % -1




% translate the dataset by +77
P2 = P + 77;

% train network without hidden layer
net2 = newff(P2, T, [], {'tansig'}, 'trainlm');
[net2,tr] = train(net2, P2, T);

% display weight and bias
w2 = net2.IW{1,1};
b2 = net2.b{1,1};
disp(w2) % -5.1132
disp(b2) % -0.1556

I generated an artificial dataset that is made of 2 populations with a normal distribution with a different mean. I plotted these populations in a histogram, and trained the network with it. I calculate the Correct classification rate, which is the percentage of the correct classified instances of the whole dataset. This is somewhere around 92% so I know the classifier works.

But, I expected net1(x) and tansig(x*w + b) to give the same output, this is not the case. What is the correct formula for calculating the output of my trained network?

And I expected net1 and net2 to have different weights and/or bias because net2 is trained on a translated version (+77) of the dataset where net1 is trained on. Why does the translation of the dataset has not much effect on the bias and weight?

Upvotes: 1

Views: 1155

Answers (1)

COOLZXxX
COOLZXxX

Reputation: 356

First off, your code leaves the default MATLAB input pre-processing intact. You can check this with:

net2.inputs{1}

When I put your code in I got this:

    Neural Network Input
              name: 'Input'
    feedbackOutput: []
       processFcns: {'fixunknowns', removeconstantrows,
                    mapminmax}
     processParams: {1x3 cell array of 2 params}
   processSettings: {1x3 cell array of 3 settings}
    processedRange: [1x2 double]
     processedSize: 1
             range: [1x2 double]
              size: 1
          userdata: (your custom info)

The important part being processFncs set to mapminmax. According to the docs, mapminmax will "Process matrices by mapping row minimum and maximum values to [-1 1]". That's why your "shifting" (re-sampling) your inputs arbitrarily had no effect.

I assume that by "calculating the output" you mean checking the performance of your network. You can do that by first simulating your network on a dataset (see doc nnet/sim) and then checking the "correctness" with the perform function. It will use the same cost function as you did when training:

% get predictions
[Y] = sim(net2,T);
% see how well we did
perf = perform(net2,T,Y);

Boomshuckalucka. Hope that helps.

Upvotes: 2

Related Questions