CODEWITHSUNDEEP
neural-network
user3689479
user3689479

Reputation: 21

Neural Network POC limited output range

To keep this as brief as possible, I have implemented a Neural Network in Java that originally was written as just an example of XOR.

This works great, so I moved on to a proof of concept on loading up the input's/output's of the NN. I've changed the algorithm to take in binary and read what number its suppose to be seeing. with 2 bits, or 3 bits as the input, I can get great results with the expected output. 

-- --> 0.000
-O --> 1.000
O- --> 2.000
OO --> 3.000

and

--- --> 0.000
--O --> 1.000
-O- --> 2.000
-OO --> 3.000
O-- --> 4.000
O-O --> 5.000
OO- --> 6.000
OOO --> 7.000

Now, once I increase the bits input / output expected number to 4, I begin to run into problems.

---- --> 0.985
---O --> 0.985
--O- --> 3.160
--OO --> 3.160
-O-- --> 4.990
-O-O --> 4.990
-OO- --> 6.747
-OOO --> 6.747
O--- --> 8.094
O--O --> 8.094
O-O- --> 9.696
O-OO --> 9.696
OO-- --> 12.531
OO-O --> 12.531
OOO- --> 14.395
OOOO --> 14.395

What is the conceptual reason for this to occur? I have tried all types of hidden layer sizes and layer numbers with no difference in results.

I will output one more larger result. 

----- --> 2.515
----O --> 2.515
---O- --> 2.515
---OO --> 2.515
--O-- --> 6.598
--O-O --> 6.598
--OO- --> 6.598
--OOO --> 6.598
-O--- --> 10.241
-O--O --> 10.241
-O-O- --> 10.241
-O-OO --> 10.241
-OO-- --> 14.364
-OO-O --> 14.364
-OOO- --> 14.364
-OOOO --> 14.364
O---- --> 18.190
O---O --> 18.190
O--O- --> 18.190
O--OO --> 18.190
O-O-- --> 22.319
O-O-O --> 22.319
O-OO- --> 22.319
O-OOO --> 22.319
OO--- --> 26.824
OO--O --> 26.824
OO-O- --> 26.824
OO-OO --> 26.824
OOO-- --> 30.966
OOO-O --> 30.966
OOOO- --> 30.966
OOOOO --> 30.966

As you can see, its possible to calculate the apparent "bunching" going on.

Thanks for any help!

Upvotes: 0

Views: 75

Answers (1)

user3689479
user3689479

Reputation: 21

Well, the answer came down to the number of bits in my response. Long story short, if you are using 1 output neuron for an integer expected output, the data type's number of bits is the output range...

The fix is output neuron return type's number of bits (Float 32, Double 64) is the range you should not exceed.

For my training set's expected results, I just ran a conversion of numberYouWant/maxNumberTheResultYouWantTheNetworkToBe * outputsDataSize

Basically, get it to count to 32 or 64 really well, and make any number system fit into that.

Thanks for no help :)

Upvotes: 1

Related Questions