CODEWITHSUNDEEP
numpymachine-learningneural-networkbackpropagation
Finn Williams
Finn Williams

Reputation: 151

Neural Network bad results

i am working on a neural network. I am exploring the effect of using a larger dataset for training. Currently i am getting crappy results. Any suggestions? I dont want to use any libraries other than numpy and please keep things simple. I am a GCSE student so i do not know a great deal about calculus either. To improve my network ive added: a second hidden layer, more epochs, a different activation function (reLU instead of sigmoid), more hidden nodes per layer ...but my results are still terrible!

import numpy as np

x = np.array([
    [0000],
    [0001],
    [0010],
    [0100],
    [1000],
    [0011],
    [0110],
    [1100],
    [1001],
    [1001],
    [1110],
    [1101],
    [1011],
    [1111],
    [1111],
    [1111],
    [1111]
    ])

y = np.array([
    [0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1]
    ]).T

w = np.random.random((1, 1))
w2 = np.random.random((1, 1))
w3 = np.random.random((1, 1))

for j in xrange(500000):
    a2 = 1/(1 + np.exp(-(np.dot(x, w))))
    a3 = 1/(1 + np.exp(-(np.dot(a2, w2))))
    a4 = 1/(1 + np.exp(-(np.dot(a3, w3))))
    a4delta = (y - a4) * (a4 * (1 - a4))
    a3delta = a4delta.dot(w3.T) * (a3 * (1 - a3))
    a2delta = a3delta.dot(w2.T) * (a2 * (1 - a2))
    w3 += a3.T.dot(a4delta)
    w2 += a2.T.dot(a3delta)
    w += x.T.dot(a2delta)
print(a4)

Upvotes: 2

Views: 190

Answers (1)

Andrey Lukyanenko
Andrey Lukyanenko

Reputation: 3851

I think that the main problem is in the input - x in your case is a vector with one feature; I don't think model can learn from it. Why don't you make in a vector with 4 features?

x = np.array([
[0,0,0,0],
[0,0,0,1],
[0,0,1,0],
[0,1,0,0],
[1,0,0,0],
[0,0,1,1],
[0,1,1,0],
[1,1,0,0],
[1,0,0,1],
[1,0,0,1],
[1,1,1,0],
[1,1,0,1],
[1,0,1,1],
[1,1,1,1],
[1,1,1,1],
[1,1,1,1],
[1,1,1,1]
])

Also change weights shape:

w = np.random.random((4, 4))
w2 = np.random.random((4, 4))
w3 = np.random.random((4, 1))

With these changes the net gives good results.

Upvotes: 1

Related Questions