CODEWITHSUNDEEP
pythonpython-3.xneural-network
Parker Anderson
Parker Anderson

Reputation: 33

Neural Net (from scratch, no tensorflow) gives incorrect answers even after hours of training

my neural network is split into 2 files, the class and the one that actually creates/runs it
I believe the problem lays in the class file.
The purpose of the NN is to be an OR gate ([1, 1] = 1, [1, 0] = 1, [0, 1] = 1, [0, 0] = 0) while [1, 1],[1, 0], and [0, 1] output 1 (as they should) [0, 0] outputs .5 when I would expect something much lower

def sigmoid(x):
    return 1/(1+np.exp(-x))

def sigder(x):
    return sigmoid(x)*(1-sigmoid(x))

class NN:

    def __init__(self, tinputs, toutputs, weights, wfn, trainlen=30000, bias=.3, lr=.05):
        self.trainlen = trainlen
        self.tinputs = tinputs
        self.toutputs = toutputs
        self.bias = bias
        self.lr = lr
        self.weights = weights
        self.wfn = wfn

    def __repr__(self):
        return f"{self.weights}"

    def train(self):
        for a in range(self.trainlen):
            self.inputs = self.tinputs

            self.inp = np.dot(self.inputs, self.weights)
            self.out = sigmoid(self.inp)

            self.error = self.out - self.toutputs

            self.derror_dout = self.error
            self.dout_din = sigder(self.out)
            self.deriv = self.derror_dout * self.dout_din

            self.inputs = self.tinputs.T
            self.derivfin = np.dot(self.inputs, self.deriv)
            self.weights -= self.lr * self.derivfin

            for b in self.deriv:
                self.bias -= self.lr * b
        
        with open(self.wfn, "wb") as f:
            np.save(f, self.weights)

    def run(self, inp, fn, gen):
        point = np.dot(inp, self.weights)
        point = sigmoid(point)
        with open(fn, "a") as f:
            f.write(f"\nGen:{gen+1}\n{inp}:{point}\n\n")

        return point

any help is great, thanks

Upvotes: 2

Views: 40

Answers (1)

Frederico Möller
Frederico Möller

Reputation: 56

Basic logic gates can be a pain on ass on NN.

I did not spot amnything wrong in your code, but by the nature of your problem and what you describe, I will bet the problem is in the training data. When you have a disbalance between the positives and negatives in a data, an NN can be biased towards the positive cases and generate false-posivtives. If your training data have equal numbers of each input, your negative cases are 25%.

There are two ways to correct this, one is changing your training set, putting more cases of [0,0] OR you can give more emphasis to the false-positive error, like multipling this error by an constant before use it to adjust the nodes weights.

Upvotes: 2

Related Questions