CODEWITHSUNDEEP
python-3.xnumpyneural-networkdeep-learning
user7362766
user7362766

Reputation:

How to update weights and biases

I am trying to write a neural-network with numpy from scratch for recognizing hand_written_digits.but I'm little bit confused in updating weights and biases

here is my code

class NeuralNetwork():

    learning_rate = 0.0001
    ephochs = 10000
    nodes_in_input_layer = 784  # features
    nodes_in_hidden_layer = 100
    nodes_in_output_layer = 10  # classes
    np.random.seed(3)


    def __init__(self):
        self.hidden_layer = {'weights': np.random.rand(self.nodes_in_input_layer, self.nodes_in_hidden_layer)*0.1,
                             'biases': np.random.rand(self.nodes_in_hidden_layer)*0.1 }

        self.output_layer = {'weights': np.random.rand(self.nodes_in_hidden_layer, self.nodes_in_output_layer)*0.1,
                             'biases': np.random.rand(self.nodes_in_output_layer)*0.1 }

        print('self.hidden_layer: ',self.hidden_layer['weights'].shape)
        print('self.output_layer: ',self.output_layer['weights'].shape)

    def fit(self, x, y, ephochs= ephochs):
        for i in range(ephochs):
            # feed forword
            z_hidden_layer = np.dot(x[i], self.hidden_layer['weights']) + self.hidden_layer['biases']
            o_hidden_layer = sigmoid(z_hidden_layer)

            z_output_layer = np.dot(o_hidden_layer, self.output_layer['weights']) + self.output_layer['biases']
            o_output_layer = sigmoid(z_output_layer)

            # back propagation
            error = o_output_layer - y[i]

            '''
            ## at output layer 

            derror_dweights = derror/do * do/dz * dz/dw

            derror/do = error
            do/dz = derivative of sigmoid(x[i]) 
            dz/dw = o_hidden_layer
            ''' 
            derror_do = error 
            do_dz =  sigmoid(z_output_layer, derivative=True)
            dz_dw =  o_hidden_layer

            nw_output_layer = derror_do * do_dz
            nw_output_layer = np.dot(nw_output_layer, dz_dw.T) 

            nb_output_layer = error

            # updating new weights and biases
            self.output_layer['weights'] = self.output_layer['weights'] - (self.learning_rate * nw_output_layer)
            self.output_layer['biases'] = self.output_layer['biases'] - (self.learning_rate * nb_output_layer)

            ## update remain weights and biases

while I'm running i got this error

nw_output_layer = np.dot(nw_output_layer, dz_dw.T) ValueError: shapes (10,) and (100,) not aligned: 10 (dim 0) != 100 (dim 0)

can anyone explain the process of updating weights and biases of this neural-network in step-by-step?

Upvotes: 0

Views: 1192

Answers (1)

Rinshan Kolayil
Rinshan Kolayil

Reputation: 1139

In neural network, you have to preprocess your data into same shape. You can try to preprocess data inside function. So that, you have to simply call the same function to preprocess the test data. This will prevent getting diffetent shape for differemnt data.

The weights are updated to measure hyperplanes for separating data. For example,

In binary classification you fill find line with slope for separating two classes with line equation Y = MX + B. The same way you need to separate multidimensional data with hyperplane by gradient descent algorithm.

W = W - learning rate * delta

Here the delta is partial derivatives of loss. By updating weights you can reduce loss. At a certain point the loss will reach some local minima. At this point (local minima) stop the epochs (number of iterations to find best accuracy).

For updating weights you can use for loop with some batch size. In neural network you have to find learning rate and epochs that suits your data. If you are using very less learning rate, it will slow down your training.

Upvotes: 1

Related Questions