Neural Network basic understanding and visualisation

Question

i am working with a book that is describing neural networks from the very beginning and even without using the numpy functions to multiply vectors etc.

So I have the following code:

weights = [0.1, 0.2, 0]

def w_sum(a, b):            # weighted sum function
    assert(len(a) == len(b))
    output = 0
    for i in range(len(a)):
        output += (a[i] * b[i])
    return output

def neural_network(input, weights):
    pred = w_sum(input, weights)     #the weighted sum function with values = dot product
    return pred



info1 = [8.5, 9.5, 9.9, 9.0]
info2 = [0.65, 0.8, 0.8, 0.9]
info3 = [1.2, 1.3, 0.5, 1.0]

input = [info1[0], info2[0], info3[0]]
pred = neural_network(input, weights)

print(pred)

So for this code I also have the following diagram: neural network basic structure

Now: What is the network putting into these three knots on the right side of that diagram? To me it seems like the output is the same number (in this case, because I am using input = [info1[0], info2[0], info3[0]] the number is: 0.98) ?

Why this diagram? It is given in the book. But what is actually happening, if it all outputs the same number (the weighted sum what equals to 0.85 + 0.13 + 0.0 = 0.98)

I mean I try to understand what this diagram wants to tell me or how I can read it.

I would like to know more about how that w_sum(a, b) function actually operates. I don't understand the structure of what is happening, especially what the assert function is doing.

EDIT: so after the enlightening answers I updated the diagram, how it actually should look regarding the code.

Thanks a lot and best wishes

Benjamin

Answer 1

The diagram (almost) describes the general structure of a Fully Connected Neural Network (it's missing the activation function).

What is the network putting into these three knots on the right side of that diagram?

Numbers. Information that is derived from the previous layer. What these numbers actually mean, and how the network process information, on an intuitive level, is a far more difficult question, for which you rarely get an answer (some CNNs have been broken down to show how images are being transformed through the network and what kind of features each layer extracts). It will be a lot better for you if you only try to understand how the network works, on an operation level (i.e. what mathematical operations happen in each layer), rather than why it works.

As for the implementation, the function w_sum() calculates the dot product of two vectors (lists), a and b. It's implementation is based on the mathematical definition of the dot product. Also, for the dot product to be defined, the vectors need to have an equal number of components (dimensions). To guarantee that, an assert statement is used first. If you are not familiar on what it does, read up on it here.

The confusing bit here is that the implementation does not fully reflect the diagram. For this implementation, the correct diagram would only have one node (the first) on the second layer.

As a final note, the lists info0,info1, info2 hold the values for the nodes in the first layer, at time i. So, at time 0, the input vector (the values of all nodes in the first layer) is [info1[0], info2[0], info3[0]], at time 1 it is [info1[1], info2[1], info3[1]], etc.

Answer 2

What you have got so far is a very simple neural network, which seems like a good first step to learn more advanced ones. It has no hidden layers and no activation functions. So yes, it doesn't do much at the moment, this is just the base for the upcoming features of the network.

It seems that you understand what the diagram is trying to tell you. There's nothing more to it at the moment than the weighted sum for each output.

The assert statement simply checks that the number of inputs correlates with the number of weights. All w_sum does is take the sum of every input multiplied by it's weight and returns it.

Answer 3

Ok let's start with the assert function, it is there to test if the input contains 3 numbers exaxtly since that's what the network is accepting in it's shape or else it will raise an error and the program will crash.

From the diagram it seems that you are conserned only about the first output right now and that's the only weights you have, the network in the code is the diagram without the other 2 nodes at the output just the one. by the way this is the simple Perceptron model. It trying to tell you that later on you can stack many of those mini models to create a bigger model.

The w_sum() method actually just does a weighted sum of your inputs, that's how the simple model works so it does input1 * w1 i+ input2 * w2 + input3 * w3 in this case the w's are the weights of the model and the input's are what is passed in the function