CODEWITHSUNDEEP
neural-network
Jadager
Jadager

Reputation: 69

Connections in an Artificial Neural Network

If I have an artificial neural network with any amount of layers and the last (exit) node only accepts 3 inputs (1,x1,x2) what do I do if the hidden layer before has 4 nodes, meaning it creates 4 connections to the last node. Do I just ignore 1? Or do I have to force the last layer to have 3 nodes?

I'm having a hard time dealing with this also with the case if the last hidden layer has only 2 or 1 nodes.

I know this isn't really a programming question but I'm not having a hard time programming this (so far) just dealing with those problems.

So yeah, the idea is that the user can tell me the amount of hidden layers and amount of nodes per layer.

Upvotes: 2

Views: 313

Answers (1)

Monkey
Monkey

Reputation: 1866

The input which is constant is called bias. Assuming you use a typical function for the neuron (sigmoid of weighted sum of the inputs) If one neuron is not connected to the bias input, then that neuron can only output 0 if input is 0... Thus you loose the universal function approximation capability of a feed-forward neural network. To avoid that, 2 ways

  • neural network got input (x1, x2). Each neuron got its own bias value, thus the neurons compute sigmoid(w1 * x1 + w2 * x2 + ... + wn * xn + bias)

  • input is (x1, x2, 1). The neurons simply compute sigmoid(w1 * x1 + w2 * x2 + ... + wn * xn). Each layer is fully connected to the previous one

    for i in [1, nb_layer - 1]:
  for neuron A in neuron_layer[i]:
    for neuron B in neuron_layer[i+1]:
      connect(A, B)

I tend to prefer the 2nd solution, as it makes the code and the data more streamlined, no special case, bias is seen a special input, a bit like ground voltage for an electric circuit.

Upvotes: 2

Related Questions