CODEWITHSUNDEEP
artificial-intelligenceneural-network
TheFisherman
TheFisherman

Reputation: 1279

Values of weights and outputs in Convolutional Neural Networks

I'm wondering what the range of values are for the weights and inputs are in a convolutional neural networks. My understanding is as follows:

If the input is a grayscale image, the input value of the first layer would be 0-255. But in the subsequent layers the input values would range from 0-1, due to the sigmoid function.

But what are the range for the weights? Is it 0-1, or can it be higher? Does this vary a lot?

Upvotes: 2

Views: 1730

Answers (1)

FrancoisBaveye
FrancoisBaveye

Reputation: 1902

Since it is using the sigmoid function, the weights usually range from -∞ to +∞ because the sigmoid of -∞ is near 0 and the sigmoid of +∞ is near 1, and you need to be capable of having near 0 and near 1 values as outputs of your neurons.

And yeah, weights can vary a lot. The more you train the NN, the more weights will get closer to their "needed" value.

For example if a neuron needs to output 0, the ideal value is -∞ because limit of the sigmoid to -∞ is 0. The more you train it, the more the weight will get closer to -∞.

Upvotes: 3

Related Questions