CODEWITHSUNDEEP
pythonconvolutiondeep-learningtensorflowconv-neural-network
user2576346
user2576346

Reputation: 115

Difference between local and dense layers in CNNs

What is the difference between a "Local" layer and a "Dense" layer in a convolutional neural network? I am trying to understand the CIFAR-10 code in TensorFlow, and I see it uses "Local" layers instead of regular dense layers. Is there any class in TF that supports implementing "Local" layers?

Upvotes: 6

Views: 2205

Answers (2)

Dung Thai
Dung Thai

Reputation: 128

Quoting from cuda-convnet:

Locally-connected layer with unshared-weight: This kind of layer is just like a convolutional layer, but without any weight-sharing. That is to say, a different set of filters is applied at every (x, y) location in the input image. Aside from that, it behaves exactly as a convolutional layer.

In the TensorFlow CIFAR-10 example, although the two layers are named local3 and local4, they are actually fully-connected layer, not locally-connected layer as specified in cuda-convnet (you can see that the output from pool2 is flattened into the input of local3 layer).

Upvotes: 5

Indie AI
Indie AI

Reputation: 601

I am quoting user2576346's comments under the question:

As I understand, either it should be densely connected or be a convolutional layer ...

No this is not true. A more accurate way to phrase that statement would be that layers are either fully connected (dense) or locally connected.

A convolutional layer is an example of a locally connected layer. In general a locally connected layer is a layer in which each of its units is only connected to a local portion of the input. A convolutional layer is a special type of local layer which exhibits a spatial translation invariance as each convolutional feature detector is strided across the entire image in local receptive windows, e.g. of size 3x3 or 5x5 for example.

Upvotes: 4

Related Questions