CODEWITHSUNDEEP
machine-learningneural-networkdeep-learningcaffe
Shai
Shai

Reputation: 114786

What is "Parameter" layer in caffe?

Recently I came across "Parameter" layer in caffe.
It seems like this layer exposes its internal parameter blob to "top".

What is this layer using for?
Can you give a usage example?

Upvotes: 2

Views: 1753

Answers (1)

hbaderts
hbaderts

Reputation: 14316

This layer was introduced in the pull request #2079, with the following description:

This layer simply holds a parameter blob of user-defined shape, and shares it as its single top.

which is exactly what you expected. This was introduced in context of the issue #1474, which basically proposes to treat parameters like normal bottom blobs. To show why this can be useful, consider the following example (taken from issue #1474, by @longjon):

The inner product layer calculates C = A * B, where C is the top blob (output), B is a bottom blob, and A must be a parameter blob. This is very restrictive, as this makes it impossible to use the inner product layer for multiplying the inner product between two bottom blobs, e.g. multiplying two input matrices. The issue #1474 suggests to make a fundamental change: make parameters independent of the layer. Instead, treat them like a normal bottom blob.

As a first step in that direction, the Parameter layer was introduced. This allows you to define a new parameter, which you can then feed into the bottom of another layer. The counterpart - a method for feeding parameters into a layer as a bottom blob - is proposed in pull request #2166, which isn't merged yet, as of Jan. 2017. Though this is not merged yet, you can still use Parameter layer to define new learnable parameters to feed into other layers as bottom blob.

Upvotes: 4

Related Questions