CODEWITHSUNDEEP
juliaflux.jl
logankilpatrick
logankilpatrick

Reputation: 14521

How to initialize weights in Flux.jl

I want to initialize some weights and bias for a specific layer in my model. How is this possible in Flux.jl?

Upvotes: 0

Views: 622

Answers (1)

logankilpatrick
logankilpatrick

Reputation: 14521

Most of the layer functions such as Dense and Conv take in the weight and bias. You can see the function definition of each by doing ? Dense or ? Conv which would reveal that for example, the Dense function can be simple called with weights and bars by doing: Dense(W::AbstractMatrix, [bias, σ]) and for the Conv function, something like this:

  julia> weight = rand(3, 4, 5);
  
  julia> bias = zeros(5);
  
  julia> c1 = Conv(weight, bias, sigmoid)  # expects 1 spatial dimension
  Conv((3,), 4=>5, σ)

where sigmoid is syntactical sugar for the sigmoid function, often denoted as σ().

Upvotes: 3

Related Questions