CODEWITHSUNDEEP
pythonpython-3.xpytorch
Qubix
Qubix

Reputation: 4353

How to use a learnable parameter in pytorch, constrained between 0 and 1?

I want to use a learnable parameter that only takes values between 0 and 1. How can I do this in pytorch?

Currently I am using:

self.beta = Parameter(torch.Tensor(1))
#initialize
zeros(self.beta)

But I am getting zeros and NaN for this parameter, as I train.

Upvotes: 6

Views: 6158

Answers (1)

Shai
Shai

Reputation: 114876

You can have a "raw" parameter taking any values, and then pass it through a sigmoid function to get a values in range (0, 1) to be used by your function.

For example:

class MyZeroOneLayer(nn.Module):
  def __init__(self):
    self.raw_beta = nn.Parameter(data=torch.Tensor(1), requires_grad=True)

  def forward(self):  # no inputs
    beta = torch.sigmoid(self.raw_beta)  # get (0,1) value
    return beta

Now you have a module with trainable parameter that is effectively in range (0,1)

Upvotes: 5

Related Questions