Reputation: 29
I'm trying to use a gradient method to find a minimizer of a function f
, where x(k+1)=x(k)-α▽f(x)
, and α=0.1
. The function f
is f(x) = 2*(x^3+1)^(-1/2)
, x(0)=1
Here is the pytorch sample
import torch
import torch.nn as nn
from torch import optim
alpha = 0.1
class MyModel(nn.Module):
def __init__(self):
super().__init__()
self.x = 1
def forward(self, x):
return 2*pow((pow(x,3)+1),(-1/2))
model = MyModel()
optimizer = optim.SGD(model.parameters(),lr=alpha)
terminationCond = False
while not terminationCond:
f = model.forward(x)
f.backward()
optimizer.step()
if x.grad < 0.001:
terminationCond = True
optimizer.zero_grad()
But I cannot output the correct value of x
, how to modify my code in order to find a minizer of a function f
?
Upvotes: 1
Views: 724
Reputation: 1320
There are a few things that need to be considered ...
x
to be a parameter (model.parameter()
) x should be nn.Parameter(torch.as_tensor([1.]))
x
to the forward method?f(x) = 2*(x^3+1)^(-1/2)
is inversely proportional to x. When x goes up in value, the function goes down. The SGD will minimize the value of the function, hence maximize x.f(0) = 2.0 f(1) = 1.41421 f(2) = 0.66666
Here is a working example of minimizing f(x) ie maximizing x.
import torch
import torch.nn as nn
from torch import optim
alpha = 0.1
class MyModel(nn.Module):
def __init__(self):
super().__init__()
self.x = nn.Parameter(torch.as_tensor([1.0]))
def forward(self):
return 2*pow((pow(self.x, 3)+1), (-1/2))
model = MyModel()
optimizer = optim.SGD(model.parameters(), lr=alpha)
terminationCond = False
print(model.x)
f = model()
f.backward()
optimizer.step()
optimizer.zero_grad()
print(model.x)
Upvotes: 1