CODEWITHSUNDEEP
computer-visionpytorchautograd
MegaNightdude
MegaNightdude

Reputation: 161

Strange behavior of Inception_v3

I am trying to create a generative network based on the pre-trained Inception_v3.

1) I fix all the weights in the model

2) create a Variable whose size is (2, 3, 299, 299)

3) create targets of size (2, 1000) that I want my final layer activations to become as close as possible to by optimizing the Variable. (I do not set the batchsize of 1, because unlike VGG16, Inception_v3 doesn't take batchsize=1, but that's not the point).

The following code should work, but gives me the error: «RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation».

# minimalist code with Inception_v3 that throws the error:

import torch
from torch.autograd import Variable
import torch.optim as optim
import torch.nn as nn
import torchvision

torch.set_default_tensor_type('torch.FloatTensor')
Iv3 = torchvision.models.inception_v3(pretrained=True)
for i in Iv3.parameters():
    i.requires_grad = False

criterion = nn.CrossEntropyLoss()

x = Variable(torch.randn(2, 3, 299, 299), requires_grad=True)
target = torch.empty(2, dtype=torch.long).random_(1000)

output = Iv3(x)
loss = criterion(output[0], target)
loss.backward()

print(x.grad)

This is very strange, because if I do the same thing with VGG16, everything works fine:

# minimalist working code with VGG16:

import torch
from torch.autograd import Variable
import torch.optim as optim
import torch.nn as nn
import torchvision

# torch.cuda.empty_cache()
# vgg16 = torchvision.models.vgg16(pretrained=True).cuda()
# torch.set_default_tensor_type('torch.cuda.FloatTensor')

torch.set_default_tensor_type('torch.FloatTensor')
vgg16 = torchvision.models.vgg16(pretrained=True)
for i in vgg16.parameters():
    i.requires_grad = False

criterion = nn.CrossEntropyLoss()

x = Variable(torch.randn(2, 3, 229, 229), requires_grad=True)
target = torch.empty(2, dtype=torch.long).random_(1000)

output = vgg16(x)
loss = criterion(output, target)
loss.backward()

print(x.grad)

Please help.

Upvotes: 0

Views: 50

Answers (1)

MegaNightdude
MegaNightdude

Reputation: 161

Thanks to @iacolippo the issue is solved. Turns out the problem was due to Pytorch 1.0.0. No problem with Pytorch 0.4.1. though.

Upvotes: 1

Related Questions