Pytorch + Residual Network throws unexpected: TypeError: 'NoneType' object is not callable

Question

I am diving into the world of Deep Learning and as Python is my most prominent programming language I decided to start with the Pytorch framework. Following a tutorial I implemented a ResNet of 50 layers. Afterwards I tried to make an ResNet-18 with simpler blocks which looked like this:

class baseBlock(torch.nn.Module):
  expansion = 1
  def __init__(self,input_planes,planes,stride=1,dim_change=None):
    super(baseBlock,self).__init__()
    #Declare convolutional layers with batch norms
    self.conv1 = torch.nn.Conv2d(input_planes, planes, stride=stride, kernel_size=3, padding=1)
    self.bn1   = torch.nn.BatchNorm2d(planes)
    self.conv2 = torch.nn.Conv2d(planes, planes, stride=1, kernel_size=3, padding=1)
    self.bn2   = torch.nn.BatchNorm2d(planes)
    self.dim_change = dim_change

  def forward(self,x):
    #Save the residue
    res = x
    output = F.relu(self.bn1(self.conv1(x)))
    output = self.bn2(self.conv2(output))

    if self.dim_change is not None:
      res = self.dim_change(res)

    output += res
    output = F.relu(output)

    return output

I divided the code into the blocks and the actual Neural Network to make it more understandable. The rest of the code for the NN is:

class ResNet(torch.nn.Module):
  def __init__(self,block,num_layers,classes=10):
    super(ResNet,self).__init__()
    #according to research paper
    self.input_planes = 64
    self.conv1  = torch.nn.Conv2d(3,64,kernel_size =3,stride=1,padding=1)
    self.bn1    = torch.nn.BatchNorm2d(64)
    self.layer1 = self._layer(block, 64, num_layers[0], stride=1)
    self.layer2 = self._layer(block, 128, num_layers[1], stride=2)
    self.layer3 = self._layer(block, 256, num_layers[2], stride=2)
    self.layer4 = self._layer(block, 512, num_layers[3], stride=2)
    self.avaragePool = torch.nn.AvgPool2d(kernel_size=4,stride=1)
    self.fc = torch.nn.Linear(512*block.expansion, classes)


  def _layer(self,block,planes,num_layers,stride=1):
    dim_change = None
    if stride != 1 or planes != self.input_planes*block.expansion:
      dim_change = torch.nn.Sequential(torch.nn.Conv2d(self.input_planes, planes*block.expansion, kernel_size = 1, stride=stride),
                                                        torch.nn.BatchNorm2d(planes*block.expansion))
      netLayers = []
      netLayers.append(block(self.input_planes,planes,stride=stride, dim_change=dim_change))

      self.input_planes = planes*block.expansion

      for i in range(1,num_layers):
        netLayers.append(block(self.input_planes,planes))
        self.input_planes = planes * block.expansion

      return torch.nn.Sequential(*netLayers)


  def forward(self,x):
    x = F.relu(self.bn1(self.conv1(x)))

    #Problem at this layer
    x = self.layer1(x) <- NoneType object is not callable
    x = self.layer2(x)
    x = self.layer3(x)
    x = self.layer4(x)

    x = F.avg_pool2d(x,4)
    x = x.view(x.size(0),-1)
    x = self.fc(x)

    return x

The problem arises on the last forward function in the layer1 that is marked on the code. If I comment it the network keeps on computing (obviously it throws an error later because the dimensions are not right but it still does something). If I change the stride on this first layer the problem also resolves but still throws an error later for the same dimensional reasons. I don't seem to figure out what the problem is. I use a standard CIFAR10 dataset to train it and the training code is as standard as it gets.

If I can provided any additional information let me know. Thank you

Answer 1

This is a simple typo. Your return is inside the if, and the only layer with stride==1 is the layer1:

def _layer(self,block,planes,num_layers,stride=1):
    dim_change = None
    if stride != 1 or planes != self.input_planes*block.expansion:
        # [...]
        return torch.nn.Sequential(*netLayers)

Therefore, for the layer1, there is no return, hence None.