CODEWITHSUNDEEP
neural-networktorchconv-neural-networkresnet
Khawar Islam
Khawar Islam

Reputation: 2924

Difference between resNet conv1 and traditional conv2d layer

I am using one layer to extract features from image. The old layer is

self.conv = nn.Conv2d(3, 64, kernel_size=3, stride=2, padding=1, bias=False)

New layer is

resConv = models.resnet50(pretrained=True)
resConv.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=2, padding=1, bias=False)
self.conv = resConv.conv1

Any performance difference? or both layers are same.

Upvotes: 0

Views: 1358

Answers (2)

Mohammad Javad
Mohammad Javad

Reputation: 167

The main difference here is, in self.Conv, weights are default (for example zero) and need to train, but in resConv.Conv1 which uses a pre-trained model, weights are tuned because it is trained with large datasets before.

performance result depends on your task but in general, it helps to get a better result (better local optimal) or need fewer epoch to train.

Upvotes: 0

David
David

Reputation: 341

Almost, there are a few differences however. According to the paper https://arxiv.org/abs/1512.03385, resnet50's conv1 layer (as you would find in torchvision.models.resnet50()) has

conv1 = {Conv2d} Conv2d(3, 64, kernel_size=(7,7), stride=(2,2), padding=(3,3), bias=False).

So the differences are
a) the kernel_size is 7x7 not 3x3
b) padding of 3x3 not 1x1 and
c) the weights from resnet50.conv1 from the pretrained model will be conditioned on the training and not initialized as random normal as the will be for nn.Conv2d(3, 64, kernel_size=3, stride=2, padding=1, bias=False)

Upvotes: 1

Related Questions