CODEWITHSUNDEEP
tensorflowpytorchconv-neural-network
Mustansar Saeed
Mustansar Saeed

Reputation: 2790

Conv2D padding in TensorFlow and PyTorch

I am trying to convert TensorFlow model to PyTorch but having trouble with padding. My code for for relevant platforms are as follow:

TensorFlow

conv1 = tf.layers.conv2d(
                inputs=input_layer,
                filters=32,
                kernel_size=[5, 5],
                padding="same",
                activation=tf.nn.relu,
                name = "conv1")

PyTorch

conv1 = nn.Conv2d(1, 32, kernel_size=5, stride=1, padding=2)

I have few questions:

  1. Are the above codes equivalent?
  2. How many paddings are added on left/right/top/bottom if we use same padding in tensorflow?
  3. How many paddings are added on left/right/top/bottom if we use padding=2 in pytorch?
  4. If the above two code snippets are not equivalent then how can we make the same conv layer?

Thanks in advance.

Upvotes: 0

Views: 1166

Answers (1)

Tanishq Gautam
Tanishq Gautam

Reputation: 21

To answer your questions:

The reason why Pytorch doesn't have padding = 'same' to quite simply put it is due to its dynamic computation graph in comparison to Tensorflow static graph.

  1. Both the codes are not equivalent as different padding is used.

  2. 'Same' padding tries to pad evenly on the left and right, but if the amount of columns to be added is odd, it will then add an extra column to the right.

  3. 'Padding = 2' in Pytorch applies 2 implicit paddings on either side.

  4. Pytorch 1.9 has added padding = 'same' for un-strided or stride = 1 convolutions. Which will work for your use case.

But for stride > 2 padding needs to be added manually.

Here is the good implementation to perform 'same' padding:-

https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/layers/padding.py#L28

Upvotes: 2

Related Questions