CODEWITHSUNDEEP
pythonpytorch
cross_knight
cross_knight

Reputation: 315

How to change PyTorch tensor into a half size and/or double size with different dimension?

I'm new to PyTorch and tensor data thing. I have a problem about switching shape of tensors.

I have two questions.

First, what should I do if I have a tensor with torch.Size([8, 512, 16, 16]) and I want to change it into torch.Size([8, 256, 32, 32]) which is double size of the original tensor.

Second, what should I do if I have a tensor with torch.Size([8, 256, 32, 32]) and I want to change it into torch.Size([8, 512, 16, 16]) which is half size of the original tensor.

In the first question, I've tried on ZeroPadding2D(8) function to reshape it into torch.Size([8, 512, 32, 32]) but I don't know how to change the 2nd dimension which is 512 into 256.

The actual usage in the first question is something like this.

x = input # torch.Size([8, 512, 16, 16])
x = layer(x) # torch.Size([8, 256, 32, 32]
x = x + input # what I want to do is adding tensor values before and after passing the layer together (like skip connection)

I expect the output of adding two tensors to be success but the actual output is an error about unequal size in dimensions

Upvotes: 2

Views: 10039

Answers (1)

Anubhav Singh
Anubhav Singh

Reputation: 8699

  • For the first case, use resize_() to change second dimension from 512 to 256 and then allocate a tensor with your padding value and the target dimensions and assign the portion for which you have data.

    import torch

target_output = torch.zeros(8, 256, 32, 32)

in_tensor = torch.randn(8, 512, 16, 16)
out_temp = in_tensor.resize_((8, 256, 16, 16))

target_output[:, :, :16, :16] = out_temp
print(target_output.shape)

# output:
# torch.Size([8, 256, 32, 32])

    You can also use torch.nn.ConstantPad2d() and then resize_() as below:

    in_tensor = torch.randn(8, 512, 16, 16)
m = nn.ConstantPad2d((8, 8, 8, 8), 0)
out_tensor = m(in_tensor).resize_(8, 256, 16, 16)
print(out_tensor.shape) 

# output:
# torch.Size([8, 256, 32, 32])

    Alternatively, you can also use torch.nn.ConstantPad2d() and copy_() like below:

    import torch.nn as nn
in_tensor = torch.randn(8, 512, 16, 16)  # shape [8, 512, 16, 16]
m = nn.ConstantPad2d((8, 8, 8, 8), 0) 
temp = m(in_tensor)                      # shape [8, 512, 32, 32]
out_tensor = torch.zeros(8, 256, 32, 32) # shape [8, 256, 32, 32]
out_tensor = out_tensor[:,:,:,:].copy_(temp[:,:256,:,:]) # shape [8, 256, 32, 32]

    You can read more about reshaping a tensor with padding in pytorch from here.

  • For the second case, you can simply use resize_() for resizing your tensor to half the size.

    in_tensor = torch.randn(8, 256, 32, 32)
out_tensor = in_tensor.resize_(8, 512, 16, 16)
print(out_tensor.shape)

# output:
# torch.Size([8, 512, 16, 16])

    Alternatively, you can use copy_ as below:

    in_tensor = torch.randn(8, 256, 32, 32)
temp_tensor = torch.zeros(8, 512, 16, 16) # shape [8, 512, 16, 16]
temp_tensor[:,:256,:,:].copy_(in_tensor[:,:,:16,:16]) # shape [8, 512, 16, 16]
out_tensor = temp_tensor # shape [8, 512, 16, 16]

    Without using copy_():

    in_tensor = torch.randn(8, 256, 32, 32)
out_tensor = torch.zeros(8, 512, 16, 16) # shape [8, 512, 16, 16]
out_tensor[:,:256,:,:] = in_tensor[:,:,:16,:16]

Upvotes: 1

Related Questions