CODEWITHSUNDEEP
pythonmachine-learningpytorch
Tom S
Tom S

Reputation: 631

Pytorch normalize 2D tensor

For more robustnes of my model I want to normalize my feature tensor.

I tried doing it the way that is to the best of my knowledge standard for pictures:

class Dataset(torch.utils.data.Dataset):
'Characterizes a dataset for PyTorch'
def __init__(self, input_tensor, transform = transforms.Normalize(mean= 0.5, std=0.5)):
    self.labels = input_tensor[:,:,-1]
    self.features = input_tensor[:,:,:-1]
    self.transform = transform

def __len__(self):
    return self.labels_planned.shape[0]

def __getitem__(self, index):

    # Load data and get label
    X = self.features[index]
    y = self.labelslabels[index]
    if self.transform:
        X = self.transform(X)   
    return X, y

But receive this error message:

ValueError: Expected tensor to be a tensor image of size (C, H, W). Got tensor.size() = torch.Size([8, 25]).

Everywhere I looked people suggest that one should use .view to generate the third dimension in order to comply with the standard shape of pictures, but this seems very odd to me. Is there maybe a cleaner way to do this. Also where should I best place the normalization? Just for the batch or for the entire train dataset?

Upvotes: 0

Views: 3069

Answers (2)

Don Feto
Don Feto

Reputation: 1486

for normalizing a 2D tensor or dataset using the Normalize Transform.

@ivan solve your problem.

but here is a generalization for any 2D dataset like Wine. as Normalize in pytorch works only with images, so you need to reshape your dataset to 3 dimensions, pass it to normalize, and then reshape it to be 2 dimensions again and return it. and you have to make sure you don't pass the labels to the normalize transform. here is a code for making things more clear.

class WineDataset(Dataset):


    def __init__(self, transform=None):
        self.xy = pd.read_csv("./data/wine.csv")
        self.transform = transform
        self.x = torch.from_numpy(self.xy.iloc[:, 1:].to_numpy())
        self.y = torch.from_numpy(self.xy.iloc[:, [0]].to_numpy())
        self.n_samples = self.xy.shape[0]
        # i could make everything i needed in the dataset using 
          preprocessing of sklearn then use torch and it  would be easier 
          that waiting for transforms to fix things

    def __getitem__(self, index):

        sample = self.x[index], self.y[index]

        if self.transform:
            # you have to pass nd.array the dataset. (should put condition for 1 sample retrieval but since you will use batch you don't have to worry about it)
            features = self.transform(sample[0].reshape(1, *sample[0].shape))
            labels = sample[1]
            sample = features.squeeze_(0), labels
        return sample

    def __len__(self):
        return self.n_samples
        # len(dataset)


transform = torchvision.transforms.Compose([
    torchvision.transforms.Normalize(mean=0.5, std=0.5)
])

# before using transform
dataset = WineDataset(transform=None)

first = dataset[0:5]

feature, labels = first

print(feature, labels)

print()
print()
# after using transform
dataset = WineDataset(transform=transform)

first = dataset[0:5]

feature, labels = first

print(feature, labels)

reference Link for wine dataset: https://archive.ics.uci.edu/dataset/109/wine

Upvotes: 0

Ivan
Ivan

Reputation: 40648

You are asking two different questions, I will try to answer both.

  • Indeed, you should first reshape to (c, h, w) where c is the channel dimension In most cases, you will need that extra dimension because most 'image' layers are built to receive 3d dimensional tensors - not counting the batch dimension - such as nn.Conv2d, BatchNorm2d, etc... I don't believe there's anyways around it, and doing so would restrict yourself to one-layer image datasets.

    You can broadcast to the desired shape with torch.reshape or Tensor.view:

    X = X.reshape(1, *X.shape)

    Or by adding an additional dimension using torch.unsqueeeze:

    X.unsqueeze(0)

  • About normalization. Batch-normalization and dataset-normalization are two different approaches.

    The former is a technique that can achieve improved performance in convolution networks. This kind of operation can be implemented using a nn.BatchNorm2d layer and is done using learnable parameters: a scale factor (~ std) and a bias (~ mean). This type of normalization is applied when the model is called and is applied per-batch.

    The latter is a pre-processing technique which allows making different features have the same scale. This normalization can be applied inside the dataset per-element. It requires you measure the mean and standard deviation of your training set.

Upvotes: 4

Related Questions