CODEWITHSUNDEEP
pytorchconv-neural-networkpython-3.8torchconv1d
FAZ
FAZ

Reputation: 31

Torch Conv 1-D Layer outputs the same value repeated

I have a torch tensors of shape (768, 8, 22), procesed by a Conv Layer 1-D Layer. The data is already standarized. The tensor is processed by the next layer:

nn.Conv1d(input_chanels = 8, output_chanels = 1, kernel_size = 3)

The output of the layer has the right dimensions, but the output matrix has the same value repeated over and over again.

output = tensor(
          
         [[[-0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856,
          -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856,
          -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856]],
          ...
          [[-0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856,
          -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856,
          -0.1856, -0.1856, -0.1856, -0.1856, -0.1856, -0.1856]]]

Upvotes: 1

Views: 107

Answers (1)

user3413723
user3413723

Reputation: 12273

I had this problem because my inputs were all really small. So to fix - either normalize input with Batch/Layer or other way. Alternatively, you can get rid of the bias term. Mathematically, the conv layer adds up the items you input after multiplying them by some number between 0 and 1 (at the start the numbers are initialized to between 0 and 1 in torch). Then what happens is the bias is added. But if your input is 1e-20, obviously the multiplication doesn't do anything, and the only thing that affects the output is the bias. So basically the bias is the only number you see outputted potentially. So check your inputs to see if they are too small and normalize.

Upvotes: 1

Related Questions