CODEWITHSUNDEEP
pythondeep-learningpytorch
Dennis Gross
Dennis Gross

Reputation: 3

How to calculate a Forward Pass with matrix operations in PyTorch?

I have got an input x, layer 1 weight matrix, and layer 2 weight matrix.

Now I want to calculate the output of this pre-trained neural network via hand:

x * weights1 * weights2

While doing this I receive a RuntimeError: The size of tensor a (6) must match the size of tensor b (4) at non-singleton dimension 1

class Net(nn.Module):

def __init__(self):
    super().__init__()
    self.fc1 = nn.Linear(4,6)
    self.fc2 = nn.Linear(6,2)
    self.fc3 = nn.Linear(2,1)

def forward(self, x):
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = F.relu(self.fc3(x))
    return x

net = Net()
X = torch.randn(1000,4)
net.fc2.weight*(net.fc1.weight * X[0])

Upvotes: 0

Views: 858

Answers (1)

Shai
Shai

Reputation: 114976

You are confusing element-wise multiplication (* operator) with matrix multiplication (@ operator).
Try:

net.fc2.weight @ (net.fc1.weight @ X[0])

Upvotes: 2

Related Questions