Reputation: 348
I did not understand why these multiplication outputs are different.
print(features*weights)
print('------------')
print(features*weights.view(5,1))
print('------------')
print(torch.mm(features,weights.view(5,1)))
outputs:
tensor([[ 0.1314, -0.2796, 1.1668, -0.1540, -2.8442]])
------------
tensor([[ 0.1314, -0.7035, -0.8472, 0.9971, -1.5130],
[ 0.0522, -0.2796, -0.3367, 0.3963, -0.6013],
[-0.1809, 0.9688, 1.1668, -1.3733, 2.0837],
[-0.0203, 0.1086, 0.1308, -0.1540, 0.2336],
[ 0.2469, -1.3224, -1.5927, 1.8745, -2.8442]])
------------
tensor([[-1.9796]])
Upvotes: 1
Views: 299
Reputation: 8719
If I'm not wrong, what you are trying to understand is:
features = torch.rand(1, 5)
weights = torch.Tensor([1, 2, 3, 4, 5])
print(features)
print(weights)
# Element-wise multiplication of shape (1 x 5)
# out = [f1*w1, f2*w2, f3*w3, f4*w4, f5*w5]
print(features*weights)
# weights has been reshaped to (5, 1)
# Element-wise multiplication of shape (5 x 5)
# out = [f1*w1, f2*w1, f3*w1, f4*w1, f5*w1]
# [f1*w2, f2*w2, f3*w2, f4*w2, f5*w2]
# [f1*w3, f2*w3, f3*w3, f4*w3, f5*w3]
# [f1*w4, f2*w4, f3*w4, f4*w4, f5*w4]
# [f1*w5, f2*w5, f3*w5, f4*w5, f5*w5]
print(features*weights.view(5, 1))
# Matrix-multiplication
# (1, 5) * (5, 1) -> (1, 1)
# out = [f1*w1 + f2*w2 + f3*w3 + f4*w4 + f5*w5]
print(torch.mm(features, weights.view(5, 1)))
Output:
tensor([[0.1467, 0.6925, 0.0987, 0.5244, 0.6491]]) # features
tensor([1., 2., 3., 4., 5.]) # weights
tensor([[0.1467, 1.3851, 0.2961, 2.0976, 3.2455]]) # features*weights
tensor([[0.1467, 0.6925, 0.0987, 0.5244, 0.6491],
[0.2934, 1.3851, 0.1974, 1.0488, 1.2982],
[0.4400, 2.0776, 0.2961, 1.5732, 1.9473],
[0.5867, 2.7701, 0.3947, 2.0976, 2.5964],
[0.7334, 3.4627, 0.4934, 2.6220, 3.2455]]) # features*weights.view(5,1)
tensor([[7.1709]]) # torch.mm(features, weights.view(5, 1))
Upvotes: 2
Reputation: 114976
It seems like both features
and weights
are 5-vectors.
- When simply multiplying them using *
operator you get their element-wise multiplication.
- When transposing one of them (using view()
) and then applying element-wise multiplication with *
operator, Pytorch broadcast the corresponding singleton dimensions resulting with outer-product of the two vectors: res_ij = w_i * f_j
.
- Finally, you apply matrix multiplication torch.mm
to the two vectors, resulting with their inner product.
Upvotes: 0