Reputation: 4729
I want to have an RNN with input size 7, hidden size 10 and output size 2.
So for an input of, say, shape 99x1x7
I expect an output of shape 99x1x2
.
For an RNN alone, I get:
model = nn.RNN(input_size=7, hidden_size=10, num_layers=1)
output,hn=model(torch.rand(99,1,7))
print(output.shape) #torch.Size([99, 1, 10])
print(hn.shape) #torch.Size([ 1, 1, 10])
So I assume I still have to put a Linear
behind it:
model = nn.Sequential(nn.RNN(input_size=7, hidden_size=10, num_layers=1),
nn.Linear(in_features=10, out_features=2))
model(torch.rand(99,1,7))
Traceback (most recent call last):
File "train_rnn.py", line 80, in <module>
main()
File "train_rnn.py", line 25, in main
model(torch.rand(99,1,7))
File "/home/.../virtual-env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/home/.../virtual-env/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/.../virtual-env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/home/.../virtual-env/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 92, in forward
return F.linear(input, self.weight, self.bias)
File "/home/.../virtual-env/lib/python3.6/site-packages/torch/nn/functional.py", line 1404, in linear
if input.dim() == 2 and bias is not None:
AttributeError: 'tuple' object has no attribute 'dim'
I guess this is because Linear
receives the tuple that RNN.forward
yields. But how am I supposed to combine the two?
Upvotes: 2
Views: 2583
Reputation: 5290
From the pytorch doc https://pytorch.org/docs/stable/nn.html?highlight=rnn#torch.nn.RNN
the output is of shape seq_len, batch, num_directions * hidden_size
So depending on what you want you might add a fc layer to get an output of size 2.
Basically, a Sequential
will apply each model on top of the output of the next_one so you must either not use a Sequential
or create a special Linear Layer that works on sequence, the following should work :
class seq_Linear(nn.module):
def __init__(self, linear):
self.linear = linear
# To apply on every hidden state
def forward(self, x):
return torch.stack([self.linear(hs) for hs in x])
# To apply on the last hidden state
def forward(self, x):
return self.linear(x[-1])
and replace your nn.Linear by a seq_Linear(nn.Linear) in your code.
Edit : If you want to create a sequence of outputs of size 2, perhaps the best way would be to stack another RNN on top of your first one with input_size 10 and output_size 2, they should be stackable inside a Sequential
without any trouble.
Upvotes: 2