CODEWITHSUNDEEP
deep-learningpytorchlstmrecurrent-neural-network
user2323232
user2323232

Reputation: 249

Creating LSTM model with pytorch

I'm quite new to using LSTM in Pytorch, I'm trying to create a model that gets a tensor of size 42 and a sequence of 62.(so 62 tensor a of size 42 each). Which means that I have 62 tensors in a sequence. Each tensor is of size 42.(shape is [62,42]. Call this input tensor.

and I want to predict a tensor of 1 with a sequence of 8 (so size 1 tensor and 8 sequences) using this. Which means that there are 8 tensors in a sequence of size 1 each. Call this label tensor.

The connection between those tensors is this: Input tensor is made of columns: A1 A2 A3 ...... A42 While label tensor if more like: A3

What I’m trying to show is that if needed label tensor can be padded with zero in all places instead of the value of A3, so it can reach a length of 42.

How can I do this? since from what I'm reading from the Pytorch documentation I can only predict in the same ratio(1 point predict 1), while I want to predict from tensor of 42 with a sequence of 62 a tensor of 1 and sequence of 8. Is it doable? Do I need to pad the predicted tensor to size 42 from 1? Thanks!

a good solution will be using seq2seq for example

Upvotes: 5

Views: 1512

Answers (1)

hola
hola

Reputation: 612

If I correctly understand your question, given a sequence of length 62 you want to predict a sequence of length 8, in the sense that the order of your outputs have an importance, this is the case if you are doing some time series forcasting). In that case using a seq2seq model will be a good choice, here is a tutorial for this link. Globbaly, you nedd to implement an encoder and a decoder, here is en exemple of such an implemtation:

class EncoderRNN(nn.Module):
    def __init__(self, input_dim=42, hidden_dim=100):
        super(EncoderRNN, self).__init__()
        self.hidden_size = hidden_size

        self.lstm = nn.LSTM(input_dim, hidden_dim)

    def forward(self, input, hidden):
        output, hidden = self.lstm(input, hidden)
        return output, hidden

    def initHidden(self):
        return torch.zeros(1, 1, self.hidden_size, device=device)


class DecoderRNN(nn.Module):
    def __init__(self, hidden_dim, output_dim):
        super(DecoderRNN, self).__init__()
        self.hidden_dim = hidden_dim

        self.lstm = nn.LSTM(hidden_dim, hidden_dim)
        self.out = nn.Linear(hidden_dim, output_dim)
        self.softmax = nn.LogSoftmax(dim=1)

   def forward(self, input, hidden):
        output, hidden = self.lstm(input, hidden)
        output = self.softmax(self.out(output[0]))
        return output, hidden

   def initHidden(self):
        return torch.zeros(1, 1, self.hidden_size, device=device)

If your the order of your 8 outputs has no importance, then you can simply add a Linear layer with 8 units after the LSTM layer. You can use this code directly in that case

  class Net(nn.Module):
      def __init__(self, hidden_dim=100, input_dim=42, output_size=8):
          super(Net, self).__init__()
          self.hidden_dim = hidden_dim

          self.lstm = nn.LSTM(input_dim, hidden_dim, batch_first=True)

          # The linear layer that maps from hidden state space to tag space
          self.fc = nn.Linear(hidden_dim, output_size_size)

      def forward(self, seq):
          lstm_out, _ = self.lstm(seq)
          output = self.fc(lstm_out)
          return output

Upvotes: 3

Related Questions