CODEWITHSUNDEEP
machine-learningnlppytorchlstmtext-processing
vc00174
vc00174

Reputation: 27

Understanding the output of LSTM predictions

The model (from main docs) is the following:

import torch.nn as nn

class RNN(nn.Module):
    def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim):
        
        super().__init__()
        
        self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)
        self.hidden2tag = nn.Linear(hidden_dim, output_dim)
        
    def forward(self, text):
                
        embeds = self.word_embeddings(text)
        lstm_out, _ = self.lstm(embeds.view(len(text), 1, -1))
        tag_space = self.hidden2tag(lstm_out.view(len(text), -1))
        tag_scores = F.log_softmax(tag_space, dim=1)   

        return tag_scores

Parameters:

INPUT_DIM = 62288
EMBEDDING_DIM = 64
HIDDEN_DIM = 128
OUTPUT_DIM = 15

Upvotes: 0

Views: 174

Answers (1)

Theodor Peifer
Theodor Peifer

Reputation: 3496

The LSTM function in Pytorch returns not just the output of the last timestep but all outputs instead (this is useful in some cases). So in your example you seem to have exactly 100 timesteps (the amount of timesteps is just your sequence length).

But since you are doing classification you just care about the last output. You can normally get it like this:

outputs, _ = self.lstm(embeddings)
# shape: batch_size x 100 x 15
output = outputs[:, -1]    
# shape: batch_size x 1 x 15

Upvotes: 1

Related Questions