CODEWITHSUNDEEP
pythonkerasnlpdeep-learninglstm
oren_isp
oren_isp

Reputation: 779

Keras LSTM model get probabilities of labels

I created a keras LSTM model to predict the next word given a sentence:

pretrained_weights = w2v_model.wv.syn0
vocab_size, emdedding_size = pretrained_weights.shape

lstm_model = Sequential()
lstm_model.add(Embedding(input_dim= vocab_size, output_dim=emdedding_size, weights=[pretrained_weights]))
lstm_model.add(LSTM(units=emdedding_size))
lstm_model.add(Dense(units=vocab_size))
lstm_model.add(Activation('softmax'))
lstm_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

lstm_model.fit(X, y, batch_size=128, epochs=3)

When X are sentences and y are the next word for each sentence. Now , I have a sentence, and 5 words, and I want to rank them by probability given the sentence. What is the best way to do so?

Upvotes: 0

Views: 1224

Answers (1)

Naga kiran
Naga kiran

Reputation: 4607

Change your activation function of LSTM output layer to 'sigmoid',it will work.

pretrained_weights = w2v_model.wv.syn0
vocab_size, emdedding_size = pretrained_weights.shape

lstm_model = Sequential()
lstm_model.add(Embedding(input_dim= vocab_size, output_dim=emdedding_size, weights=[pretrained_weights]))
lstm_model.add(LSTM(units=emdedding_size))
lstm_model.add(Dense(units=vocab_size))
lstm_model.add(Activation('sigmoid'))
lstm_model.compile(optimizer='adam', loss='mean_squared_error')

lstm_model.fit(X, y, batch_size=128, epochs=3)

Upvotes: 1

Related Questions