CODEWITHSUNDEEP
pythontensorflowkeraslstmrecurrent-neural-network
Jsevillamol
Jsevillamol

Reputation: 2543

Processing sequences of variable length within a batch in Keras

Suppose I have some data of shape (?, 10).

That is, the features of the data are sequences of variable length and each element of the sequence is expressed by 10 numbers.

We want to feed this to a LSTM, so we prepare each batch to have length (32, m, 10), where m is the max sequence length of the examples within the batch.

The examples within the batch of sequence length shorter than m are padded with zeroes.

Now, we want to feed this to a LSTM and we want the LSTM to stop updating the output on the padding inputs.

In Tensorflow, this would be done via the parameter sequence_length of its dynamic_rnn.

How can I achieve the same effect in Keras?

Upvotes: 0

Views: 1376

Answers (1)

nuric
nuric

Reputation: 11225

You need to use a Masking which will produce a mask that allows the LSTM to skip those padded values. From the documentation:

model = Sequential()
model.add(Masking(mask_value=0., input_shape=(None, 10)))
model.add(LSTM(32))

The LSTM above will now skip timesteps where all the 10 features are 0 padded. Note: if you are returning sequences then the previous hidden state will be returned:

x = [2, 0, 1, 0] # for example
# will produce
y = [h1, h1, h2, h2] # so 0 pads are skipped
# but returning sequences repeats the last hidden state for masked values

Upvotes: 2

Related Questions