CODEWITHSUNDEEP
recurrent-neural-networklstm
Dr. Jekyll
Dr. Jekyll

Reputation: 406

RNN/LSTM library with variable length sequences without bucketing or padding

The problem I try to solve is a classification problem with 4 parallel inputs batches of sequences. To do so, I need 4 RNN/LSTM in parallel that merge in a fully connected layer. The issue is that in each parallel batch, the sequences have a variable length.

I cannot use padding to the maximum sequence length because it use too much RAM. Actually, some sequences are really long. I cannot use padding to a reduced length because the model cannot predict the output. I need the full sequence, I cannot know in advance where the interesting part of the sequence is.

I cannot use bucketing because if I split a sequence in one batch, I would have to do it the same way for each sequence with the same index in the 3 others batches. As the parallel sequences do not have the same length, the model will try to associate lots of empty sequences to either one or the other class.

In theory a RNN/LSTM should be able to learn sequences with different length without sequence manipulation. Unfortunately I do not know an implementation that enable me to do so. Does a such RNN/LSTM library exist (any language) ?

Upvotes: 1

Views: 560

Answers (1)

mikal94305
mikal94305

Reputation: 5083

Theano can handle variable length sequences, but Tensorflow cannot. You can test with this Theano, and let us know your results.

Upvotes: 2

Related Questions