CODEWITHSUNDEEP
machine-learningneural-networkartificial-intelligencedeep-learninglstm
Leb_Broth
Leb_Broth

Reputation: 1111

LSTM/RNN many to one

I have the below dataset for a chemical process comprised of 5 consecutive input vectors to produce 1 output. Each input is sampled every minute while the output os sample every 5.

dataset

While I believe the output depends on the 5 previous input vectors, than I decided to look for LSTMs for my design. After a lot of research on how should be my LSTM architecture, I concluded that I should mask some of the output sequence by zeros and only leave the last output. The final architecture is below according to my dataset:

lstm architecture

My question is: What should be my 3D input tensor parameters? E.g. [5, 5, ?]? And also what should be my "Batch size"? Should it be the quantity of my samples?

Upvotes: 5

Views: 8762

Answers (2)

SaTa
SaTa

Reputation: 2692

Looking at the previous answer, I would say that you do not have to do a many-to-one architecture. It really depends on the problem you have. For example, if you system has a lot of dependencies from the past, i.e. more that 5 samples in your case, it would be better to do many-to-many architecture but with different input and output frequencies. But if you think that the previous 5 samples do not impact your next 5 samples. then a many-to-one architecture would do it.

Also, if you your problem is regression, you can use a Dense layer as the output of an LSTM cell is a tanh with output range of (-1, 1).

Upvotes: -1

uyaseen
uyaseen

Reputation: 1201

Since you are going for many to one sequence modelling, you don't need to pad zeros to your output (it's not needed). The easiest thing would be to perform classification at last time-step i.e after RNN/LSTM sees the 5th input. The dimension of your 3D input tensor will be [batch_size, sequence_length, input_dimensionality], where sequence_length is 5 in your case (row 1-5, 7-11, 13-17 etc.), and input_dimensionality is also 5 (i.e. column A- E). Batch_size depends on the number of examples (also how much reliable is your data), if you have more than 10,000 examples then batch size of 30-50 should be okay (read this explanation about choosing the appropriate batch size).

Upvotes: 9

Related Questions