CODEWITHSUNDEEP
python-3.xkerassequencelstmautoencoder
Simran Agarwal
Simran Agarwal

Reputation: 19

Memory error while running LSTM Autoencoders

I am setting up an LSTM Autoencoder with multivariant time sequences. Each of my sequence has different time steps(approximately 30 million steps in one sequence) and 6 features. I know to give one sequence as input to the LSTM Autoencoder, I have to reshape my sequence as (1,30million,6).I reshaped all of my 9 sequences in a similar manner. I want the autoencoder to reconstruct my sequences. However my program is crashing due to large number of time steps in each sequence. How can I solve this memory error. Even if I am giving data in batch size,my program is running out of memory. I am new to machine learning and sequence learning, so please help me with the same.My network is attached below: `

def repeat_vector(args):

 [layer_to_repeat, sequence_layer] = args

 return RepeatVector(K.shape(sequence_layer)[1])(layer_to_repeat)

encoder_input = Input(shape=(None, self._input_features))

encoder_output = LSTM(self._latent_space)(encoder_input)

decoder_input = Lambda(repeat_vector, output_shape=(None, self._latent_space))([encoder_output, encoder_input])

decoder_output = LSTM(self._input_cells, return_sequences=True)(decoder_input)

self._autoencoder = Model(encoder_input, decoder_output) `

I have already tried to take input via hdf files.

Upvotes: 1

Views: 891

Answers (1)

Ujjwal Saxena
Ujjwal Saxena

Reputation: 409

I am not sure what system configuration are you using. OOMs can be solved from both software and hardware ends. If you are using a system with say 4GB RAM and some i5 processor(assuming it's intel), it might not work. If you are working on a GPU(which is not very likely.) it should not be a hardware issue.

If your system has a graphic card, well then you can optimize the code a bit.

  1. Try a batch size of 1.
  2. If you have a pre-processing queue etc. try to tweak the queue size.
  3. I would suggest you to try this for a smaller series once before going in for the complete thing, and check if it works.
  4. If you take the time step to be large, it will lose precision and if it's too small, well then it's heavy to compute. Check for each one of them, if the time step can be increased, without compromising much on precision.
  5. You can use PCA for knowing the important features and reduce dimensionality. You can also use random forest as a preprocessing step to know the feature importance and reduce the features with less inportance.

Upvotes: 2

Related Questions