CODEWITHSUNDEEP
kerasdeep-learningneural-networklstmrecurrent-neural-network
Niccolò Ajroldi
Niccolò Ajroldi

Reputation: 45

keras, LSTM - predict on inputs of different length?

I have fitted an LSTM that deals with inputs of different length:

model = Sequential()
model.add(LSTM(units=10, return_sequences=False, input_shape=(None, 5)))
model.add(Dense(units=1, activation='sigmoid'))

Having fitted the model, I want to test it on inputs of different size.

x_test.shape # = 100000
x_test[0].shape # = (1, 5)
x_test[1].shape # = (3, 5)
x_test[2].shape # = (8, 5)

Testing on single instances j is not a problem (model.predict(x_test[j]), but looping on all of them is really slow.

Is there a way of speeding up the computation? model.predict(x_test) does not work.
Thank you!

Upvotes: 0

Views: 907

Answers (2)

Niccolò Ajroldi
Niccolò Ajroldi

Reputation: 45

The best solution that I have found so far is grouping together data windows with the same length. For my problem, it's enough to significantly speed up the computation.

Hope this trick would help other people.

import numpy as np

def predict_custom(model, x):
    """x should be a list of np.arrays with different number of rows, but same number of columns"""
    
    # dictionary with key = length of the window, value = indices of samples with such length
    dic = {}
    for i, x in enumerate(x):
        if dic.get(x.shape[0]):
            dic[x.shape[0]].append(i)
        else:
            dic[x.shape[0]] = [i]
    
    y_pred = np.full((len(x),1), np.nan)
    
    # loop over dictionary and predict together samples of the same length
    for key, indexes in dic.items():
        # select samples of the same length (conversion to np.array is used for subsetting "x" using "indexes")
        x = np.asarray(x, dtype=object)[indexes].tolist()
        # gather such samples in a 3D np.array
        x_3d = np.stack(x, axis=0)
        # use dictionary values to insert results in the correspondent row of y_pred
        y_pred[indexes] = model.predict(x_3d)
        
    return y_pred

Upvotes: 0

Ali Haider
Ali Haider

Reputation: 388

The most common way to speed up model inference is to run inference on GPU, instead of the CPU (I'm assuming you are not already doing that). You can set up GPU support by following the official guide here. Unless you are explicitly asking keras to run inference on CPU, your code should work as is, without any changes. To confirm if you are using GPU, you can use this article.

Hope the answer was helpful!

Upvotes: 1

Related Questions