CODEWITHSUNDEEP
pythontensorflowkeras
Vishwas Abhyankar
Vishwas Abhyankar

Reputation: 406

Unable to use Deep Learning RNN models trained on GPU instance, for the inference on CPU-only instance

I trained a deep learning model on GPU instance to speedup training process. But unfortunately our product instances are CPU-only instances. When I tried to use GPU trained model with CuDNNGRU on CPU-only instance, I got below error.

ValueError: CuDNNGRU is not compatible with GRU(reset_after=False)

similar kind of error you will get if you use CuDNNLSTMs

ValueError: CuDNNLSTM is not compatible with LSTM(reset_after=False)

Upvotes: 1

Views: 1362

Answers (1)

Vishwas Abhyankar
Vishwas Abhyankar

Reputation: 406

Instead of using CuDNNGRU or CuDNNLSTM, use normal GRU or LSTM with following options

Code that throws error when CuDNNGRU or CuDNNLSTM models trained on GPU throw error while inference on CPU-only instances.

CuDNNGRU

if tf.test.is_gpu_available():
    return tf.keras.layers.CuDNNGRU(units, 
                                    return_sequences=True, 
                                    return_state=True,
                                    recurrent_initializer='glorot_uniform')
else:
    return tf.keras.layers.GRU(units,
                               return_sequences=True,
                               return_state=True, 
                               recurrent_initializer='glorot_uniform')

or

CuDNNLSTM

if tf.test.is_gpu_available():
    return tf.keras.layers.CuDNNLSTM(units, 
                                    return_sequences=True, 
                                    return_state=True,
                                    recurrent_initializer='glorot_uniform')
else:
    return tf.keras.layers.LSTM(units,
                               return_sequences=True,
                               return_state=True, 
                               recurrent_initializer='glorot_uniform')

Please use below code, so that you can use GPU trained models for the inference on CPU-only instances.

GRU

if tf.test.is_gpu_available():
    return tf.compat.v1.keras.layers.CuDNNGRU(units,
                                              return_sequences=True,
                                              return_state=True,
                                              recurrent_initializer='glorot_uniform')
else:
    return tf.keras.layers.GRU(units, 
                               return_sequences=True,
                               return_state=True,
                               recurrent_activation='sigmoid',
                               reset_after='True',
                               recurrent_initializer='glorot_uniform')

or

LSTM

if tf.test.is_gpu_available():
    return tf.compat.v1.keras.layers.CuDNNLSTM(units,
                                              return_sequences=True,
                                              return_state=True,
                                              recurrent_initializer='glorot_uniform')
else:
    return tf.keras.layers.LSTM(units, 
                               return_sequences=True,
                               return_state=True,
                               recurrent_activation='sigmoid',
                               reset_after='True',
                               recurrent_initializer='glorot_uniform')

So by just using reset_after=True and recurrent_activation='sigmoid' options, my problem got solved.

Upvotes: 2

Related Questions