CODEWITHSUNDEEP
python-3.xtensorflowtf.kerastensorflow-hub
Ashwan1
Ashwan1

Reputation: 3

Problem using Elmo from tensorflow hub as custom tf.keras layer during prediction

I am trying to use Elmo from tensorflow hub with tf.keras, to perform NER. Training is fine and loss is decreasing, also test set gives good results. But I am unable to predict, as I get following error:

2019-05-02 15:41:42.785946: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
Traceback (most recent call last):
  File "elmo_eva_brain.py", line 668, in <module>
    np.array([['hello', 'world'] + ['--PAD--'] * 18])))
  File "/home/ashwanipandey/eva_ml/experimental/eva_brain/venv/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1113, in predict
    self, x, batch_size=batch_size, verbose=verbose, steps=steps)
  File "/home/ashwanipandey/eva_ml/experimental/eva_brain/venv/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 329, in model_iteration
    batch_outs = f(ins_batch)
  File "/home/ashwanipandey/eva_ml/experimental/eva_brain/venv/lib64/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3076, in __call__
    run_metadata=self.run_metadata)
  File "/home/ashwanipandey/eva_ml/experimental/eva_brain/venv/lib64/python3.6/site-packages/tensorflow/python/client/session.py", line 1439, in __call__
    run_metadata_ptr)
  File "/home/ashwanipandey/eva_ml/experimental/eva_brain/venv/lib64/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: len(seq_lens) != input.dims(0), (256 vs. 1)
         [[{{node Embed/elmo/elmo_module_apply_tokens/bilm/ReverseSequence}}]]
         [[{{node Tag/t_output/transpose_1}}]]

256 is my batch size during training. I am trying to predict just one sentence.

I tried to search a lot on internet, but all in vane. Any help is much appreciated. I can definitely get predictions if I repeat my vector 256 times and set batch_size to 256 during prediction. But as you can see this is highly inefficient workaround.

Here is code for custom layer

class ElmoEmbeddingLayer(keras.layers.Layer):
    def __init__(self, dimensions=1024, batch_size=512, word_size=20, **kwargs):
        self.dimensions = 1024
        self.trainable = True
        self.batch_size = _BATCH_SIZE
        self.word_size = _WORD_SIZE
        super().__init__(**kwargs)

    def build(self, input_shape):
        self.elmo = hub.Module('https://tfhub.dev/google/elmo/2', trainable=self.trainable,
                               name=f"{self.name}_module")
        super().build(input_shape)

    def call(self, x, mask=None):
        result = self.elmo(inputs={
            "tokens": K.cast(x, tf.string),
            "sequence_len": K.constant(self.batch_size*[self.word_size], dtype=tf.int32)
        },
            as_dict=True,
            signature='tokens',
        )['elmo']
        return result

    def compute_mask(self, inputs, mask=None):
        return K.not_equal(inputs, '--PAD--')

    def compute_output_shape(self, input_shape):
        return (None, self.word_size, self.dimensions)

    def get_config(self):
        config = {
            'dimensions': self.dimensions,
            'trainable': self.trainable,
            'batch_size': self.batch_size,
            'word_size': self.word_size
        }
        base_config = super().get_config()
        return dict(list(base_config.items()) + list(config.items()))

Here is my model architecture: model architecture

Upvotes: 0

Views: 1100

Answers (2)

Stamatis Outsios
Stamatis Outsios

Reputation: 26

I had the same problem with you, working on an RNN ELMo pos-tagger model. Finally I followed the solution predicting in batches and keeping the test sample I want:

model.predict([X_test[:split_te]], batch_size=256)[0]

For more ideas (like copying weights) look here!

Upvotes: 1

Stamatis Outsios
Stamatis Outsios

Reputation: 26

The number of samples (in train and also test set) must be divisible by the batch_size. Otherwise the last batch in keras will break the architecture. So for example a solution is to use samples until split_tr for training and split_te for predicting:

split_tr = (X_train.shape[0]//BATCH_SIZE)*BATCH_SIZE
split_te = (X_test.shape[0]//BATCH_SIZE)*BATCH_SIZE
model.fit(X_train_text[:split_tr], y_train[:split_tr], batch_size=BATCH_SIZE, epochs=15, validation_data=(X_test_text[:split_te], y_test[:split_te]), verbose=1)

Upvotes: 0

Related Questions