CODEWITHSUNDEEP
pythontensorflowtensorflow2.0recurrent-neural-network
James
James

Reputation: 18379

TensorFlow text generation RNN example failing on TF 2.6, tf.sparse.to_dense(), Invalid argument: indices[1] = [0] is repeated

I am trying to run through the TensorFlow text generation RNN example,

https://github.com/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb

Running on a local Windows computer with TensorFlow 2.6 installed.

I was able to run through and train the RNN model successfully. I was getting a "Tensor' object has no attribute 'numpy" error but added,

tf.compat.v1.enable_eager_execution()

and this resolved it.

But now trying to test the model with some text I am getting the error,

Invalid argument: indices[1] = [0] is repeated

This occurs in tf.sparse.to_dense inside the OneStep function.

class OneStep(tf.keras.Model):
  def __init__(self, model, chars_from_ids, ids_from_chars, temperature=1.0):
    super().__init__()
    self.temperature = temperature
    self.model = model
    self.chars_from_ids = chars_from_ids
    self.ids_from_chars = ids_from_chars

    print(len(ids_from_chars.get_vocabulary()))
    # Create a mask to prevent "[UNK]" from being generated.
    skip_ids = self.ids_from_chars(['[UNK]'])[:, None]
    sparse_mask = tf.SparseTensor(
        # Put a -inf at each bad index.
        values=[-float('inf')]*len(skip_ids),
        indices=skip_ids,
        # Match the shape to the vocabulary
        dense_shape=[len(ids_from_chars.get_vocabulary())])
    print(sparse_mask)
    self.prediction_mask = tf.sparse.to_dense(sparse_mask)

I added some debug to print the ids_from_chars

76
SparseTensor(indices=tf.Tensor(
[[0]
[0]], shape=(2, 1), dtype=int64), values=tf.Tensor([-inf -inf], shape=(2,), dtype=float32), dense_shape=tf.Tensor([76], shape=(1,), dtype=int64))
2021-08-25 15:28:23.935194: W tensorflow/core/framework/op_kernel.cc:1692] OP_REQUIRES failed at sparse_to_dense_op.cc:162 : Invalid argument: indices[1] = [0] is repeated
Traceback (most recent call last):
File "app.py", line 1041, in test_nlp_text_generation
result = text_generation.predictionfunction(text, analytic_id)
File "D:\Projects\python-run-2\text_generation.py", line 238, in predictionfunction
one_step_model = OneStep(model, chars_from_ids, ids_from_chars)
File "D:\Projects\python-run-2\text_generation.py", line 166, in __init__
self.prediction_mask = tf.sparse.to_dense(sparse_mask)
File "D:\Users\james\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\sparse_ops.py", line 1721, in sparse_tensor_to_dense
name=name)
File "D:\Users\james\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_sparse_ops.py", line 3161, in sparse_to_dense
_ops.raise_from_not_ok_status(e, name)
File "D:\Users\james\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 6941, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[1] = [0] is repeated [Op:SparseToDense]

Also, I had this example running fine on my computer previously. Just had reinstalled TensorFlow and was trying the demo from scratch again.

Any idea what is causing this error, or how to fix it?

Upvotes: 0

Views: 272

Answers (1)

FancyXun
FancyXun

Reputation: 1298

I reproduced this error through the code below

import tensorflow as tf

#there are same values in the tensor
skip_ids = tf.constant([[0], [0]], dtype=tf.int64)

sparse_mask = tf.SparseTensor(
    # Put a -inf at each bad index.
    values=[-float('inf')] * len(skip_ids),
    indices=skip_ids,
    # Match the shape to the vocabulary
    dense_shape=[76])
print(sparse_mask)

prediction_mask = tf.sparse.to_dense(sparse_mask)

Your indices has same value, it does not allow to assign values in the same position. Just get unique values in indices tensor before:

import tensorflow as tf

skip_ids = tf.constant([[0], [0]], dtype=tf.int64)

# get unique indices
tmp1 = tf.reshape(skip_ids, shape=(-1,))
uniques, idx, counts = tf.unique_with_counts(tmp1)
uniques_ids = tf.expand_dims(uniques, axis=1)


sparse_mask = tf.SparseTensor(
    # Put a -inf at each bad index.
    values=[-float('inf')] * len(uniques_ids),
    indices=uniques_ids,
    # Match the shape to the vocabulary
    dense_shape=[76])
print(sparse_mask)

prediction_mask = tf.sparse.to_dense(sparse_mask)

print(prediction_mask)

My tensorflow version is 2.1.0

Upvotes: 1

Related Questions