ValueError: Dimension (-1) must be in the range [0, 2) in Keras

Question

Suddenly I have this error with kears with tensorflow backend (python2.7) , same error with every code. I thought its keras 1 and 2 incompatibility but it was not

Dimension (-1) must be in the range [0, 2), where 2 is the number of dimensions in the input. for 'metrics/acc/ArgMax' (op: 'ArgMax') with input shapes: [?,?], [].

'I update both tensorflow and keras like similar problem (link ↓↓) but still same error ValueError: Dimension (-1) must be in the range [0, 2) The full code (example)

**Code updated the whole code** 

using TensorFlow backend.
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA   library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA   library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA  library libcurand.so locally
60000 train samples
10000 test samples
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 512)               401920    
 _________________________________________________________________
dropout_1 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_2 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 10)                5130      
=================================================================
Total params: 669,706
Trainable params: 669,706
Non-trainable params: 0
_________________________________________________________________
Traceback (most recent call last):
  File "mnist_mlp.py", line 48, in <module>
    metrics=['accuracy'])
  File "/home/usr/miniconda2/lib/python2.7/site-packages/keras/models.py", line 784, in compile
    **kwargs)
  File "/home/usr/miniconda2/lib/python2.7/site-packages/keras/engine/training.py", line 924, in compile
    handle_metrics(output_metrics)
  File "/home/usr/miniconda2/lib/python2.7/site-packages/keras/engine/training.py", line 921, in handle_metrics
    mask=masks[i])
  File "/home/usr/miniconda2/lib/python2.7/site-packages/keras/engine/training.py", line 450, in weighted
    score_array = fn(y_true, y_pred)
  File "/home/usr/miniconda2/lib/python2.7/site-packages/keras/metrics.py", line 25, in categorical_accuracy
    return K.cast(K.equal(K.argmax(y_true, axis=-1),
  File "/home/usr/miniconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 1333, in argmax
    return tf.argmax(x, axis)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 249, in argmax
    return gen_math_ops.arg_max(input, axis, name)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 168, in arg_max
    name=name)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
    op_def=op_def)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2242, in create_op
    set_shapes_for_outputs(ret)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1617, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1568, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
    debug_python_shape_fn, require_shape_fn)
  File "/home/usr/.local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Dimension (-1) must be in the range [0, 2), where 2 is the number of dimensions in the input. for 'metrics/acc/ArgMax' (op: 'ArgMax') with input shapes: [?,?], [].'

Answer 1

I faced same error message (due to default Digital Ocean app) when tried to load a already saved model from my mac to DigOcean. Updated tensorflow using:

pip3 install --upgrade tensorflow

and 1.3.0 was installed and the problem got resolved when I restarted the jupyter kernel.

Answer 2

I just started playing with Keras and I incurred in the same problem. I followed the different workarounds proposed on different forums - including running an upgrade of tensorflow/keras itself - but that did not seem to work for me.

The problem seems to be that the argmax function in Keras. backend is called, by default, with a axis=-1, which is out of range, given that only [0, 2) are legit.

My solution has been rewriting the categorical accuracy function:

import keras.backend as K

def get_categorical_accuracy_keras(y_true, y_pred):
    return K.mean(K.equal(K.argmax(y_true, axis=1), K.argmax(y_pred, axis=1)))

(I found the formula in this thread)

which should be equivalent to the following function, which leverages numpy library:

import numpy as np

def get_categorical_accuracy(y_true, y_pred):
    return (np.argmax(y_true, axis=1) == np.argmax(y_pred, axis=1)).mean()

Using get_categorical_accuracy_keras function in the model compilation:

model.compile(loss=losses.categorical_crossentropy, optimizer='adam', metrics=[get_categorical_accuracy_keras])

seems to solve the problem.

Of course, I would like to use the already defined accuracy myself, so any suggestion in that sense is welcome