Numpy sum() got an 'keepdims' error

Question

This is a paragraph of a neural network code example:

def forward_step(X, W, b, W2, b2):
    hidden_layer = np.maximum(0, np.dot(X, W) + b)
    scores = np.dot(hidden_layer, W2) + b2
    exp_scores = np.exp(scores)
    probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
    ...

The last line of the code shown above threw an error:

<ipython-input-49-d97cff51c360> in forward_step(X, W, b, W2, b2)
     14     scores = np.dot(hidden_layer, W2) + b2
     15     exp_scores = np.exp(scores)
---> 16     probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
     17     corect_logprobs = -np.log(probs[range(X.shape[0]), y])

/Users/###/anaconda/lib/python3.6/site-packages/numpy/core/fromnumeric.py in sum(a, axis, dtype, out, keepdims)
   1810             pass
   1811         else:
-> 1812             return sum(axis=axis, dtype=dtype, out=out, **kwargs)
   1813     return _methods._sum(a, axis=axis, dtype=dtype,
   1814                          out=out, **kwargs)

TypeError: sum() got an unexpected keyword argument 'keepdims'

There is a similar question Numpy sum keepdims error which says that the edition of numpy should be greater than 1.7. I have checked my edition of numpy:

import numpy
numpy.version.version
>> 1.12.1

Now I am confused about how this error occurred.

Answer 1

Note that under the keepdims argument in the docs for numpy.sum() it states:

keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.
If the default value is passed, then keepdims will not be passed through to the sum method of sub-classes of ndarray, however any non-default value will be. If the sub-classes sum method does not implement keepdims any exceptions will be raised.

So it states here that if you're using a sub-class of numpy.ndarray, then you'll get this error if the corresponding sum function for the sub-class hasn't been defined with it.

Notice that in your error it references line 1812 in numpy/core/fromnumeric.py. Take a look at that in context in the actual numpy 1.12.x source:

kwargs = {}
if keepdims is not np._NoValue:
    kwargs['keepdims'] = keepdims
if isinstance(a, _gentype):
    res = _sum_(a)
    if out is not None:
        out[...] = res
        return out
    return res
if type(a) is not mu.ndarray:
    try:
        sum = a.sum
    except AttributeError:
        pass
    else:
        return sum(axis=axis, dtype=dtype, out=out, **kwargs)
return _methods._sum(a, axis=axis, dtype=dtype,
                     out=out, **kwargs)

Two things are important to note here: the sum function did parse your keepdims variable, since it pulled it above line 1812 and tried to put it in another function, so you know the error wasn't the way you used the variable. The other important thing is that the line 1812 which you're erroring on is only executing if type(a) is not mu.ndarray, i.e., if you're using a different class than ndarray. And this is exactly what the documentation is referencing. If you have a different class, then they need to implement this sum function with the keepdims argument, and if they don't it will raise an error.

Other classes like np.matrix for example will have a different sum function, and it seems that, even in numpy 1.13.x, sum for np.matrix types does not support the keepdim argument (because in numpy, matrices always are 2D). For example, it works fine with a np.array:

>>> import numpy as np
>>> A = np.eye(4)
>>> A
array([[ 1.,  0.,  0.,  0.],
       [ 0.,  1.,  0.,  0.],
       [ 0.,  0.,  1.,  0.],
       [ 0.,  0.,  0.,  1.]])
>>> np.sum(A, axis=1, keepdims=True)
array([[ 1.],
       [ 1.],
       [ 1.],
       [ 1.]])

But with a np.matrix, it doesn't:

>>> import numpy.matlib
>>> B = np.matlib.eye(4)
>>> np.sum(B, axis=1, keepdims=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../numpy/core/fromnumeric.py", line 1832, in sum
    return sum(axis=axis, dtype=dtype, out=out, **kwargs)
TypeError: sum() got an unexpected keyword argument 'keepdims'

But, most array/matrix type objects can be easily cast to an array in numpy with np.array(<object>), and this should solve the problem for most sub-classed objects in numpy and likely your problem. You can also simply wrap the result back into a np.matrix if you need to.

>>> B = np.matlib.eye(4)
>>> B = np.array(B)
>>> np.sum(B, axis=1, keepdims=True)
array([[ 1.],
       [ 1.],
       [ 1.],
       [ 1.]])

However, if your class of object is a np.matrix type, then the keepdims argument is pointless. Matrices are always 2D, so the sum function won't reduce a dimension, and thus the argument wouldn't do anything. This is why it isn't implemented for matrices.