categorical_crossentropy loss , no attribute 'get_shape'

Question

I want to get categorical_crossentropy loss for my model(for training) which has last layer: model.add(Dense(num_classes, activation='softmax')) . so i take output of this layer and use follwing code to get loss output using following code:

from sklearn.metrics import confusion_matrix
from __future__ import print_function
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K
from keras.callbacks import TensorBoard
import numpy as np
from sklearn.model_selection import train_test_split
import tensorflow as tf
batch_size = 128
num_classes = 3
epochs = 1

# input image dimensions
img_rows, img_cols = 28, 28


(x_train, y_train), (x_test, y_test) = mnist.load_data()
x1_train=x_train[y_train==0]; y1_train=y_train[y_train==0]
x1_test=x_test[y_test==0];y1_test=y_test[y_test==0]
x2_train=x_train[y_train==1];y2_train=y_train[y_train==1]
x2_test=x_test[y_test==1];y2_test=y_test[y_test==1]
x3_train=x_train[y_train==2];y3_train=y_train[y_train==2]
x3_test=x_test[y_test==2];y3_test=y_test[y_test==2]

X=np.concatenate((x1_train,x2_train,x3_train,x1_test,x2_test,x3_test),axis=0)
Y=np.concatenate((y1_train,y2_train,y3_train,y1_test,y2_test,y3_test),axis=0)

# the data, shuffled and split between train and test sets
x_train, x_test, y_train, y_test = train_test_split(X,Y)

if K.image_data_format() == 'channels_first':
    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

model = Sequential()
model.add(Conv2D(1, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Flatten())

model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test))
model.summary()
get_dense_layer_output = K.function([model.layers[0].input],
                                  [model.layers[3].output])
layer_output = get_dense_layer_output([x_train])[0]
g=K.categorical_crossentropy(layer_output, y_train)

and getting error: AttributeError: 'numpy.ndarray' object has no attribute 'get_shape'. How to solve it.

Answer 1

For all practical purposes, Dr. Snoopy's' answer below is correct.

For educational purposes (or whatever other purposes you may have), here is what is happening with your script & error:

In all Keras loss functions, such as K.categorical_crossentropy(), the arguments are tensors (which do have a get_shape argument), and not numpy arrays (which do not) - check the docs. So, you should convert your numpy arrays to tensors, but before that, you have to cast them in the same type (they are not), otherwise you will get an error:

y_train.dtype
# dtype('float64')

layer_output.dtype
# dtype('float32')

y_train = y_train.astype('float32')
y_true = K.constant(y_train)
y_pred = K.constant(layer_output)
g = K.categorical_crossentropy(target=y_true, output=y_pred)
g
# <tf.Tensor 'Neg_1:0' shape=(16327,) dtype=float32>

As you can see, the result g is a (Tensorflow) tensor, which needs to be evaluated:

ce = K.eval(g)  # 'ce' for cross-entropy
type(ce)
# numpy.ndarray
ce.shape
# (16327,)

The result is a numpy array with the loss for each one of your 16,327 training samples.

Answer 2

Your network is already being trained using categorical_crossentropy. To get the loss value on a dataset you can just use model.evaluate

values = model.evaluate(X_train, y_train)

Then values[0] will be the loss value and values[1] is the accuracy metric.