CODEWITHSUNDEEP
pythontensorflowtensorflow2.0tf.keras
Vahid Mirjalili
Vahid Mirjalili

Reputation: 6501

When evaluating a TensorFlow model, how can I see the values of a specific hidden layer in a single forward call?

I define a simple sequential model as follows: 

m = tf.keras.models.Sequential([
    tf.keras.layers.Dense(10, input_shape=(3,), name='fc1', activation='relu'),
    tf.keras.layers.Dense(3, input_shape=(3,), name='fc2'),
])

m.summary()

Model: "sequential_6"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
fc1 (Dense)                  (None, 10)                40        
_________________________________________________________________
fc2 (Dense)                  (None, 3)                 33        
=================================================================
Total params: 73
Trainable params: 73
Non-trainable params: 0
_________________________________________________________________

Now, I get a hidden layer layer_fc1 as follows:

layer_fc1 = m.get_layer('fc1')
layer_fc1
<tensorflow.python.keras.layers.core.Dense at 0x7f6fcc7d9eb8>

Now, i want tosee when I evaluate this model, I want to see the values of the layer fc1.

The evaluation of the entire network and layer_fc1 in separate forward-calls is as follows:

t = tf.random.uniform((1, 3))

m(t)
<tf.Tensor: id=446892, shape=(1, 3), dtype=float32, numpy=array([[ 0.0168661 , -0.12582873, -0.0308626 ]], dtype=float32)>

layer_fc1(t)
<tf.Tensor: id=446904, shape=(1, 10), dtype=float32, numpy=
array([[0.        , 0.        , 0.00877494, 0.05680769, 0.08027849,
        0.12362152, 0.        , 0.        , 0.22683921, 0.17538759]],
      dtype=float32)>

So, is there any way that with a single forward call with m(t), I also get the values at that hidden layer? This way, the computations will be more efficient.

Upvotes: 1

Views: 458

Answers (1)

Stewart_R
Stewart_R

Reputation: 14515

You can do this easily using the functional API as follows:

inpt = tf.keras.layers.Input(shape=(3,))
fc1_out = tf.keras.layers.Dense(10, name='fc1', activation='relu')(inpt)
fc2_out = tf.keras.layers.Dense(3, name='fc2')(fc1_out)
m = tf.keras.models.Model(inputs=inpt, outputs=[fc2_out, fc1_out])

t = tf.random.uniform((1,3))
m(t)

which gives the output you are looking for:

[<tf.Tensor: id=149, shape=(1, 3), dtype=float32, numpy=array([[-0.20491418, -0.33935153,  0.07777037]], dtype=float32)>,
 <tf.Tensor: id=145, shape=(1, 10), dtype=float32, numpy=
 array([[0.        , 0.5071918 , 0.        , 0.24756521, 0.05489198,
         0.31997102, 0.        , 0.23788954, 0.01050918, 0.24083027]],
       dtype=float32)>]

I'm less familiar with the Sequential API but I'd expect this to be impossible in the Sequential API because, to me, this is NOT a sequential model where one layer follows the other from input to output.

Upvotes: 1

Related Questions