CODEWITHSUNDEEP
tensorflowmachine-learningkerasdeep-learningneural-network
Savannah Madison
Savannah Madison

Reputation: 667

Trouble understanding parts of concepts in creating custom callbacks in keras

import keras
import numpy as numpy

class ActivationLogger(keras.callbacks.Callback):
    def set_model(self,model):
        self.model = model //inform the callback of what model we will be calling
        layer_outputs = [layer.output for layer in model.layers]
        self.activations_model = keras.models.Model(model.input,layer_outputs)//returns activation of every layer
        
    def on_epoch_end(self,epoch,logs = None):
        if self.validation_data is None:
            raise RuntimeError("Requires validation_data")
        validation_sample = self.validation_data[0][0:1]
        activations = self.activations_model.predict(validation_sample) #computes activation of every epoch 
        f = open('activations_at_epoch_' + str(epoch) + '.npz', 'w')
        np.savez(f, activations)
        f.close()

While I was reading this code to create custom callbacks,I couldn't understand few lines of code.I know what are callbacks. What I understood from the above code is that we inherit the super class keras.callbacks.Callback and in the set_model fucntion, we inform the callback of what model it will be calling. I am not able to understand the below line, why does keras.models.Model take model.input?

self.activations_model = keras.models.Model(model.input,
layer_outputs)

and the line activations = self.activations_model.predict(validation_sample)

The further lines just save the numpy arrays to the drive. Also is the callback created,called on every epoch?

Upvotes: 1

Views: 87

Answers (1)

devspartan
devspartan

Reputation: 622

Let's say i have an simple model

model = Sequential()
model.add(Dense(32, input_shape=(784, 1), activation='relu'))
model.add(Dense(16, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(4, activation='softmax'))

cb = ActivationLogger()
cb.set_model(model)

Now let me go through line by line of function set_model:

self.model = model  
self.model.summary() = Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 32)                25120     
_________________________________________________________________
dense_1 (Dense)              (None, 16)                528       
_________________________________________________________________
dropout (Dropout)            (None, 16)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 4)                 68        
=================================================================
Total params: 25,716
Trainable params: 25,716
Non-trainable params: 0

second line:

layer_outputs = [layer.output for layer in model.layers]
print(layer_outputs) = [<tf.Tensor 'dense/Relu:0' shape=(None, 32) dtype=float32>, <tf.Tensor 'dense_1/Relu:0' shape=(None, 16) dtype=float32>, <tf.Tensor 'dropout/cond/Identity:0' shape=(None, 16) dtype=float32>, <tf.Tensor 'dense_2/Softmax:0' shape=(None, 4) dtype=float32>]

layer_outputs contains all the tensors or the layers of the models and the third line:

self.activations_model = keras.models.Model(model.input,layer_outputs)

Now in this line, it creates a model with input shape corresponding to original model(model.input = it gives the input tensor or layer of a model. you can also checkout the output shape of a model using model.output)

so self.activation_model is model with one input shape((784, ) in this case) and output at every layer

so when you feed any input through this model it will give you a list of outputs correspond to every layer

Normally output will be a numpy array of shape (none, 4) (taking main Sequential model)

but self.activation will give you a list a numpy arrays. So the line

activations = self.activations_model.predict(validation_sample)

activation just contains the predictions of self.activation_model which a nothing but a list of numpy arrays

 [(none, 32)(output of first layer), (None, 16)(output of 2nd), (none, 16)(dropout lyr), (none, 4)(final)

i would suggest you to read about keras Model Function api which is used to make models with many input and outputs

Upvotes: 1

Related Questions