How to view initialized weights (i.e. before training)?

Question

I am using Keras to generate a simple single layer feed forward network. I'd like to get a better handle on the values of the weights when they are initialized via the kernel_initializer argument.

Is there a way I can view the values of the weights just after initialisation (i.e. before the training is complete)?

Answer 1

You need to specify the dimensions of the input to the first layer otherwise it will give you an empty list. Compare both results from both prints the only difference is in the initialization of the shape of the input.

from keras import backend as K
import numpy as np 
from keras.models import Sequential
from keras.layers import Dense
# first model without input_dim prints an empty list   
model = Sequential()
model.add(Dense(5, weights=[np.ones((3,5)),np.zeros(5)], activation='relu'))
print(model.get_weights())


# second model with input_dim prints the assigned weights
model1 = Sequential()
model1.add(Dense(5,  weights=[np.ones((3,5)),np.zeros(5)],input_dim=3, activation='relu'))
model1.add(Dense(1, activation='sigmoid'))

print(model1.get_weights())

Answer 2

Just use get_weights() on the model. For example:

i = Input((2,))
x = Dense(5)(i)

model = Model(i, x)

print model.get_weights()

This will print a 2x5 matrix of weights and a 1x5 matrix of biases:

[array([[-0.46599612,  0.28759909,  0.48267472,  0.55951393,  0.3887372 ],
   [-0.56448901,  0.76363671,  0.88165808, -0.87762225, -0.2169953 ]], dtype=float32), 
 array([ 0.,  0.,  0.,  0.,  0.], dtype=float32)]

Biases are zero since the default bias initializer is zeros.

Answer 3

The answer given by @Chris_K should work - model.get_weights() prints correct initialization weights before fit is called. Try running this code as a sanity check - it should print two matrices (for two layers) that are non-zero, then print two matrices that are zero:

from keras.models import Sequential
from keras.layers import Dense
import keras
import numpy as np

X = np.random.randn(10,3)
Y = np.random.randn(10,)

# create model
model1 = Sequential()
model1.add(Dense(12, input_dim=3, activation='relu'))
model1.add(Dense(1, activation='sigmoid'))

print(model1.get_weights())

# create model
model2 = Sequential()
model2.add(Dense(12, input_dim=3, kernel_initializer='zero', activation='relu'))
model2.add(Dense(1, kernel_initializer='zero', activation='sigmoid'))

print(model2.get_weights())

Here's the output I'm seeing:

[
array([[-0.08758801, -0.20260376,  0.23681498, -0.59153044, -0.26144034,
         0.48446459, -0.02285194,  0.0874517 ,  0.0555284 , -0.14660612,
         0.05574059, -0.14752924],
       [ 0.20496374, -0.4272995 ,  0.07676286, -0.38965166,  0.47710329,
        -0.26640627, -0.33820981, -0.48640659,  0.11153179, -0.01180136,
        -0.52833426,  0.56279379],
       [-0.12849617,  0.2982074 ,  0.38974017, -0.58133346, -0.09883761,
         0.56037289,  0.57482034,  0.08853614,  0.14282584, -0.52498174,
        -0.35414279, -0.49750996]], dtype=float32), array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.], dtype=float32), array([[-0.65539688],
       [-0.58926439],
       [ 0.6232332 ],
       [-0.6493122 ],
       [ 0.57437611],
       [-0.42971158],
       [ 0.66621709],
       [-0.17393446],
       [ 0.57196724],
       [-0.01042461],
       [ 0.32426012],
       [-0.08326346]], dtype=float32), array([ 0.], dtype=float32)]
[array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]], dtype=float32), array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.], dtype=float32), array([[ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.]], dtype=float32), array([ 0.], dtype=float32)]