CODEWITHSUNDEEP
kerasneural-networkbinaryinitialization
Yevhenii
Yevhenii

Reputation: 43

How can I set binary weights values (0,1) or (-1,1) to the layer in Keras?

I would like to ask if I can set the weights initializer in (any) Keras layer to binary values - for example for the weights of simple Dense layer to be 0 and 1 only? This would be helpful for instance in the case of the Conv1D layer to relax the computational time.

Thank you, J

Upvotes: 1

Views: 583

Answers (1)

Coderji
Coderji

Reputation: 7745

Yes this is possible by creating a custom initializer:

def binary_weights(shape, dtype=tf.float32):
    """This function generates weights of random 0s and 1s based on the provided shape"""
    # build logits matrix:
    logits = tf.fill((shape[0], 2), 0.5)
    # uniformly pick the class.
    return tf.cast(tf.random.categorical(tf.math.log(logits), shape[1]), dtype=dtype)

Then when you specify the layer:

model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(units, kernel_initializer=binary_weights, input_shape=[num_features,]),
    ...
])

To check the generated weights:

print(model.layers[0].get_weights()[0])

Upvotes: 2

Related Questions