CODEWITHSUNDEEP
pythontensorflowkerasresnetconv-neural-network
FF0605
FF0605

Reputation: 441

How to make Keras have two different initialisers in a dense layer?

I have two separately designed CNNs for two different features(image and text) of the same data, and the output has two classes

In the very last layer:

for image (resnet), I would like to use "he_normal" as the initializer

flatten1 = Flatten()(image_maxpool)
dense = Dense(output_dim=2, kernel_initializer="he_normal")(flatten1)

but for the text CNNs, i would like to use the default "glorot_normal"

flatten2 = Flatten()(text_maxpool)
output = Dense(output_dim=2, kernel_initializer="glorot_normal")(flatten2)

the flatten1 and flatten2 have sizes:

flatten_1 (Flatten) (None, 512)

flatten_2 (Flatten) (None, 192)

is there anyway i can concate these two flatten layers and have a long dense layer with a size 192+512 = 704, where the first 192 and second 512 has two seperate kernel_initializer, and produce a 2-class outputs?

something like this:

merged_tensor = merge([flatten1, flatten2], mode='concat', concat_axis=1)
output = Dense(output_dim=2, 
    kernel_initializer for [:512]='he_normal',
    kernel_initializer for [512:]='glorot_normal')(merged_tensor)

Edit: I think I have gotten this work by having the following codes(thanks to @Aechlys):

def my_init(shape, shape1, shape2):
    x = initializers.he_normal()(shape1)
    y = initializers.glorot_normal()(shape2)
    return tf.concat([x,y], 0)

class_num = 2

flatten1 = Flatten()(image_maxpool)
flatten2 = Flatten()(text_maxpool)

merged_tensor = concatenate([flatten1, flatten2],axis=-1)

output = Dense(output_dim=class_num, kernel_initializer=lambda shape: my_init(shape,\
              shape1=(512,class_num),\
              shape2=(192,class_num)),\
              activation='softmax')(merged_tensor)

I have to manually add the shape size 512 and 192, because I failed to get the size of flatten1 and flatten1 via the code

flatten1.get_shape().as_list()

,which gave me [none, none], althought it should be [None, 512], other than that it should be fine

Upvotes: 1

Views: 1268

Answers (1)

Aechlys
Aechlys

Reputation: 1306

Oh my, have I had fun with this one. You have to create your own kernel intializer:

def my_init(shape, dtype=None, *, shape1, shape2):
    x = keras.initializers.he_normal()(shape1, dtype=dtype)
    y = keras.initializers.glorot_normal()(shape2, dtype=dtype)
    return tf.concat([x,y], 0)

Then you will call it via lambda function within the Dense function:

Unfortunately, as you can see, I have not been able to deduce the shape programatically, yet. I may update this answer when I do. But, if you know the shape beforehand you can pass them as constants:

DENSE_UNITS = 64

input_t = Input((1,25))
input_i = Input((1,35))
input_a = Concatenate(axis=-1)([input_t, input_i])

dense = Dense(DENSE_UNITS, kernel_initializer=lambda shape: my_init(shape, 
                                shape1=(int(input_t.shape[-1]), DENSE_UNITS),
                                shape2=(int(input_i.shape[-1]), DENSE_UNITS)))(input_a)
tf.keras.Model(inputs=[input_t, input_i], outputs=dense)

Out: <tensorflow.python.keras._impl.keras.engine.training.Model at 0x19ff7baac88>

Upvotes: 3

Related Questions