CODEWITHSUNDEEP
pythonkeras
DamianJ
DamianJ

Reputation: 539

Multi-input models using Keras (Model API)

I've been trying to construct a multiple input model using Keras. I am coming from using the sequential model and having only one input which was fairly straight-forward. I have been looking at the documentation (https://keras.io/getting-started/functional-api-guide/) and some answers here on StackOverflow (How to "Merge" Sequential models in Keras 2.0?). Basically what I want is to have two inputs train one model. One input is a piece of text and the other is a set of hand-picked features that were extracted from that text. The hand-picked feature vectors are of a constant length. Below is what I've tried so far:

    left = Input(shape=(7801,), dtype='float32', name='left_input')
    left = Embedding(7801, self.embedding_vector_length, weights=[self.embeddings],
                     input_length=self.max_document_length, trainable=False)(left)

    right = Input(shape=(len(self.z_train), len(self.z_train[0])), dtype='float32', name='right_input')

    for i, filter_len in enumerate(filter_sizes):
        left = Conv1D(filters=128, kernel_size=filter_len, padding='same', activation=c_activation)(left)
        left = MaxPooling1D(pool_size=2)(left)

    left = CuDNNLSTM(100, unit_forget_bias=1)(left)
    right = CuDNNLSTM(100, unit_forget_bias=1)(right)

    left_out = Dense(3, activation=activation, kernel_regularizer=l2(l_2), activity_regularizer=l1(l_1))(left)
    right_out = Dense(3, activation=activation, kernel_regularizer=l2(l_2), activity_regularizer=l1(l_1))(right)
    for i in range(self.num_outputs):
        left_out = Dense(3, activation=activation, kernel_regularizer=l2(l_2), activity_regularizer=l1(l_1))(left_out)
        right_out = Dense(3, activation=activation, kernel_regularizer=l2(l_2), activity_regularizer=l1(l_1))(right_out)

    left_model = Model(left, left_out)
    right_model = Model(right, right_out)

    concatenated = merge([left_model, right_model], mode="concat")
    out = Dense(3, activation=activation, kernel_regularizer=l2(l_2), activity_regularizer=l1(l_1), name='output_layer')(concatenated)

    self.model = Model([left_model, right_model], out)
    self.model.compile(loss=loss, optimizer=optimizer, metrics=[cosine, mse, categorical_accuracy])

This gives the error:

TypeError: Input layers to a `Model` must be `InputLayer` objects. Received inputs: Tensor("cu_dnnlstm_1/strided_slice_16:0", shape=(?, 100), dtype=float32). Input 0 (0-based) originates from layer type `CuDNNLSTM`.

Upvotes: 5

Views: 13013

Answers (1)

parsethis
parsethis

Reputation: 8078

The error is clear (and you're almost there). The code is currently attempting to set the inputs as the models [left_model, right_model], instead the inputs must be Input layers [left, right]. The relevant part of the code sample above should read:

self.model = Model([left, rigt], out)

see my answer here as reference: Merging layers especially the second example.

Upvotes: 5

Related Questions