CODEWITHSUNDEEP
pythonkerasensemble-learning
colt.exe
colt.exe

Reputation: 728

Ensembling models with different inputs

I instantiated 3 simple not so deep models(could be deeper as I like) and trained them with 3 seperate inputs. To be clear, the first input is the face image, second is eyes and the third is mouth (for facial expression recognition). I'd like to ensemble a model that I feed with 3 inputs (with all same class label ofc) and get a single label output. The motivation is that 90% accurate models may perform better when ensembled together. It should look like this:

Facial input-----------Eyes Input------------Mouth Input
 (100x100x1)          (100x100x1)          (100x100x1) 
     |                     |                     | 

   .....                 ......                .....

     |                     |                     |
      ___________________________________________
                 Some concatenation over here
                           |
                   | Output Label|

Or should I complete forget this; acquire the response of bottom Dense layers of each model after test, and combine them to a feature vector and classify. Clueless over here...

Upvotes: 2

Views: 875

Answers (1)

markuscosinus
markuscosinus

Reputation: 2267

Suppose your models are called model1, model2 and model3. You can use the functional API to merge your models like this:

input1 = model1.input
input2 = model2.input
input3 = model3.input

m1 = model1(input1)
m2 = model2(input2)
m3 = model3(input3)

output = concatenate([m1, m2, m3]) # this is the concatenation
output = Dense(10)(output) # just an example for what you could add to the model
output = Activation('softmax')(output)

model = Model(inputs=[input1,input2,input3], outputs=output)

model can then be trained and it will train all three models simultaniously. If you only want to train the part of the model on top of the three models, you can use the trainable parameter.

Upvotes: 3

Related Questions