CODEWITHSUNDEEP
pythonkeras
Smitha
Smitha

Reputation: 79

Keras model for multiple inputs with different number of channels

I am trying to make a convolutional neural network using Keras. I want two models which take different inputs, but share the same architecture and weights - the desired architecture is shown below. How would I define such a model using Keras?

Classifier 1 takes only input a whereas classifier 2 takes both a and b inputs. SO they can be concatenated to increase the depth to

I tried to implement this as follows:

from keras.layers import Input,Conv2D,Activation,Flatten,Concatenate, MaxPooling2D, Dense
from keras.models import Model

def combined_model():

    a = Input(shape=(100,100,None))   ## should not change this
    fe = Conv2D(3, (1,1), strides=(2,2), padding='same', kernel_initializer=initl)(a) 
    fe = MaxPooling2D()(fe)
    fe = Flatten()(fe)
    fe = Dense(3)(fe)
    class1 = Activation('sigmoid')(fe)
    c_model = Model(a, class1, name = "c_model")
    c_model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    fe = Dense(1)(fe)
    class2 = Activation('sigmoid')(fe)
    d_model = Model(a, class2, name = "d_model")
    d_model.compile(loss= 'binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    return c_model, d_model

c,d = combined_model()

However, it results in the error:

ValueError: The channel dimension of the inputs should be defined. Found `None`

Upvotes: 2

Views: 1668

Answers (1)

Masoud Maleki
Masoud Maleki

Reputation: 873

Simple answer: Just define input shape as follows and set channel shape explicitly:

a = Input(shape=(100, 100, 3))

And if you want to have multiple inputs (with different channel shapes), so you can simply create multiple inputs:

a = Input(shape=(100, 100, 3))
b = Input(shape=(100, 100, 3)) # or Input(shape=(100, 100, 1))

Also in your sample you are by mistake connected output of one dense layer to another. But based on provided model architecture picture, you want to connect output of flatten layer to each dense layer (You can see implementation details in the following).

Implementation:

In the following code I set input shape correctly and also I passed output of flatten layer to each dense (3 and 1) layer (You wrongly were passed output of one dense to another).

Important note: You want to pass two input images to 'classifier 2' But you can't pass two images directly to a CNN layer, instead you can use separate CNN for each input and then concatenate their results, as shown in the following code:

from keras.layers import Input,Conv2D,Activation,Flatten,Concatenate, MaxPooling2D, Dense, concatenate
from keras.models import Model

def combined_model():
    a = Input(shape=(100, 100, 3))
    b = Input(shape=(100, 100, 3))
    fe = Conv2D(3, (1,1), strides=(2,2), padding='same', kernel_initializer=initl)(a) 
    fe = MaxPooling2D()(fe)
    flattened1 = Flatten()(fe)

    fe = Conv2D(3, (1,1), strides=(2,2), padding='same')(b) 
    fe = MaxPooling2D()(fe)
    flattened2 = Flatten()(fe)

    fe = Dense(3)(flattened1)
    class1 = Activation('sigmoid')(fe)
    c_model = Model(a, class1, name = "c_model")
    c_model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

    fe = Dense(1)(concatenate([flattened1, flattened2]))
    class2 = Activation('sigmoid')(fe)
    d_model = Model([a, b], class2, name = "d_model")
    d_model.compile(loss= 'binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    return c_model, d_model

c,d = combined_model()
from keras.utils import plot_model
plot_model(c, 'c.png')
plot_model(d, 'd.png')

Models architecture (I've shown two models together, because they have shared layers): enter image description here

Upvotes: 3

Related Questions