CODEWITHSUNDEEP
pythonimage-processingkerasconv-neural-network
SCool
SCool

Reputation: 3375

How to add age and gender data to images for a convolutional neural network?

I'm currently working on classifying melanoma images via Keras CNN. The images come with metadata - age, gender and lesion location.

Right now I'm just using the images, and results are OK-ish, but I'd like to see what would happen if I added the metadata to the model.

How exactly can I add age, gender etc data to the images?

Here's a screenshot of the exported Dataframe - in red is what I'm using for the CNN, and I'd like to add the green section: https://i.sstatic.net/2xvde.jpg

If there is a standard method of doing this I'd very much appreciate it if you could share it. Thanks

I've tried to google 'add demographic data to CNN' or 'add gender to CNN' etc and can't find any info.

My current model structure is:

model = Sequential()

    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape))
    model.add(MaxPool2D(pool_size=(2, 2)))
    model.add(BatchNormalization())
    
    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape))
    model.add(MaxPool2D(pool_size=(2, 2)))
    model.add(BatchNormalization())
    

    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape))
    model.add(BatchNormalization())
    
    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape))
    model.add(BatchNormalization())
    
    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu'))
    model.add(BatchNormalization())
    model.add(MaxPool2D(pool_size=(2, 2)))
        
    model.add(Conv2D(32, (3, 3), activation='relu'))
    model.add(BatchNormalization())
    
    model.add(Conv2D(32, (3, 3), activation='relu'))
    model.add(BatchNormalization())
    
    model.add(MaxPool2D(pool_size=(2, 2)))
    
    model.add(Flatten())
    
    model.add(Dense(128, activation='relu'))
    model.add(Dropout(0.2))
    
    model.add(Dense(64, activation='relu'))
    model.add(Dropout(0.2))
    
    model.add(Dense(num_classes, activation='sigmoid'))
    
    adam = Adam(lr=0.001)
    
    model.compile(loss='binary_crossentropy',
                  optimizer=adam,
                  metrics=[km.binary_precision(), km.binary_recall(),km.binary_f1_score()])

Upvotes: 3

Views: 1565

Answers (1)

Romain Thalineau
Romain Thalineau

Reputation: 54

What you describe is actually a multi inputs model. In this case you could consider having the image feeding the CNN, and the metadata feeding a fully connected networks and then merging the outputs of these two sub networks to get your prediction.

Check out the Keras doc about it: https://keras.io/guides/functional_api/#models-with-multiple-inputs-and-outputs

Upvotes: 3

Related Questions