CODEWITHSUNDEEP
pythontensorflowkeras
Hawkeye001
Hawkeye001

Reputation: 811

Setting up a CNN network with multi-label classification

I have a set of 100x100 images, and an output array corresponding to the size of the input (i.e. length of 10000), where each element can be an 1 or 0.

I am trying to write a python program using TensorFlow/Keras to train a CNN on this data, however, I am not sure how to setup the layers to handle it, or the type of network to use.

Currently, I am doing the following (based off the TensorFlow tutorials):

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(100, 100)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10000, activation=tf.nn.softmax)
])
model.compile(optimizer=tf.train.AdamOptimizer(), 
          loss='sparse_categorical_crossentropy',
          metrics=['accuracy'])

However, I can't seem to find what type of activation I should be using for the output layer to enable me to have multiple output values? How would I set that up?

Upvotes: 0

Views: 1138

Answers (1)

Amir
Amir

Reputation: 16587

I am not sure how to setup the layers to handle it.

Your code is one way to handle that but as you might read in literature, is not the best one. State-of-the-art models usually use 2D Convolution Neural Networks. E.g:

    img_input = keras.layers.Input(shape=img_shape)
    conv1 = keras.layers.Conv2D(16, 3, activation='relu', padding='same')(img_input)
    pol1 = keras.layers.MaxPooling2D(2)(conv1)
    conv2 = keras.layers.Conv2D(32, 3, activation='relu', padding='same')(pol1)
    pol2 = keras.layers.MaxPooling2D(2)(conv2)
    conv3 = keras.layers.Conv2D(64, 3, activation='relu', padding='same')(pol2)
    pol3 = keras.layers.MaxPooling2D(2)(conv3)
    flatten = keras.layers.Flatten()(pol3)
    dens1 = keras.layers.Dense(512, activation='relu')(flatten)
    dens2 = keras.layers.Dense(512, activation='relu')(dens1)
    drop1 = keras.layers.Dropout(0.2)(dens2)
    output = keras.layers.Dense(10000, activation='softmax')(drop1)

I can't seem to find what type of activation I should be using for the output layer to enable me to have multiple output values

Softmax is a good choice. It squashes a K-dimensional vector of arbitrary real values to a K-dimensional vector of real values, where each entry is in the range (0, 1].

You can pas output of your Softmax to top_k function to extract top k prediction:

softmax_out = tf.nn.softmax(logit)
tf.nn.top_k(softmax_out, k=5, sorted=True)

If you need multi-label classification you should change the above network. Last Activation function will change to sigmoid:

output = keras.layers.Dense(10000, activation='sigmoid')(drop1)

Then use tf.round and tf.where to extract labels:

indices = tf.where(tf.round(output) > 0.5)
final_output = tf.gather(x, indices)

Upvotes: 2

Related Questions