CODEWITHSUNDEEP
pythonpandasnumpykeras
user1595443
user1595443

Reputation: 227

Which Keras output layer / loss function to use in the following scenario

I got the following data sample:

[1,2,1,4,5],[1,2,1,4,5],[0,2,7,0,1] with a label of [1,0,1] 
....
[1,9,1,4,5],[1,5,1,4,5],[0,7,7,0,1] with a label of [0,1,1]

I can't train it on a single series of [1,2,1,4,5] with a label of 1 or 0, as the whole row got a meaningful context information to it, so the whole 15 input digits should be inferred together.

It's not your typical classification, and it doesn't seem as a regression issue either. Also, the data is not related to imagery, it's taken from a scientific domain.

Obviously I am feeding the data as a flat 15 input node to the net

model = Sequential(
    [
      Dense(units=16,input_shape = scaled_train_samples[0].shape,activation='relu'),                
      Dense(units=32,activation='relu'),
      Dense(units=3,activation='???'),
    ])

Which activation output function would be ideal in such case?

Upvotes: 1

Views: 50

Answers (1)

theastronomist
theastronomist

Reputation: 1056

I would recommend having 3 outputs to the network. Since the data can affect the 3 "sub-labels", the network only branches apart on the classification layer. If you want, you can add more layers to each specific branch.

I'm assuming that each "sub-label" is binary classification, so that's why I chose sigmoid (returns value from 0 to 1, so larger number means network thinks it's class 1 over class 0)

To do this, you would have to change to the Functional API like this:

from keras.layers import Input, Dense
from keras.models import Model

visible = Input(shape=(scaled_train_samples[0].shape))
model = Dense(16, input_shape = activation='relu')(visible)
model = Dense(32,activation='relu')(model)
model = Dense(16,activation='relu')(model)
out1 = Dense(units=1,activation='sigmoid',name='OUT1')(model)
out2 = Dense(units=1,activation='sigmoid',name='OUT2')(model)
out3 = Dense(units=1,activation='sigmoid',name='OUT3')(model)

finalModel = Model(inputs=visible outputs=[out1, out2, out3])

optimizer = Adam(learning_rate=.0001)
losses = {
    'OUT1': 'binary_crossentropy',
    'OUT2': 'binary_crossentropy',
    'OUT3': 'binary_crossentropy',
    }
model.compile(optimizer=optimizer, loss=losses, metrics={'OUT1':'accuracy', 'OUT2':'accuracy', 'OUT3':'accuracy'})

Upvotes: 1

Related Questions