CODEWITHSUNDEEP
pythontensorflowkerasneural-network
Victor H. De Oliveira Côrtes
Victor H. De Oliveira Côrtes

Reputation: 168

How to use tfa.losses.TripletSemiHardLoss() with softplus?

I want to train a tensoflow neural network using triplet loss and a softplus function as used in article "In Defense of the Triplet Loss for Person Re-Identification" (2017). I found loss function How to use tfa.losses.TripletSemiHardLoss() and the function tf.nn.softplus(), but I'm not able to use them together. The network I want to train is:

model1 = tf.keras.Sequential([
    tf.keras.layers.Conv2D(filters=64, kernel_size=2, padding='same', activation='relu', input_shape=(28,28,1)),
    tf.keras.layers.MaxPooling2D(pool_size=2),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(pool_size=2),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(256, activation=None), # No activation on final dense layer
    tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1)) # L2 normalize embeddings
])´

I know how to use only the loss function tfa.losses.TripletSemiHardLoss() as follows

model1.compile(
    optimizer=tf.keras.optimizers.Adam(0.001),
    loss=tfa.losses.TripletSemiHardLoss())

But I don't know how to use it with tf.nn.softplus().

Upvotes: 0

Views: 289

Answers (2)

Victor H. De Oliveira Côrtes
Victor H. De Oliveira Côrtes

Reputation: 168

Loss function tfa.losses.TripletSemiHardLoss() has a parameter soft. So to use softplus function, you just need to do

model1.compile(
    optimizer=tf.keras.optimizers.Adam(0.001),
    loss=tfa.losses.TripletSemiHardLoss(soft=True))

Upvotes: 0

Nicolas Gervais
Nicolas Gervais

Reputation: 36634

Why don't you just add the activation in your model as the last layer?

model1 = tf.keras.Sequential([
    tf.keras.layers.Conv2D(filters=64, kernel_size=2, padding='same', 
                           activation='relu', input_shape=(28,28,1)),
    tf.keras.layers.MaxPooling2D(pool_size=2),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(pool_size=2),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(256, activation=None), # No activation on final dense layer
    tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1)) # L2 normalize embeddings,
    tf.keras.layers.Activation('softplus')
])

Upvotes: 1

Related Questions