CODEWITHSUNDEEP
tensorflowkerasdeep-learninginitializationgreedy
ABIM
ABIM

Reputation: 374

Greedy Initialization in Keras

I'm currently coding up a feed-forward network in Tensorflow and I want to make a custom initializer which initializes each layer using a (externally defined) function of the point with the highest MSE.

Pseudo-code:

 Given any input data being passed to my current layer:

 - Identify datapoint *X* with highest MSE from target
 - Initialize at *f(X)*

Sorry if I posted no pseudocode but I have absolutely no idea how to go about it in Keras (Python not R).

Upvotes: 1

Views: 219

Answers (1)

user11530462
user11530462

Reputation:

Since you want your Weights to be updated with respect to Maximized Loss rather than Minimized Loss (from the comment), it can be achieved by passing -loss, as shown in the code below:

Tensorflow Version 2.x:

loss = tf.keras.losses.MSE()
opt = tf.keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=opt, loss = -loss)

Tensorflow Version 1.x:

loss = tf.reduce_mean(tf.keras.losses.MSE(y_true, y_pred))
trainm = tf.train.GradientDescentOptimizer(0.01).minimize(-loss)

Hope this helps.

Upvotes: 1

Related Questions