CODEWITHSUNDEEP
tensorflowkerastensorflow2.0tf.keras
spb
spb

Reputation: 175

Tensorflow.Keras Adam Optimizer Instantiation

While training CNN, I found that loss is reduced faster when the optimizer is set as optimizer instance

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), 
       .......

then when the optimizer is set as String (name of optimizer)

`model.compile(optimizer='adam', .......)

Since default learning_rate=0.001, why they are working differently, what is the difference between them?

Upvotes: 0

Views: 737

Answers (1)

Innat
Innat

Reputation: 17219

Technically there should not be any differences. If you follow the adam string parameter in the source file here, you would see

all_classes = {
     'adam': adam_v2.Adam,
      ...
}

This adam_v2.Adam has linked with three implementations placed in three different places, i.e

  1. Source Code tensorflow/python/tpu/tpu_embedding_v2_utils.py
- @tf_export("tpu.experimental.embedding.Adam")
class Adam(_Optimizer):
  1. Source Code tensorflow/python/keras/optimizer_v1.py
class Adam(Optimizer):
  1. Source Code tensorflow/python/keras/optimizer_v2/adam.py
@keras_export('keras.optimizers.Adam')
class Adam(optimizer_v2.OptimizerV2):

Now, check the source code of tf.keras.optimizers.Adam, click view source on GitHub button, you would redirect to here - which is number 3 from above.

Upvotes: 2

Related Questions