CODEWITHSUNDEEP
tensorflowobject-detection
Giacomo Bartoli
Giacomo Bartoli

Reputation: 742

Tensorflow Object Detection: use Adam instead of RMSProp

I'm training a CNN with this [.config file][1]:

rms_prop_optimizer: {
    learning_rate: {
      exponential_decay_learning_rate {
        initial_learning_rate: 0.004
        decay_steps: 800720
        decay_factor: 0.95
      }
    }
   momentum_optimizer_value: 0.9
   decay: 0.9
   epsilon: 1.0
}

}
As you can see there is a rms_prop as optimizer. What if I would like to use Adam? How am I supposed to edit this file?

Upvotes: 6

Views: 7064

Answers (2)

Dennis Chicaiza
Dennis Chicaiza

Reputation: 31

Sorry if I am late but I am researching this field nowadays.

I realized we can check inside this directory for some examples of ADAM optimizers:

Tensorflow/models/research/object_detection/builders/optimizer_builder_tf2_test.py

Here is the optimizer config into my pipeline.config without scheduling:

optimizer {
    adam_optimizer: {
      learning_rate: {
        constant_learning_rate {
          learning_rate: 0.0001
        }
      }
    }
    use_moving_average: false
  }

Upvotes: 0

Giovanni Cavallin
Giovanni Cavallin

Reputation: 159

if I'm right, you're trying to use the object_detection model with a pre-trained network offered by Tensorflow, am I right? Then, if you know a little of programming, you can take a look at models/research/object_detection/builders/optimizer_builder.py and see which are the optimizer that can be used and with which parameters. Instead if you just want a out-of-the-box solution, this is how I did:

optimizer {
    # momentum_optimizer {
    adam_optimizer: {
      learning_rate: {
        manual_step_learning_rate {
          initial_learning_rate: .0002
          schedule {
            step: 4500
            learning_rate: .0001
          }
          schedule {
            step: 7000
            learning_rate: .00008
          }
          schedule {
            step: 10000
            learning_rate: .00004
          }
        }
      }
      # momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }

In my (little) experience I noticed that using the same learning_experience as momentum_optimizer makes the learning too fast and/or brings to NaN Losses, so I usually decrease it of 10 times or more. I'm trying just now. :)

Upvotes: 12

Related Questions