Optimizer with train_on_batch?

Question

Is there any point of using an optimizer like adam if I only use train_on_batch()?

I do the training in a loop manually (for adversarial training) but I want the learning rate to be automatically adjusted.

Answer 1

Even if you only use train_on_batch the same optimizer instance will be reused in tensorflow, so the stats should be correctly tracked.