CODEWITHSUNDEEP
tensorflow2.0
Ohad Meir
Ohad Meir

Reputation: 714

Is there a replacement for global_step in TensorFlow 2.0?

It seems like global_step is missing from TensorFlow 2.0.

I have several callbacks that are interested in current training progress and I'm not sure if I need to implement my own step counter or depend on epochs count instead...

Any recommendations for replacement?

Upvotes: 5

Views: 3652

Answers (2)

Rob Hall
Rob Hall

Reputation: 2823

Working in TensorFlow 2.3.1 and its Keras API.

Instances of tf.keras.optimizers.Optimizer inherit an iterations property. The implementation shows that this is a counter which is incremented after each training step. Access it from the optimizer before compiling the model.

import tensorflow as tf

tf.compat.v1.disable_eager_execution()  # see note

optimizer = tf.keras.optimizers.Adam()
training_step = optimizer.iterations

model = Model(inputs,outputs)
model.compile(
    loss=my_adaptive_loss_function(training_step),
    optimizer=optimizer)

Note: In my setup, I had to disable eager execution in order to use this variable otherwise I received the following TypeError. You may be able to avoid this if your implementation is less kludgy than mine.

TypeError: An op outside of the function building code is being passed
a "Graph" tensor. It is possible to have Graph tensors
leak out of the function building context by including a
tf.init_scope in your function building code.
For example, the following function will fail:
  @tf.function
  def has_init_scope():
    my_constant = tf.constant(1.)
    with tf.init_scope():
      added = my_constant * 2
The graph tensor has name: pulse_features:0

During handling of the above exception, another exception occurred:

Upvotes: -1

nessuno
nessuno

Reputation: 27042

Right now is better to declare our own global_step = tf.Variable(1, name="global_step") and use it manually.

Looking at the documentation there is not a drop-in replacement for tf.train.get_or_create_global_step and the only part of the documentation that is about a step is the experimental section of the tf.summary module: https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/summary/experimental

Upvotes: 4

Related Questions