CODEWITHSUNDEEP
pythontensorflowmachine-learning
Alexis
Alexis

Reputation: 57

Tensor must be from the same graph as Tensor in LinearClassifier with AdagradDA Optimizer

I have been trying to use tf.optimizer.AdagradDA with a Linear Classifier. Here is my code:

def input_fn():
    features_in = { "random_var": tf.convert_to_tensor(np.random.randn(100)) }
    labels_in = tf.convert_to_tensor(np.random.choice([0, 1], size=(100,), p=[1./2, 1./2]))
    dataset =  tf.data.Dataset.from_tensors((features_in, labels_in))
    dataset.repeat(2)
    dataset.batch(1)

    iterator = dataset.make_one_shot_iterator()
    features, labels = iterator.get_next()
    return features, labels

def main():
    global_step_tensor = tf.Variable(0, trainable=False, name='global_step')

    optimizer = tf.train.AdagradDAOptimizer(learning_rate=0.001, global_step=global_step_tensor)
    columns = [tf.feature_column.numeric_column('random_var')]
    model = tf.estimator.LinearClassifier(optimizer=optimizer, feature_columns=columns)
    model.train(input_fn=input_fn)

When I'm running the code I'm getting one of the following errors:

When I run the same code with a FtrlOptimizer or a AdagradOptimize the code runs properly. So I'm guessing, the issue has to be related with the global_step_tensor instanciation.

Here are the things that I have tried so far:

Upvotes: 1

Views: 472

Answers (1)

Milad Shahidi
Milad Shahidi

Reputation: 713

Caveat: This works in TensorFlow 1.10.0 (and possible later). Apparently it does not work in TensorFlow 1.9.0 (and possibly earlier). See comment below by ZakJ.

You need to change the way you construct your optimizer object as follows:

optimizer = lambda: tf.train.AdagradDAOptimizer(learning_rate=0.001,global_step=tf.train.get_global_step())

1) Notice the lambda. This will create a callable, as opposed to a Tensorflow operation, which is what AdagradDAOptimizer would return on its own (without the lambda).

2) You can use tf.train.get_global_step() directly and don't have to define a global step tensor.

I'm not quite sure why you need to create a callable by adding the lambda in this case. But I guess whenever you want to use an optimizer that needs global_step (or you want to pass it, for example for learning rate decay) you need to set the optimizer to a callable (hence the need for lambda) instead of a Tensorflow operation.

This is also what the learning rate decay example does in the docs here.

This is the complete code I successfully ran on TensorFlow 1.10.0:

import tensorflow as tf
import numpy as np

def input_fn():
    features_in = { "random_var": tf.convert_to_tensor(np.random.randn(100)) }
    labels_in = tf.convert_to_tensor(np.random.choice([0, 1], size=(100,), p=[1./2, 1./2]))
    dataset =  tf.data.Dataset.from_tensors((features_in, labels_in))
    dataset.repeat(2)
    dataset.batch(1)

    iterator = dataset.make_one_shot_iterator()
    features, labels = iterator.get_next()
    return features, labels

def main():
    optimizer = lambda: tf.train.AdagradDAOptimizer(learning_rate=0.001,global_step=tf.train.get_global_step())
    columns = [tf.feature_column.numeric_column('random_var')]
    model = tf.estimator.LinearClassifier(optimizer=optimizer, feature_columns=columns)
    model.train(input_fn=input_fn)

Upvotes: 1

Related Questions