CODEWITHSUNDEEP
pythontensorflow
jxy
jxy

Reputation: 784

`gradient` gives AttributeError when building a TensorFlow graph

I'm trying to learn how to build a graph in TensorFlow, but got stuck in a seemingly trivial operation. This is what I have,

import tensorflow as tf
def loss(x, y):
  tf.reduce_mean(tf.square(x - y))
xx = tf.random_normal([])
noise = tf.random_normal([])
yy = 3 * xx + 2 + noise
W = tf.get_variable("W", [])
W.assign(5)
b = tf.get_variable("b", [])
b.assign(0)
with tf.GradientTape() as t:
  current_loss = loss(W*xx+b, yy)
dW = t.gradient(current_loss, W)

At this point I got an AttributeError, as follows

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-26d05a240ccc> in <module>()
      1 with tf.GradientTape() as t:
      2   current_loss = loss(W*xx+b, yy)
----> 3 dW = t.gradient(current_loss, W)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/backprop.py in gradient(self, target, sources, output_gradients, unconnected_gradients)
    944         flat_sources,
    945         output_gradients=output_gradients,
--> 946         unconnected_gradients=unconnected_gradients)
    947 
    948     if not self._persistent:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/imperative_grad.py in imperative_grad(tape, target, sources, output_gradients, unconnected_gradients)
     70       sources,
     71       output_gradients,
---> 72       compat.as_str(unconnected_gradients.value))

AttributeError: 'NoneType' object has no attribute '_id'

What am I doing wrong and how do I get the gradient? Thanks in advance.

Upvotes: 1

Views: 3221

Answers (2)

Vlad
Vlad

Reputation: 8585

Your loss() function doesn't return anything. That's why you have AttributeError (because current_loss is None). You should add return statement.

Regarding your comment for the previous answer. GradientTape is used in eager execution so you should add tf.enable_eager_execution() at the beginning of your program. If you want to build in graph mode you should use tf.gradients() or the compute_gradients() method of subclasses of tf.train.Optimizer(e.g. tf.train.GradientDescentOptimizer).

Upvotes: 1

Walfits
Walfits

Reputation: 446

The code above needs to be modified slightly if you don't want to use eager execution. When you want to use TensorFlow not in eager execution mode, you need to have two principal parts:

  1. Building the graph
  2. Running the graph

So I re-wrote the example above following this logic:

import tensorflow as tf
import numpy as np

# Building the graph
xx = tf.constant(np.random.normal(loc=0.0, scale=1.0))
noise = tf.constant(np.random.normal(loc=0.0, scale=1.0))
yy = 3 * xx + 2 + noise

W = tf.Variable(5.0, name="W")
b = tf.Variable(0.0, name="b")

current_loss = tf.reduce_mean(tf.square(tf.scalar_mul(W,xx)+b - yy))
dW = tf.gradients(current_loss, W, name='dW')[0]

# Initialisation operation
init = tf.global_variables_initializer()

# Creating a session and running the graph
with tf.Session() as sess:
    sess.run(init)

    dW_np = sess.run(dW)

print(dW_np)

Upvotes: 2

Related Questions