CODEWITHSUNDEEP
variablesscopetensorflow
cpu stat
cpu stat

Reputation: 23

Tensor Flow variable scope:

Why do we use tf.variable_scope:

I know the basic idea that its something link creating an instance. But in the code below and many other codes the scope is not retrieved anywhere by using tf.get_variable. What purpose the scope is serving then? What will happen if I dont use it?

with tf.variable_scope("placeholder"):
    input = tf.placeholder(tf.float32, shape=[None, 1024])
    y_true = tf.placeholder(tf.int32, shape=[None, 1])
with tf.variable_scope('FullyConnected'):
    w = tf.get_variable('w', shape=[1024, 1024], 
initializer=tf.random_normal_initializer(stddev=1e-1))
    b = tf.get_variable('b', shape=[1024], 
initializer=tf.constant_initializer(0.1))
    z = tf.matmul(input, w) + b
    y = tf.nn.relu(z)

I do not see the scope being used anywhere later. What purpose is it serving. I want to know what happens if I remove it

Upvotes: 1

Views: 288

Answers (2)

David Parks
David Parks

Reputation: 32111

When you visualize that graph in tensorboard you'll have an "aha" moment.

When you debug some annoying problem and need to know what tensor is causing this or that error message you'll have another "aha" moment.

Do you absolutely need to do either of these things? No, not really. You'll probably still survive. But why not make your future life easier?

I use variable scopes extensively in summary statistics because they group things in tensorboard nicely. I don't usually do it to quite the extent shown by the OP, but it really doesn't hurt to do so.

Upvotes: 1

user2489252
user2489252

Reputation:

One of reasons why we use variable scopes is that it is useful to summarize ops in Tensorboard. For instance, consider the following example:

g = tf.Graph()

with g.as_default() as g:

    tf_x = tf.Variable([[1., 2.], 
                        [3., 4.],
                        [5., 6.]], 
                       name='tf_x_0',
                       dtype=tf.float32)

    tf_y = tf.Variable([[7., 8.], 
                        [9., 10.],
                        [11., 12.]], 
                       name='tf_y_0',
                       dtype=tf.float32)

    # add custom name scope
    with tf.name_scope('addition'):
        output = tf_x + tf_y

    # add custom name scope
    with tf.name_scope('matrix_multiplication'):
        output = tf.matmul(tf.transpose(tf_x), output)

with tf.Session(graph=g) as sess:
    sess.run(tf.global_variables_initializer())
    file_writer = tf.summary.FileWriter(logdir='logs/2', graph=g)
    result = sess.run(output)
    print(result)

enter image description here

Upvotes: 1

Related Questions