CODEWITHSUNDEEP
pythonvariablestensorflowgpu
Yanghoon
Yanghoon

Reputation: 572

What's the usage of variable_scope with name_or_scope = None

When we use variable_scope, we usually set the first argument (name_or_scope) as a name of variable_scope. And when we set the reuse variable as 'True', we can share the variable within the variable_scope. However, I found that inside the tensorflow API (such as Bahdanau attention, or tf.layers.Dense), there are variable scope whose first argument (name_or_scope) is set as None and the second argument (default_name) is set as what we thought as scope name. I checked the code of tf.variable_scope and found that:

If name_or_scope is not None, it is used as is. If scope is None, then default_name is used. In that case, if the same name has been previously used in the same scope, it will be made unique by appending _N to it.

In this case, I can't use it within for loop because each time the variable_scope is called, it will create new variable_scope with different name.

I can't find anywhere explaining this function (I mean name_or_scope set as None).

Anyone to explain this?

Upvotes: 1

Views: 774

Answers (1)

Maxim
Maxim

Reputation: 53778

You're right, when tf.variable_scope is called with name_or_scope=None, a default_name argument is used and it's uniquified. So calling it in a loop will create different scopes.

Not sure about Bahdanau attention, but tf.layers.Dense can be created with a custom scope via _scope argument:

layer = Dense(units, ..., _scope=name)

In fact, you can call tf.layers.dense with a specified name and this name is going to define tf.variable_scope(name). This way allows you to create dense layers in a loop.

Upvotes: 1

Related Questions