CODEWITHSUNDEEP
pythontensorflow
Omer Lubin
Omer Lubin

Reputation: 679

Python, Tensorflow - Variables are not getting initialized

Prelude: I am trying to create a model for the CIFAR10 database (Image Recognition), and for some reason the training seemed to literally do nothing. On another thread someone helped me conclude that my weights are not changing, nor getting initialized, which may cause the 'not changing' part of the problem.

I am trying to initialize my weights with the tf.truncated_normal function and my weights with the tf.zeros function. I created a few functions to make my code easier to read:

# Creates a weight variable
def weight(shape):
    initial = tf.truncated_normal(shape, stddev=0.01)
    return tf.Variable(initial)

# Creates a bias variable
def bias(shape):
    initial = tf.zeros(shape=shape)
    return tf.Variable(initial)

# Creates a FCL
def hiddenLayer(prev, prev_size, size):
    W = weight([prev_size, size])
    B = bias([size])
    S = tf.nn.relu(tf.matmul(prev, W) + B)

    return W, B, S

# Creates the output layer (same but with softmax)
def outputLayer(prev, prev_size, size):
    W = weight([prev_size, size])
    B = bias([size])
    S = tf.nn.softmax(tf.matmul(prev, W) + B)

    return W, B, S

But, the variables just didn't get initialized (as I always got the corresponding error when I tried to print the variables before training). So, I thought it may have something to do with the functions and the returning of the variables, so I wrote all that mess in my main function (I think that's a common thing to have(?)):

#   Layer 1
    W1 = tf.Variable(tf.truncated_normal([data_size, l1_size], stddev=0.01))
    B1 = tf.Variable(tf.zeros([l1_size]))
    S1 = tf.nn.relu(tf.matmul(data, W1) + B1)

#   Layer 2
    W2 = tf.Variable(tf.truncated_normal([l1_size, l2_size], stddev=0.01))
    B2 = tf.Variable(tf.zeros([l2_size]))
    S2 = tf.nn.relu(tf.matmul(S1, W2) + B2)

#   Output Layer
    W3 = tf.Variable(tf.truncated_normal([l2_size, output_size], stddev=0.01))
    B3 = tf.Variable(tf.zeros([output_size]))
    result = tf.nn.softmax(tf.matmul(S2, W3) + B3)

But the error keeps showing up:

Traceback (most recent call last):
  File "C:\...\model.py", line 89, in <module>
    main()
  File "C:\...\model.py", line 69, in main
    print(sess.run(B3, feed_dict={data: batch_data, labels: batch_labels}) * 100, "%")
...
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use
uninitialized value Variable_5
...

Keep in mind that this error reads Variable_5, and when I tried to output W3 and not B3, I got the same one but for Variable_4, which makes sense because it's the one created before B3. This means it isn't something about a specific variable, this is for all of them. Am I missing something?

Upvotes: 0

Views: 85

Answers (1)

Ekaba Bisong
Ekaba Bisong

Reputation: 2982

Before executing your variables using a TensorFlow Session object, all the variables must be initialized. The method tf.global_variables_initializer(), initializes all the global trainable variables in the graph that are local to the machine. Whereas, the method tf.local_variables_initializer initializes all the variables shared across a distributed environment. It is good practice to run both. See an example:

with tf.Session() as sess:
    # initialize all variables
    tf.global_variables_initializer().run()
    tf.local_variables_initializer().run()

Upvotes: 1

Related Questions