CODEWITHSUNDEEP
pythontensorflowdeep-learning
Neutrino
Neutrino

Reputation: 35

Tensorflow does not terminate using batches

I'm new at using tensorflow and have some struggle dealing with it. I try to run a simple classification work using the softmax model similar the MNIS example.

I tried creating batches of my data and put dem into the run method. My first approach was using

sess.run(train_step, feed_dict={x: feature_batch, y_: labels_batch})

which led to the error that tensors can't be put to feed_dict.

After some research, I found that I should use.

feat, lab = sess.run([feature_batch, feature_batch])
sess.run(train_step, feed_dict={x: feat, y_: lab})

After trying it my script won't terminate calculation but does also not print any error.

Has anyone some hints why it is not working?

The hole file looks like:

def input_pipeline(filename='dataset.csv', batch_size=30, num_epochs=None):
    filename_queue = tf.train.string_input_producer([filename], num_epochs=num_epochs, shuffle=True)
    features, labels = read_from_cvs(filename_queue)

    min_after_dequeue = 10000
    capacity = min_after_dequeue + 3 * batch_size
    feature_batch, label_batch = tf.train.shuffle_batch(
        [features, labels], batch_size=batch_size, capacity=capacity,
        min_after_dequeue=min_after_dequeue)
    return feature_batch, label_batch


def tensorflow():
    x = tf.placeholder(tf.float32, [None, num_attributes])
    W = tf.Variable(tf.zeros([num_attributes, num_types]))
    b = tf.Variable(tf.zeros([num_types]))

    y = tf.nn.softmax(tf.matmul(x, W) + b)
    y_ = tf.placeholder(tf.float32, [None, num_types])
    cross_entropy = tf.reduce_mean(
        tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))
    train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

    sess = tf.InteractiveSession()

    tf.global_variables_initializer().run()

    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)

    feature_batch, label_batch = input_pipeline()

    for _ in range(1200):
        feat, lab = sess.run([feature_batch, feature_batch])
        sess.run(train_step, feed_dict={x: feat, y_: lab})

    coord.request_stop()
    coord.join(threads)

    correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
    #print(sess.run(accuracy, feed_dict={x: feature_batch, y_: label_batch}))

Upvotes: 1

Views: 167

Answers (2)

mrry
mrry

Reputation: 126184

I suspect the problem is the order of these two lines:

threads = tf.train.start_queue_runners(coord=coord)

feature_batch, label_batch = input_pipeline()

The call to tf.train.start_queue_runners() will start background threads for all input pipeline stages that have been defined up to that point. The call to input_pipeline() creates two new input pipeline stages (in the calls to tf.train.string_input_producer() and tf.train.shuffle_batch()). This means that the background threads for the two new stages will not be started, and the program will hang.

The solution is to reverse the order of these lines:

feature_batch, label_batch = input_pipeline()

threads = tf.train.start_queue_runners(coord=coord)

Upvotes: 2

Sraw
Sraw

Reputation: 20264

You can directly use tensors in your model definition. For example:

def tensorflow():
    x, y_ = input_pipeline()
    W = tf.Variable(tf.zeros([num_attributes, num_types]))
    b = tf.Variable(tf.zeros([num_types]))

    y = tf.nn.softmax(tf.matmul(x, W) + b)
    cross_entropy = tf.reduce_mean(
        tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))
    train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

    sess = tf.InteractiveSession()

    tf.global_variables_initializer().run()

    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)



    for _ in range(1200):
        sess.run(train_step)

Or you should use placeholder in tf.train.shuffle_batch. For example:

#...omit
features_placeholder = tf.placeholder(...)
labels_placeholder = tf.placeholder(...)
x, y_ = tf.train.shuffle_batch(
        [features_placeholder, labels_placeholder], batch_size=batch_size, capacity=capacity,
        min_after_dequeue=min_after_dequeue)
W = tf.Variable(tf.zeros([num_attributes, num_types]))
b = tf.Variable(tf.zeros([num_types]))
#...omit
for _ in range(1200):
    sess.run(train_step, feed_dict={features_placeholder: ..., labels_placeholder: ...})

Upvotes: 2

Related Questions