Train Error with CNN in Tensorflow

Question

I'm trying to set up a CNN to classify images with two possible outputs and I'm using Tensorflow to do this. I followed the tutorials and then moved on to adapting the CNN they had used to my problem, but it hasn't worked out very well.

The first thing I changed was how the images for my problem were loaded in. I have another script that writes the location of all the images along with their expected output (separated by a space). I used some code I found elsewhere to load in the images, which gives out the batches (label_batch and image_batch). However, as this format is different from the tutorial, I don't know how to do the training loop. I've tried a variety of things from taking indexes of the batches to running sess.run() there and tried a bunch of things I found online, but so far nothing has helped.

Sorry if this is something very simple, I'm fairly new to this and just starting to feel my way through.

My Code:

#imports tensorflow
import tensorflow as tf
sess = tf.InteractiveSession()

#weight generation with small amount of noise in normal dist & slight + bias bc ReLU nuerons
def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.1)
  return tf.Variable(initial)

#bias generation with small amount of noise in normal dist & slight + bias bc ReLU nuerons
def bias_variable(shape):
  initial = tf.constant(0.1, shape=shape)
  return tf.Variable(initial)

#creates conv layer with stride 1 and 0 padding
def conv2d(x, W):
  return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

#creates max_pool layer thats 2x2
def max_pool_2x2(x):
  return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')    

#reads image file names and respective labels
def read_labeled_image_list(image_list_file):
    f = open(image_list_file, 'r')
    filenames = []
    labels = []
    for line in f:
    filename, label = line[:-1].split(' ')
    filenames.append(filename)
    labels.append(int(label))
    return filenames, labels

#image name to image 
def read_images_from_disk(input_queue):
    label = input_queue[1]
    file_contents = tf.read_file(str(input_queue[0]))
    example = tf.image.decode_png(file_contents, channels=1)
    return example, label

# Reads paths of images together with their labels
image_list, label_list = read_labeled_image_list("images.txt")

images = tf.convert_to_tensor(image_list)
labels = tf.convert_to_tensor(label_list)

# Makes an input queue
input_queue = tf.train.slice_input_producer([images, labels], shuffle=True)

image, label = read_images_from_disk(input_queue)
image.set_shape([28,28,1])

#Image and Label Batching
image_batch, label_batch = tf.train.batch([image, label],batch_size=50, allow_smaller_final_batch = True)

#placeholder define vars?
x = tf.placeholder(tf.float32, shape=[None, 784])
y_ = tf.placeholder(tf.float32, shape=[None, 2])

#conv layer 1, 5x5 patch with 32 features
W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

#4D tensor, 2 and 3 is w and h, 4th is color channels
x_image = tf.reshape(x, [-1,28,28,1])

#sets up forward for x_image
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

#second conv layer, 64 feature extraction
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

#converts from feature to 1024
W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

#prevents overfitting, disabled during testing (todo: potentially     remove for us, is complex)
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

#softmax layer with 10 outputs
W_fc2 = weight_variable([1024, 2])
b_fc2 = bias_variable([2])
y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)


#training: ADAM optimizer with overfitting help and logging every 100th     iteration
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
sess.run(tf.initialize_all_variables())

for i in range(20000):
  #What do I do here!?!??!
  #imgs, lbls = sess.run([image_batch, label_batch])
  #imgs = image_batch[i]
  #lbls = label_batch[i]
  if i%100 == 0:
    train_accuracy = accuracy.eval(feed_dict={ x:imgs, y_: lbls, keep_prob: 1.0})
    print("step %d, training accuracy %g"%(i, train_accuracy))
  train_step.run(feed_dict={x: imgs, y_: lbls, keep_prob: 0.5})

#prints final accuracy, to be updated
print("test accuracy %g"%accuracy.eval(feed_dict={
    x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))

Update: Tensorflow, train_step feed incorrect I found this and attempted to implement the second answer but I get this error:

W tensorflow/core/framework/op_kernel.cc:936] Unimplemented: File system scheme Tensor("input_producer/Gather not implemented

When I tried the first solution, I got this:

Traceback (most recent call last):

  File "Tester.py", line 77, in <module>
    h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
  File "Tester.py", line 22, in conv2d
     return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
  TypeError: DataType uint8 for attr 'T' not in list of allowed values: float16, float32, float64

Update 2 So I realized x and y have to be equal to image_batch and label_batch respectively and it worked once I casted it to float32 using tf.cast(image_batch, tf.float32). However, now the train line still fails with this printed twice in a row:

 W tensorflow/core/framework/op_kernel.cc:936] Unimplemented: File system scheme Tensor("input_producer/Gather not implemented

Answer 1

cast the inputs of conv2d to tf.float32_ref, tf.float32