CODEWITHSUNDEEP
tensorflow
MindSeeker
MindSeeker

Reputation: 600

tf.train.shuffle_batch() ValueError: Cannot infer Tensor's rank: Tensor("PyFunc:0", dtype=uint8)

I am trying to feed my image data from my TFRecord files into tf.train.shuffle_batch(). I have a load_img_file() function that reads the TFRecord files, does preprocessing, and returns the images and one-hot labels in the format [[array of images, np.uint8 format], [array of labels, np.uint8 format]]. I made the op

load_img_file_op = tf.py_func(self.load_img_file, [], [np.uint8, np.uint8])

which converts that function into an op. I have verified that that op works by doing

data = tf.Session().run(load_img_file_op)
for n in range(50): #go through images
    print data[1][n] #print one-hot label
    self.image_set.display_img(data[0][n]) #display image

which successfully prints the one-hot labels and displays the corresponding images.

However, when I try to do something like

self.batch = tf.train.shuffle_batch(load_img_file_op, batch_size=self.batch_size, capacity=q_capacity, min_after_dequeue=10000)

I get the error

raise ValueError("Cannot infer Tensor's rank: %s" % tl[i]) ValueError: Cannot infer Tensor's rank: Tensor("PyFunc:0", dtype=uint8)"

I have tried many variations to try to match what the guide does:

Nothing seems to work, but I feel like I am following the format of the guide and various other tutorials I have seen. What am I doing wrong? I apologize if my mistake is stupid or trivial (searches for this error do not seem to return anything relevant to me). Thank you for your help and time!

Upvotes: 0

Views: 1338

Answers (1)

MindSeeker
MindSeeker

Reputation: 600

The link in the comments helped a lot; thank you! (The answer is that you have to give the shape when using py_func.) Since I had to figure out a little bit more on top of that I will post the complete solution:

I had to make my function return two separate values so that they would be two different tensors and could be shaped separately:

return images, labels

Then, proceeding as in the question above, but shaping:

load_img_file_op = tf.py_func(self.load_img_file, [], [np.uint8, np.uint8]) # turn the function into an op
images, labels = load_img_file_op
images.set_shape([imgs_per_file, height * width])
labels.set_shape([imgs_per_file, num_classes])
self.batch = tf.train.shuffle_batch([images, labels], batch_size=self.batch_size, capacity=q_capacity, min_after_dequeue=1000, enqueue_many = True)

The enqueue_many is important so that the images will enter the queue individually.

Upvotes: 3

Related Questions