tensorflow input pipeline returns multiple values

Question

I'm trying to make an input pipeline in tensorflow for image classification, therefore I want to make batches of images and corresponding labels. The Tensorflow document suggests that we can use tf.train.batch to make batches of inputs:

train_batch, train_label_batch = tf.train.batch(
[train_image, train_image_label],
batch_size=batch_size,
num_threads=1,
capacity=10*batch_size,
enqueue_many=False,
shapes=[[224,224,3], [len(labels),]],
allow_smaller_final_batch=True
)

However, I'm thinking would it be a problem if I feed in the graph like this:

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=train_label_batch, logits=Model(train_batch)))

The question is does the operation in the cost function dequeues images and their corresponding labels, or it returns them separately? Therefore causing the training with wrong images and labels.

Answer 1

There are several things you need to consider to preserve the ordering of images and labels.

let's say we need a function that gives us images and labels.

def _get_test_images(_train=False):


"""
Gets the test images and labels as a batch

Inputs:
======
_train      : Boolean if images are from training set
random_crop     : Boolean if random cropping is allowed
random_flip         : Boolean if random horizontal flip is allowed
distortion      : Boolean if distortions are allowed

Outputs:
========
images_batch    : Batch of images containing BATCH_SIZE images at a time
label_batch     : Batch of labels corresponding to the images in images_batch
idx         : Batch of indexes of images
"""

#get images and labels
_,_img_names,_img_class,index= _get_list(_train = _train)

#total number of distinct images used for train will be equal to the images
#fed in tf.train.slice_input_producer as _img_names

img_path,label,idx = tf.train.slice_input_producer([_img_names,_img_class,index],shuffle=False)

img_path,label,idx = tf.convert_to_tensor(img_path),tf.convert_to_tensor(label),tf.convert_to_tensor(idx)
img_path = tf.cast(img_path,dtype=tf.string)

#read file 
image_file = tf.read_file(img_path)

#decode jpeg/png/bmp
#tf.image.decode_image won't give shape out. So it will give error while resizing
image = tf.image.decode_jpeg(image_file)

#image preprocessing
image = tf.image.resize_images(image, [IMG_DIM,IMG_DIM])

float_image = tf.cast(image,dtype=tf.float32)

#subtracting mean and divide by standard deviation
float_image = tf.image.per_image_standardization(float_image)

#set the shape
float_image.set_shape(IMG_SIZE)
labels_original = tf.cast(label,dtype=tf.int32)
img_index = tf.cast(idx,dtype=tf.int32)

#parameters for shuffle
batch_size = BATCH_SIZE
min_fraction_of_examples_in_queue = 0.3
num_preprocess_threads = 1
num_examples_per_epoch = MAX_TEST_EXAMPLE
min_queue_examples = int(num_examples_per_epoch *
                       min_fraction_of_examples_in_queue)

images_batch, label_batch,idx = tf.train.batch(
        [float_image,label,img_index],
        batch_size=batch_size,
        num_threads=num_preprocess_threads,
        capacity=min_queue_examples + 3 * batch_size)

# Display the training images in the visualizer.
tf.summary.image('images', images_batch)

return images_batch, label_batch,idx

Here,tf.train.slice_input_producer([_img_names,_img_class,index],shuffle=False) is an interesting thing to look at where if you put shuffle=True it will shuffle all three arrays in coordination.

Second thing is, num_preprocess_threads. As long as you are using single threads for dequeue operation, batches will come out in a deterministic way. But more than one threads will shuffle the arrays randomly. for example for image 0001.jpg if True label is 1 you might get 2 or 4. Once its dequeue it is in tensor form. tf.nn.softmax_cross_entropy_with_logits shouldn't have problem with such tensors.