tensorflow random shuffle queue: insufficient elements

Question

I am testing a model by training it on different images. I've got different folder with a different number of images in them. When I change from a folder with only 20 images in it, to one with 100, or 10'000 images in it, the program crashes straight away complaining that the random shuffle queue has insufficient number of elements.

I did not alter anything else in the code, simply pointing it to different directories,which definitely are not empty. Do anyone have an idea to why this happens, and how to solve it?

Error Message

tensorflow/core/kernels/queue_base.cc:294] _0_READ_DATA/input_producer: Skipping cancelled enqueue attempt with queue not closed
Traceback (most recent call last):
  File "test_vgg19.py", line 102, in <module>
    images, labels = sess.run([imb_batch1,label_batch1])
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 766, in run
    run_metadata_ptr)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 964, in _run
    feed_dict_string, options, run_metadata)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
    target_list, options, run_metadata)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_3_READ_DATA/shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 30, current size 0)
     [[Node: READ_DATA/shuffle_batch = QueueDequeueMany[_class=["loc:@READ_DATA/shuffle_batch/random_shuffle_queue"], component_types=[DT_FLOAT, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](READ_DATA/shuffle_batch/random_shuffle_queue, READ_DATA/shuffle_batch/n)]]

Caused by op u'READ_DATA/shuffle_batch', defined at:
  File "test_vgg19.py", line 56, in <module>
    imb_batch1,label_batch1 = input_pipeline()
  File "test_vgg19.py", line 53, in input_pipeline
    min_after_dequeue=min_after_dequeue)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 917, in shuffle_batch
    dequeued = queue.dequeue_many(batch_size, name=name)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1099, in _queue_dequeue_many
    timeout_ms=timeout_ms, name=name)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
    op_def=op_def)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/ssoderli/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1128, in __init__
    self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_READ_DATA/shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 30, current size 0)
     [[Node: READ_DATA/shuffle_batch = QueueDequeueMany[_class=["loc:@READ_DATA/shuffle_batch/random_shuffle_queue"], component_types=[DT_FLOAT, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](READ_DATA/shuffle_batch/random_shuffle_queue, READ_DATA/shuffle_batch/n)]]

Input pipeline - Code Snippet

def read_my_file_format(filename_queue):
    reader = tf.WholeFileReader()
    key, record_string = reader.read(filename_queue)
    example = tf.image.decode_png(record_string)
    return example, key


def input_pipeline( bsize=30, num_epochs=None):
    filename_queue = tf.train.string_input_producer(
        tf.train.match_filenames_once("/home/20images/*.png"), num_epochs=num_epochs, shuffle=True)
    example, label = read_my_file_format(filename_queue)
    min_after_dequeue = bsize
    capacity = min_after_dequeue + 3 * 8
    example_batch, label_batch =  tf.train.shuffle_batch(
        [example, label], batch_size=bsize, capacity=capacity,
        min_after_dequeue=min_after_dequeue)
    return  example_batch, label_batch

EDIT

Turned out that some images where of different sizes, which caused the problem. Though I'm curious to how that error message is produced. I don't see how images of different sized could produce that error.

Answer 1

It turned out that there were some images of larger dimensions in the folders for which the errors occurred. Removing them and it worked fine.