CODEWITHSUNDEEP
pythonkeras
BigBoy1337
BigBoy1337

Reputation: 4963

Why does this python generator have no output according to keras?

EDIT: updating all the code to organize this question, same issue and question though. 

def extract_hypercolumn(model, layer_indexes, instance):
    layers = [model.layers[li].output for li in layer_indexes]
    get_feature = K.function([model.layers[0].input],layers)
    assert instance.shape == (1,3,224,224)
    feature_maps = get_feature([instance])
    hypercolumns = []
    for convmap in feature_maps:
        for fmap in convmap[0]:
            upscaled = sp.misc.imresize(fmap, size=(224, 224),
                                        mode="F", interp='bilinear')
            hypercolumns.append(upscaled)

    return np.asarray(hypercolumns)

def get_arrays(each_file):
    img = color.rgb2lab(io.imread(each_file)[..., :3])
    X = img[:,:,:1]
    y = img[:,:,1:]
    X_rows,X_columns,X_channels=X.shape
    y_rows,y_columns,y_channels=y.shape
    X_channels_first = np.transpose(X,(2,0,1))
    X_sample = np.expand_dims(X_channels_first,axis=0)
    X_3d = np.tile(X_sample,(1,3,1,1))
    hc = extract_hypercolumn(model,[3,8],X_3d)
    hc_expand_dims = np.expand_dims(hc,axis=0)
    y_reshaped = np.reshape(y,(y_rows*y_columns,y_channels))
    classed_pixels_first = KNN.predict_proba(y_reshaped)
    classed_classes_first = np.transpose(classed_pixels_first,(1,0))
    classed_expand_dims = np.expand_dims(classed_classes_first,axis=0)
    print "hypercolumn shape: ",hc_expand_dims.shape,"classified output color shape: ",classed_expand_dims.shape
    return hc_expand_dims,classed_expand_dims


def generate_batch():
    files = glob.glob('../manga-resized/sliced/*.png')
    while True:
        random.shuffle(files)
        for fl in files:
            yield get_arrays(fl)

colorize = Colorize()
colorize.compile(optimizer=sgd,loss='categorical_crossentropy',metrics=["accuracy"])


colorize.fit_generator(generate_batch(),samples_per_epoch=1,nb_epoch=5)

Here is the traceback (using Tensorflow) :

    Using TensorFlow backend.
output shape:  (None, 112, 228, 228)
output_shape after reshaped:  (None, 112, 51984)
Epoch 1/5
Traceback (most recent call last):
  File "load.py", line 152, in <module>
    colorize.fit_generator(generate_batch(),samples_per_epoch=1,nb_epoch=5)
  File "/Users/alex/anaconda2/lib/python2.7/site-packages/keras/models.py", line 651, in fit_generator
    max_q_size=max_q_size)
  File "/Users/alex/anaconda2/lib/python2.7/site-packages/keras/engine/training.py", line 1358, in fit_generator
    'or (x, y). Found: ' + str(generator_output))
Exception: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/alex/anaconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/Users/alex/anaconda2/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/alex/anaconda2/lib/python2.7/site-packages/keras/engine/training.py", line 404, in data_generator_task
    generator_output = next(generator)
StopIteration

And using theano - note that here hyper column and the classed labels are successfully printed - it seems like this is closer to working:

UPDATE: it works using theano! Im satisfied. however, the question still stands with tensor flow I guess

Now, when I try: 

for a, b in generate_batch(): print(a, b)

or 

print list(islice(generate_batch(), 3))

EDIT: New development - They work!

This works perfectly at least it prints out numpy arrays rather than erring out. However, the Keras issue remains

This makes me wonder if I am simply running into a limitation of keras - since there is so much preprocessing of the data - feeding the image into VGG, extracting the hyper columns, performing the KNN classification on the labels, etc. The fit generator is trying to get batches but doing a ton of work to do it. Perhaps it is too much so it just sees the return value as empty because it is taking so much memory/bandwidth.

I know tensor flow for instance has an entire queueing system built out for this exact issue. It would be awesome to know if this is what Im experiencing - as opposed to implementation error. Any keras experts out there care to weight in??? :)

Upvotes: 2

Views: 3939

Answers (2)

ColdSprintMountain
ColdSprintMountain

Reputation: 1

I met exactly the same problem with theano backend. I probe the problem by finding that when I increase the "max_q_size" more, this bug came earlier. That means it is a queue problem and related with the enqueue operation!!!

In fact, in my case,a missing "while True" in the batch_generator incur this bug:when training in an epoch approaching the point that all the trainning samples available in your generator were all loaded in the queue, then the generator will have to enqueue "None" to the queue as the "next_sample" and the fit_generator will finally met this "None" and report the bug you mentioned.

Upvotes: 0

sytrus
sytrus

Reputation: 777

generator should be infinite (loops through data) in fit_generator.

c.f. keras documentation on fit_generator

The generator is expected to loop over its data indefinitely.

Try changing your function generate_batch to:

def generate_batch():
    files = glob.glob('../manga-resized/sliced/*.png')
    while True:
        random.shuffle(files)
        for fl in files:
            yield get_arrays(fl)

Also:

I think the problem of your code comes from the line 

y_reshaped = (y,(y_rows*y_columns,y_channels))

This line doesn't seem to do the reshaping at all. It just create a tuple with 2 elements: the numpy array y and the tuple (y_rows*y_columns,y_channels).

I suppose you should write something like

y_reshaped = np.reshape(y,(y_rows*y_columns,y_channels))

Upvotes: 3

Related Questions