CODEWITHSUNDEEP
cntk
OlavT
OlavT

Reputation: 2666

How to get labels from minibatch?

I'm working on this tutorial:

https://github.com/Microsoft/CNTK/blob/master/Tutorials/CNTK_201B_CIFAR-10_ImageHandsOn.ipynb

The test / train data files are simple tab separated text files containing image filenames and correct labels like this:

...\data\CIFAR-10\test\00000.png    3
...\data\CIFAR-10\test\00001.png    8
...\data\CIFAR-10\test\00002.png    8

How can I extract the original labels from a minibatch?

I have tried with this code:

reader_test = MinibatchSource(ImageDeserializer('test_map.txt', StreamDefs(
    features = StreamDef(field='image', transforms=transforms), # first column in map file is referred to as 'image'
    labels   = StreamDef(field='label', shape=num_classes)      # and second as 'label'
)))

test_minibatch = reader_test.next_minibatch(10)
labels_stream_info = reader_test['labels']
orig_label = test_minibatch[labels_stream_info].value
print(orig_label)

<cntk.cntk_py.Value; proxy of <Swig Object of type 'CNTK::ValuePtr *' at 0x0000000007A32C00> >

But, as you see above the results are not an array with the labels.

What is the correct code to get to the labels?

This code works, but then it uses a different file format and not the ImageDeserializer.

File format:

|labels 0 0 1 0 0 0 |features 0
|labels 1 0 0 0 0 0 |features 457

Working code:

mb_source = text_format_minibatch_source('test_map2.txt', [
    StreamConfiguration('features', 1),
    StreamConfiguration('labels', num_classes)])

test_minibatch = mb_source.next_minibatch(2)

labels_stream_info = mb_source['labels']
orig_label = test_minibatch[labels_stream_info].value
print(orig_label)

[[[ 0.  0.  1.  0.  0.  0.]]
 [[ 1.  0.  0.  0.  0.  0.]]]

How can I get to the labels in the input when using the ImageDeserializer?

Upvotes: 4

Views: 767

Answers (2)

Anton Schwaighofer
Anton Schwaighofer

Reputation: 3149

I just tried to repro - I think there is some strange bug lurking here. My hunch is that in fact the labels object is not returned as a valid numpy array. I inserted the following debug output into the train_and_evaluate function in the tutorial CNTK_201B:

for epoch in range(max_epochs):       # loop over epochs
    sample_count = 0
    while sample_count < epoch_size:  # loop over minibatches in the epoch
        data = reader_train.next_minibatch(min(minibatch_size, epoch_size - sample_count), input_map=input_map) # fetch minibatch.
        print("Features:")
        print(data[input_var].shape)
        print(data[input_var].value.shape)
        print("Labels:")
        print(data[label_var].shape)
        print(data[label_var].value.shape)

That outputs:

Training 116906 parameters in 10 parameter tensors.
Features:
(64, 1, 3, 32, 32)
(64, 1, 3, 32, 32)
Labels:
(64, 1, 10)
()

The labels come out as what appears to be a numpy.ndarray, but it does not have a valid shape.

I'd call that a bug.

Upvotes: 1

Morgan Funtowicz
Morgan Funtowicz

Reputation: 85

Can you try using :

orig_label = test_minibatch[labels_stream_info].value

Upvotes: 2

Related Questions