TensorFlow: How to assign labels to image dataset when creating batches?

Question

Yes, I have searched SO, Reddit, GitHub, Google Plus etc etc. I am running Python 3 with TensorFlow on Windows 10 64 bit. My goal is to read a bunch of images and assign labels to them for training.

I'm trying to turn my list of labels into a usable "object" for sess.run(train_step, feed_dict={imgs:batchX,lbls:batchY}). My images import fine, since right before that I call the function to create batches (code below). In the function, I can successfully create the images numpy array. But, I don't know where to start for assigning my labels.

My labels.txt file is in the format

data/cats/cat (1) copy.png,1
data/cats/cat (2) copy.png,1
data/cats/cat (3) copy.png,1
and so on for about 300 lines

Where data/cats/cat (x) copy.png is the file and 1 is the Class (in this case a Cat). The file is read into a regular array (or list?) called labels_list, with each line being a new element in the array. When I print the labels_list, it shows

['data/cats/cat (1) copy.png,1' 'data/cats/cat (2) copy.png,1'
 'data/cats/cat (3) copy.png,1' 'data/cats/cat (4) copy.png,1'
 'data/cats/cat (5) copy.png,1' 'data/cats/cat (6) copy.png,1'
  (alot more lines of this)
 'data/cats/cat (295) copy.png,1' 'data/cats/cat (296) copy.png,1'
 'data/cats/cat (297) copy.png,1' 'data/cats/cat (298) copy.png,1']

I don't know how to make a usable numpy array for my train_step (code below). I tried googling but most solutions use label lists with integers only, but I need to use the path of the files.

Any help appreciated, thanks :)

Code: (and my GitHub github.com/supamonkey2000/jm-uofa)

import tensorflow as tf
import numpy as np
import os
import sys
import cv2


content = [] # Where images are stored
labels_list = [] # Stores the image labels, still not 100% working


########## File opening function
with open("data/cats/files.txt") as ff:
    for line in ff:
        line = line.rstrip()
        content.append(line)
#################################

########## Labels opening function
with open("data/cats/labels.txt") as fff:
    for linee in fff:
        linee = linee.rstrip()
        labels_list.append(linee)
    labels_list = np.array(labels_list)
###############################


############ Function used to create batches for training
def create_batches(batch_size):
    images1 = [] # Array to hold images within the function
    for img1 in content: # Read the images from content[] in a loop
        thedata = cv2.imread(img1) # Load the image
        thedata = thedata.flatten() # Convert the image to a usable numpy array
        images1.append(thedata) # Append the image to the images1 array
    images1 = np.array(images1) # Convert images1[] to numpy array

    print(labels_list) # Debugging purposes

    while(True):
        for i in range(0,298,10):
            yield(images1[i:i+batch_size],labels_list[i:i+batch_size])
#########################################################


imgs = tf.placeholder(dtype=tf.float32,shape=[None,786432]) # Images placeholder
lbls = tf.placeholder(dtype=tf.float32,shape=[None,10]) # Labels placeholder

W = tf.Variable(tf.zeros([786432,10])) # Weights
b = tf.Variable(tf.zeros([10])) # Biases

y_ = tf.nn.softmax(tf.matmul(imgs,W) + b) # Something complicated

cross_entropy = tf.reduce_mean(-tf.reduce_sum(lbls * tf.log(y_),reduction_indices=[1])) # Cool spacey sounding thing that does cool stuff
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy) # When this is called use the GDO to train the model

sess = tf.InteractiveSession() # Setup the session
tf.global_variables_initializer().run() # Initialize the variables

############################## Training steps for teaching the model
for i in range(10000): # Run for 10,000 steps
    for (batchX,batchY) in create_batches(10): # Call a batch to be used
        sess.run(train_step, feed_dict={imgs:batchX, lbls: batchY}) # Train the model with the batch (THIS IS GIVING ME TONS OF ISSUES)
###################################################################


correct_prediction = tf.equal(tf.argmax(y_,1),tf.argmax(lbls,1)) # Find out if the program tested properly (I think?)
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) # Find the accuracy of the model

print(sess.run(accuracy, feed_dict={imgs:content, lbls:labels_list})) # Print the accuracy of the model !!! imgs:content may be incorrect, must look into it

Answer 1

As far a as I understand you have 2 files.
- data/cats/files.txt Which contains the url to your files
- data/cats/labels.txt Which also contains the url to your files and the corresponding label.

I would suggest to only use the labels file because the url and the label are linked here.

Now carrying on to the labeling.

When you read the labels.txt file already create the output labels. I've put some comments in the code.

import re
import numpy
import cv2

label_map = [] # the map that holds the link between the label and the one_hot_encoded label
file_info = [] # holds all your file locations and the label of the file
#read all the lines. A line should look like this: mycatimage.png,1 
with open('labels.txt') as f:
    rows = [re.split(",", line.rstrip("\n")) for line in f]

for row in rows:
    file_info.append(row)

label_column = [line[1] for line in rows] # line[1] is based on your 'data/cats/cat (1) copy.png,1' example where the label is the second index

unique_labels = list(set(label_column)) # set gives unique values
# now the onehot encoding of the labels which basically means everything 0 except 1
for label in unique_labels:
    output_values = np.zeros(len(unique_labels), dtype=np.int)
    output_values [unique_labels.index(label)] = 1
    label_map.append({'name': label , 'value': output_values })


# Write the found labels to the label file if you want for later use. We will use the label_map variable for now
with open("mylabelfile.txt", 'w+') as lf:
    for label in label_map:
        lf.write(label['name'] + ',' + ','.join(str(x) for x in label['value']) + "\n")  # writes --> Christina,0,0,1,0\n

Now on to the batch function :)

def get_one_hot_encoded_array_for_label(label):
    for existing in label_map:
        if existing['name'] == label:
            return existing['value']

def create_batches(batch_size):
    images1 = [] # Array to hold images within the function
    labels_list = []
    for img1 in file_info: # Read the images from file_info[] in a loop
        image_location = img1[0]
        image_label = img1[1]
        thedata = cv2.imread(image_location ) # Load the image
        thedata = thedata.flatten() # Convert the image to a usable numpy array
        images1.append(thedata) # Append the image to the images1 array
        outputvalues = get_one_hot_encoded_array_for_label(image_label )
        labels_list.append(outputvalues) # where we fill the labels list with the one hot encoded values.
    images1 = np.array(images1) # Convert images1[] to numpy array
    labels_list = np.array(labels_list)
    print(labels_list) # Debugging purposes

    while(True):
        for i in range(0,298,10):
            yield(images1[i:i+batch_size],labels_list[i:i+batch_size])

This should provide the one hot encoded values your batchY. Wrote this based on my own network but haven't tested it with image data. Can you confirm that is works or tell where it breaks? And if something is unclear please ask:)