CODEWITHSUNDEEP
pythonmachine-learningkeras
Ciaran O Brien
Ciaran O Brien

Reputation: 384

Keras input_tensor shape for transfer learning

I am running a CNN for classification of medical scans using Keras and transfer learning with imagenet and InceptionV3. I am building the model with some practice data of size X_train = (624, 128, 128, 1) and Y_train = (624, 2).

I am trying to resize the input_tensor to suit the shape of my images (128 x 128 x 1) using the below code. 

input_tensor = Input(shape=(128, 128, 1)) 
base_model = InceptionV3(input_tensor=input_tensor,weights='imagenet',include_top=False)

Doing this I get a value error:

ValueError: Dimension 0 in both shapes must be equal, but are 3 and 32. Shapes 
are [3,3,1,32] and [32,3,3,3]. for 'Assign_753' (op: 'Assign') with input 
shapes: [3,3,1,32], [32,3,3,3]

Is there a way to allow this model to accept my images in their format?

Edit: For what its worth, here is the code to generate the training data.

X = []
Y = []
for subj, subj_slice in slices.items():
    # X.extend([s[:, :, np.newaxis, np.newaxis] for s in slice])
    subj_slice_norm = [((imageArray - np.min(imageArray)) / np.ptp(imageArray)) for imageArray in subj_slice]
    X.extend([s[ :, :, np.newaxis] for s in subj_slice_norm])
    subj_status = labels_df['deadstatus.event'][labels_df['PatientID'] == subj]
    subj_status = np.asanyarray(subj_status)
    #print(subj_status)
    Y.extend([subj_status] * len(subj_slice))

X = np.stack(X, axis=0)
Y = to_categorical(np.stack(Y, axis=0))]

n_samp_train = int(X.shape[0]*0.8)
X_train, Y_train = X[:n_samp_train], Y[:n_samp_train]

Edit2: I think the other alternative would be to take my X which is shape (780, 128, 128, 1), clone each of the 780 images and append two as dummies. Is this possible? Resulting in (780, 128, 128, 3).

Upvotes: 1

Views: 3090

Answers (1)

Kaustubh Olpadkar
Kaustubh Olpadkar

Reputation: 109

We can use the existing keras layers to convert the existing image shape to the expected shape for the pre-trained model rather than using the numpy for replicating channels. As replicating channels before training may consume 3x the memory, but integrating this processing at runtime will save up a lot of memory.

You can proceed this way.

Step 1: Create a Keras Model that converts your input images to the shape that can be fed as the input for the base_model as follows:

from keras.models import Model
from keras.layers import RepeatVector, Input, Reshape

inputs = Input(shape=(128, 128, 1))
reshaped1 = Reshape(target_shape=((128 * 128 * 1,)))(inputs)
repeated = RepeatVector(n=3)(reshaped1)
reshaped2 = Reshape(target_shape=(3, 128, 128))(repeated)
input_model = Model(inputs=inputs, outputs=reshaped2)

Step 2: Define pre-trained model InceptionV3 as follows:

base_model = InceptionV3(input_tensor=input_model.output, weights='imagenet', include_top=False)

Step 3: Combine both the models as follows:

combined_model = Model(inputs=input_model.input, outputs=base_model.output)

The advantage of this method is that the keras model itself will take care of the image processing stuff like channel replication at runtime. Thus, we need not replicate the image channels by ourselves with numpy and the results will be memory efficient.

Upvotes: 1

Related Questions