CODEWITHSUNDEEP
pythontensorflowdeep-learningneural-networktensorflow2.0
LuckyPotatoe
LuckyPotatoe

Reputation: 130

Cascaded Neural Network architecture and input using TensorFlow

I am currently replicating a machine learning architecture found in this paper: https://downloads.hindawi.com/journals/acisc/2018/1439312.pdf

Specifically on page 4 of the paper: The model architecture provided by the paper

Any ideas on how to implement this on TensorFlow? My current model looks like this: TensorFlow Summary

BATCH_SIZE = 32

INPUT_SIZE1 = (15, 30, 3)
INPUT_SIZE2 = (30, 60, 3)
INPUT_SIZE3 = (40 ,80, 3)

LEARNING_RATE = 0.001
EPOCHS = 10

CNN_CR1_INPUT = keras.Input(shape = INPUT_SIZE1)
CNN_CR1 = Conv2D(64, (5, 5), strides=2, padding='same', activation='relu')(CNN_CR1_INPUT)
CNN_CR1 = MaxPooling2D(3,3)(CNN_CR1)
CNN_CR1 = Conv2D(64, (3,3), strides=1, padding='same', activation='relu')(CNN_CR1)
CNN_CR1 = Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(CNN_CR1)
CNN_CR1 = Flatten()(CNN_CR1)
CNN_CR1_OUTPUT = Dense(1)(CNN_CR1)

CNN_CR2_INPUT = keras.Input(shape = INPUT_SIZE2)
CNN_CR2 = Conv2D(64, (5,5), strides=2, padding='same', activation='relu')(CNN_CR2_INPUT)
CNN_CR2 = MaxPooling2D(3, 3)(CNN_CR2)
CNN_CR2 = Conv2D(64, (3,3), strides=1, padding='same', activation='relu')(CNN_CR2)
CNN_CR2 = Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(CNN_CR2)
CNN_CR2 = Flatten()(CNN_CR2)
CNN_CR2_OUTPUT = Dense(1)(CNN_CR2)

CNN_CR3_INPUT = keras.Input(shape = INPUT_SIZE3)
CNN_CR3 = Conv2D(64, (5,5), strides=2, padding='same', activation='relu')(CNN_CR3_INPUT)
CNN_CR3 = MaxPooling2D(3, 3)(CNN_CR3)
CNN_CR3 = Conv2D(64, (3,3), strides=1, padding='same', activation='relu')(CNN_CR3)
CNN_CR3 = Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(CNN_CR3)
CNN_CR3 = Flatten()(CNN_CR3)
CNN_CR3_OUTPUT = Dense(1)(CNN_CR3)

# SUGGESTION: This kinda weird. If this works, we only need 1 valitadation datagen? Not sure how it all connect together.
CNN_MAX = Maximum()([CNN_CR1_OUTPUT, CNN_CR2_OUTPUT, CNN_CR3_OUTPUT])

CNN_MODEL = keras.Model(inputs=[CNN_CR1_INPUT, CNN_CR2_INPUT, CNN_CR3_INPUT], outputs=[CNN_MAX])

I am not sure if the model I make is correct or not and I need some assistance. Also, how do you create the input pipeline for a cascading neural network like this one? I already tried this:

TRAIN_DATAGEN1 = ImageDataGenerator(
    # SUGGESTION: Not sure if this is needed??
    rescale = 1/255.0
)

TRAIN_GENERATOR1 = TRAIN_DATAGEN1.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['train']['images'], '0'),
    target_size = (15, 30),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

TEST_DATAGEN1 = ImageDataGenerator(
    rescale = 1/255.0,
)

TEST_GENERATOR1 = TEST_DATAGEN1.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['test']['images'], '0'),
    target_size = (15, 30),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

# CNN 2
TRAIN_DATAGEN2 = ImageDataGenerator(
    rescale = 1/255.0
)

TRAIN_GENERATOR2 = TRAIN_DATAGEN2.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['train']['images'], '1'),
    target_size = (30, 60),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

TEST_DATAGEN2 = ImageDataGenerator(
    rescale = 1/255.0,
)

TEST_GENERATOR2 = TEST_DATAGEN2.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['test']['images'], '1'),
    target_size = (30, 60),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

# CNN 3
TRAIN_DATAGEN3 = ImageDataGenerator(
    rescale = 1/255.0
)

TRAIN_GENERATOR3 = TRAIN_DATAGEN3.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['train']['images'], '2'),
    target_size = (40 ,80),
    class_mode = 'binary',
    batch_size = BATCH_SIZE
)

TEST_DATAGEN3 = ImageDataGenerator(
    rescale = 1/255.0,
)

TEST_GENERATOR3 = TEST_DATAGEN3.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['test']['images'], '2'),
    target_size = (40 ,80),
    class_mode = 'binary',
    batch_size = BATCH_SIZE
)

But it spits out an error when I try to fit it.

ValueError: Failed to find data adapter that can handle input: (<class 'list'> containing values of types {"<class 'keras.preprocessing.image.DirectoryIterator'>"}), <class 'NoneType'>

After digging through the documentation, I am aware that TensorFlow cannot handle a bunch of ImageDataGenerator together in a list. But I'm not sure how to feed images to the model without ImageDataGenerator.

So, to summarize:

Upvotes: 2

Views: 709

Answers (1)

Kailash S Prem
Kailash S Prem

Reputation: 11

You could try building the model with tf.data - load the datas with keras.utils , for data augmentation you can add layers on to model architecture (tf.keras.layers.Rescaling,tf.keras.layers.RandomRotation etc). There are other augmentation methods that are given in the docs below which are more efficient in utilizing the GPU that ImageDataGenerator

https://www.tensorflow.org/tutorials/images/data_augmentation#data_augmentation_2 https://www.tensorflow.org/api_docs/python/tf/keras/utils https://www.tensorflow.org/guide/keras/preprocessing_layers

Upvotes: 1

Related Questions