CODEWITHSUNDEEP
pythonimagetensorflowconv-neural-network
user12128336
user12128336

Reputation:

How to feed extra data along with image to CNN?

Note: I am using TensorFlow to create my neural network.

Info: I have an image with some extra data which I would like to feed into my neural network (first couple layers of the network are conv layers).

The Problem: The extra data is just a couple of integers that describe the image, not an image, so I wouldn't be able to feed the integers into the network without repeating them until they match the height of the image and append an array of the repeated integers to the side of the image. Doing this would provide a lot of unnecessary data since it is being repeated and is very unelegant.

My Question: Is there a way to add data to the neural network after the conv layer that way the extra data could skip past the conv layers and go directly to the dense layer with the image after the image has already been processed and flatten? If this is not possible would there be a better method of passing extra data alongside an image into a neural network with conv layers?

Upvotes: 6

Views: 934

Answers (1)

Chris K
Chris K

Reputation: 1723

Tensorflow supports deep networks with multiple inputs. Here's a toy example of a network with image and vector input:

from tensorflow.keras.layers import Input, Conv2D, Dense, Activation, Flatten, concatenate
from tensorflow.keras.models import Model

image_input = Input((64,64,3))
x = Conv2D(32, kernel_size=8, strides=4)(image_input)
x = Flatten()(x)
x = Dense(64)(x)

vector_input = Input((10,))
y = Dense(64)(vector_input)

z = concatenate([x, y])
z = Dense(64)(z)
z = Activation('softmax')(z)

model = Model([image_input, vector_input], [z])

Upvotes: 4

Related Questions