CODEWITHSUNDEEP
python-3.xtensorflowkerasdeep-learningkeras-layer
Eran
Eran

Reputation: 555

Merge specific input coordinates in Keras

I have a large input vector (1000 features) to a Sequential model. The model is mainly a dense network. I know that features 1-50 are coordinate-wise highly correlated to features 51-100 (1 with 51, 2 with 52 etc.) and I want to take advantage of that.

Is there a way to add a layer to my existing model to reflects that? (joining input 1 and 51 to a neuron, 2 and 52 etc.)

Or maybe the only option is to change the input structure to 50 tensors (of 1x2) and one large vector of 900 features? (I would like to avoid that since it means re-writing my feature preparation code)

Upvotes: 2

Views: 545

Answers (1)

today
today

Reputation: 33460

I think the first dense layer would find out this relationship, of course if you define and train the model properly. However, if you would like to process the first 100 feature separately, one alternative is to use Keras functional API and define two Input layers, one for the first 100 features and another for the rest of 900 features:

input_100 = Input(shape=(100,))
input_900 = Input(shape=(900,))

Now you can process each one separately. For example, you can define two separate Dense layers connected to each one and then merge their outputs:

dense_100 = Dense(50, activation='relu')(input_100)
dense_900 = Dense(200, activation='relu')(input_900)

concat = concatenate([dense_100, dense_900])

# define the rest of your model ...

model = Model(inputs=[input_100, input_900], outputs=[the_outputs_of_model])

Of course, you need to feed the input layers separately. For that you can easily slice the training data:

model.fit([X_train[:,:100], X_train[:,100:]], y_train, ...)

Update: If you specifically want the features 1 and 51, 2 and 52, etc. to have a separate neuron (which, at least, I can't comment on the efficiency of it without experimenting on data), you can use LocallyConnected1D layer with kernel size and no. filters of 1 (i.e. it has the same behavior as applying a separate Dense layer on each two related features):

input_50_2 = Input(shape=(50,2))

local_out = LocallyConnected1D(1, 1, activation='relu')(input_50_2)
local_reshaped = Reshape((50,))(local_out)  # need this for merging since local_out has shape of (None, 50, 1)
# or use the following:
# local_reshaped = Flatten()(local_out)

concat = concatenation([local_reshaped, dense_900])

# define the rest of your model...

X_train_50_2 = np.transpose(X_train[:,:100].reshape((2, 50)))

model.fit([X_train_50_2, X_train[:,100:]], y_train, ...)

Upvotes: 1

Related Questions