CODEWITHSUNDEEP
nlpneural-networkdeep-learningkerasrecurrent-neural-network
Alex
Alex

Reputation: 828

Train some embeddings, keep others fixed

I do sequence classification with Keras, using an RNN and embeddings. My sequences are a bit weird. I have words mixed with special symbols. Words are associated with fixed, pre-trained embeddings, but the special symbol embeddings have to be modified during training.

In an Embedding layer during learning, how can I keep some embeddings fixed while updating others? Is there a way to mask those indices which shouldn't be modified? Or is this a case for a custom Embedding layer?

Upvotes: 1

Views: 345

Answers (1)

Ishamael
Ishamael

Reputation: 12795

I do not believe that this is achievable with the existing Embedding layer. To get around it I would just create a custom layer that builds two embedding layers internally, and only puts the embedding matrix of one of them into the trainable_parameters.

Upvotes: 1

Related Questions