CODEWITHSUNDEEP
python
ISETHS GT
ISETHS GT

Reputation: 69

CNN weights on RNN model

will it be possible to use CNN weights on an RNN model like can you use the weights of a CNN learning what each letter looks like and using the weights from that CNN to translate a whole sentence?

Upvotes: 0

Views: 41

Answers (1)

Gil Pinsky
Gil Pinsky

Reputation: 2493

It is possible. For an RNN you have to define what is the hidden state so you may set it to be the output of the CNN on the previous element in the sequence. There are several models with state-of-the-art performance on sentence translation. One of them is the Transformer which is makes use of convolutions, as in CNNs, and self-attention. However, it doesn't do so in an RNN scheme, as RNNs are slower to train and evaluate due to their sequential nature. Also note that RNNs are inferior (mainly) due to their problem of short-term memory. You may want to read on LSTMs (that became their successor).

Upvotes: 1

Related Questions