CODEWITHSUNDEEP
pythonmachine-learningkerasneural-network
zicxor
zicxor

Reputation: 57

ValueError: Layer weight shape (43, 100) not compatible with provided weight shape (412457, 400)

I prepared a small data set for the project. And it gives

ValueError: Layer weight shape (43, 100) not compatible with provided weight shape (412457, 400)

error. I think there is a problem with tokenizers.

X and Y for train_test_split

X = []
sentences = list(titles["title"])
for sen in sentences:
    X.append(preprocess_text(sen))

y = titles['Unnamed: 1']



X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)

Tokenizer here

tokenizer = Tokenizer(num_words=5000)
tokenizer.fit_on_texts(X_train)

X_train = tokenizer.texts_to_sequences(X_train)
X_test = tokenizer.texts_to_sequences(X_test)

vocab_size = len(tokenizer.word_index) + 1 #vocab_size 43

maxlen = 100

X_train = pad_sequences(X_train, padding='post', maxlen=maxlen)
X_test = pad_sequences(X_test, padding='post', maxlen=maxlen)

So, my pretrained word2vec model has (412457, 400) shape.

from numpy import array
from numpy import asarray
from numpy import zeros

from gensim.models import KeyedVectors
embeddings_dictionary = KeyedVectors.load_word2vec_format('drive/My Drive/trmodel', binary=True)

I used my pretrained word2vec model instead of GloVe. (vocab_size: 43, 100, weights from embeddings_dictionary.vectors)

from keras.layers.recurrent import LSTM

model = Sequential()
embedding_layer = Embedding(vocab_size, 100, weights=[embeddings_dictionary.vectors], input_length=maxlen , trainable=False)
model.add(embedding_layer)
model.add(LSTM(128))

model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])

ValueError: Layer weight shape (43, 100) not compatible with provided weight shape (412457, 400)

Upvotes: 1

Views: 3720

Answers (1)

Geeocode
Geeocode

Reputation: 5797

If you want to use pretrained weights, then you have to pass the approprite size parameters to the Embedding layer so, that it can assign the pretrained weight matrix to the Embedding layer's weights matrix.

Thus you have to make the following:

embedding_layer = Embedding(412457, 400, weights=[embeddings_dictionary.vectors], input_length=maxlen , trainable=False)

Before the training you have to change padding conforming it to the Embedding layer:

maxlen = 400

X_train = pad_sequences(X_train, padding='post', maxlen=maxlen)
X_test = pad_sequences(X_test, padding='post', maxlen=maxlen)

Upvotes: 2

Related Questions