Extract Keras concatenated layer of 3 embedding layers, but it's an empty list

Question

I am constructing a Keras Classification model with Multiple Inputs (3 actually) to predict one single output. Specifically, my 3 inputs are:

1. Actors
2. Plot Summary
3. Relevant Movie Features

Output:

1. Genre tags

Python Code (create the multiple input keras)

def kera_multy_classification_model():

    sentenceLength_actors = 15
    vocab_size_frequent_words_actors = 20001

    sentenceLength_plot = 23
    vocab_size_frequent_words_plot = 17501

    sentenceLength_features = 69
    vocab_size_frequent_words_features = 20001

    model = keras.Sequential(name='Multy-Input Keras Classification model')

    actors = keras.Input(shape=(sentenceLength_actors,), name='actors_input')
    plot = keras.Input(shape=(sentenceLength_plot,), name='plot_input')
    features = keras.Input(shape=(sentenceLength_features,), name='features_input')

    emb1 = layers.Embedding(input_dim = vocab_size_frequent_words_actors + 1,
                            # based on keras documentation input_dim: int > 0. Size of the vocabulary, i.e. maximum integer index + 1.
                            output_dim = Keras_Configurations_model1.EMB_DIMENSIONS,
                            # int >= 0. Dimension of the dense embedding
                            embeddings_initializer = 'uniform', 
                            # Initializer for the embeddings matrix.
                            mask_zero = False,
                            input_length = sentenceLength_actors,
                            name="actors_embedding_layer")(actors)
    encoded_layer1 = layers.LSTM(100)(emb1)

    emb2 = layers.Embedding(input_dim = vocab_size_frequent_words_plot + 1,
                            output_dim = Keras_Configurations_model2.EMB_DIMENSIONS,
                            embeddings_initializer = 'uniform',
                            mask_zero = False,
                            input_length = sentenceLength_plot,
                            name="plot_embedding_layer")(plot)
    encoded_layer2 = layers.LSTM(100)(emb2)

    emb3 = layers.Embedding(input_dim = vocab_size_frequent_words_features + 1,
                            output_dim = Keras_Configurations_model3.EMB_DIMENSIONS,
                            embeddings_initializer = 'uniform',
                            mask_zero = False,
                            input_length = sentenceLength_features,
                            name="features_embedding_layer")(features)
    encoded_layer3 = layers.LSTM(100)(emb3)

    merged = layers.concatenate([encoded_layer1, encoded_layer2, encoded_layer3])

    layer_1 = layers.Dense(Keras_Configurations_model1.BATCH_SIZE, activation='relu')(merged)

    output_layer = layers.Dense(Keras_Configurations_model1.TARGET_LABELS, activation='softmax')(layer_1)

    model = keras.Model(inputs=[actors, plot, features], outputs=output_layer)

    print(model.output_shape)

    print(model.summary())

    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['sparse_categorical_accuracy'])

Model's Structure

My problem:

After successfully fitting and training the model on some training data, I would like to extract the embeddings of this model for later use. My main approach before using a multiple input keras model, was to train 3 different keras models and extract 3 different embedding layers of shape 100. Now that I have the multiple input keras model, I want to extract the concatenated embedding layer with output shape (None, 300).

Although, when I try to use this python command:

embeddings = model_4.layers[9].get_weights()
print(embeddings)

or

embeddings = model_4.layers[9].get_weights()[0]
print(embeddings)

I get either an empty list (1st code sample) either an IndenError: list index out of range (2nd code sample).

Thank you in advance for any advice or help on this matter. Feel free to ask on the comments any additional information that I may have missed, to make this question more complete.

Note: Python code and model's structure have been also presented to this previously answered question

Answer 1

Concatenate layer does not have any weights (it does not have trainable parameter as you ca see from your model summary) hence your get_weights() output is coming empty. Concatenation is an operation.
For your case you can get weights of your individual embedding layers after training.

model.layers[3].get_weights() # similarly for layer 4 and 5

Alternatively if you want to store your embedding in (None, 300) you can use numpy to concatenate weights.

out_concat = np.concatenate([mdoel.layers[3].get_weights()[0], mdoel.layers[4].get_weights()[0], mdoel.layers[5].get_weights()[0]], axis=-1)

Although you can get output tensor of concatenate layer:

out_tensor = model.layers[9].output
# <tf.Tensor 'concatenate_3_1/concat:0' shape=(?, 300) dtype=float32>