Combine output of one model with another

Question

I am currently working on a model that predicts the category of a product given its description. The code looks something like this for the model

deep_inputs = Input(shape=(maxlen,))
embedding_layer = Embedding(vocab_size, 100, weights=[embedding_matrix], trainable=False)(deep_inputs)
LSTM_Layer_1 = LSTM(128)(embedding_layer)
dense_layer_1 = Dense(31, activation='softmax')(LSTM_Layer_1)
L1model = Model(inputs=deep_inputs, outputs=dense_layer_1)

L1model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['acc'])

Now the way the data is, categories are usually hierarchical and I want to encode this information in the model ( not sure if this is correct ).

So you will usually have categories like this:

Category Level 1 > Category Level 2 > Category Level 3 ... and so on. 

The above model shown only predicts level 1. I want to take this prediction along with the original description text as inputs to predict level 2. Can someone help me what the best way to go about doing this is ?

I tried something like this, but I am not sure if this is the right way of combining the two together.

from keras.models import load_model
model = load_model('l1classifier.h5')
Y_predict = model.predict(X_train)
concatenate = Concatenate()
prediction = Input(shape=(Y_predict.shape))
combined = concatenate([prediction, L2model.output])

Answer 1

The extended code illustrating how the naming helps managing training in case of multiple outputs.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np

maxlen=100
vocab_size=10

deep_inputs = keras.Input(shape=(maxlen,),name="textin")
embedding_layer = layers.Embedding(vocab_size, 100)(deep_inputs)
LSTM_Layer_1 = layers.LSTM(128)(embedding_layer)

#Let's add categorical levels...
first_category_level_result = layers.Dense(31, activation='softmax',name="first_category_level")(LSTM_Layer_1)
second_category_level_result = layers.Dense(31, activation='softmax',name="second_category_level")(first_category_level_result)
third_category_level_result = layers.Dense(31, activation='softmax',name="third_category_level")(second_category_level_result)

#Let's configure three outputs according to categorical levels...
L1model = keras.Model(inputs=deep_inputs, outputs=[first_category_level_result,second_category_level_result,third_category_level_result])

L1model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['acc'])

#For evaluation let's plot the structure of the network:
keras.utils.plot_model(L1model,'categorical_classifier_network.png')

#Demonstrate:
test_input_for_network=tf.ones((1,100))

output_of_the_network=L1model(test_input_for_network)

first_category_prediction=np.argmax(output_of_the_network[0].numpy())
second_category_prediction=np.argmax(output_of_the_network[1].numpy())
third_category_prediction=np.argmax(output_of_the_network[2].numpy())

print("According to the AI the product belongs in general level to class ",first_category_prediction, "and there to subcategory ", second_category_prediction, " and in fine-grain level in this category to ", third_category_prediction)

#Training example
X_train=tf.ones((10,100))

y_train_L1=tf.ones((10,31))
y_train_L2=tf.ones((10,31))
y_train_L3=tf.ones((10,31))

history=L1model.fit(
    {"textin":X_train},
    {"first_category_level":y_train_L1,"second_category_level":y_train_L2,"third_category_level":y_train_L3},
    epochs=2,
    batch_size=3,
)

#Demonstrate to verfiy all works fine...
test_input_for_network=tf.ones((1,100))

output_of_the_network=L1model(test_input_for_network)

first_category_prediction=np.argmax(output_of_the_network[0].numpy())
second_category_prediction=np.argmax(output_of_the_network[1].numpy())
third_category_prediction=np.argmax(output_of_the_network[2].numpy())

print("After training according to the AI the product belongs in general level to class ",first_category_prediction, "and there to subcategory ", second_category_prediction, " and in fine-grain level in this category to ", third_category_prediction)

Answer 2

By slightly modifying your code to a form:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np

maxlen=100
vocab_size=10

deep_inputs = keras.Input(shape=(maxlen,))
embedding_layer = layers.Embedding(vocab_size, 100)(deep_inputs)
LSTM_Layer_1 = layers.LSTM(128)(embedding_layer)

#Let's add categorical levels...
first_category_level_result = layers.Dense(31, activation='softmax')(LSTM_Layer_1)
second_category_level_result = layers.Dense(31, activation='softmax')(first_category_level_result)
third_category_level_result = layers.Dense(31, activation='softmax')(second_category_level_result)

#Let's configure three outputs according to categorical levels...
L1model = keras.Model(inputs=deep_inputs, outputs=[first_category_level_result,second_category_level_result,third_category_level_result])

L1model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['acc'])

#For evaluation let's plot the structure of the network:
keras.utils.plot_model(L1model,'categorical_classifier_network.png')

#Demonstrate:
test_input_for_network=tf.ones((1,1))

output_of_the_network=L1model(test_input_for_network)

first_category_prediction=np.argmax(output_of_the_network[0].numpy())
second_category_prediction=np.argmax(output_of_the_network[1].numpy())
third_category_prediction=np.argmax(output_of_the_network[2].numpy())

print("According to the AI the product belongs in general level to class ",first_category_prediction, "and there to subcategory ", second_category_prediction, " and in fine-grain level in this category to ", third_category_prediction)

...you can find the logic how to add several categories for the neural network and how to see the results of categorization. The example code saves also the network structure as a image like:

...where please note the output size of the model is three according number of categories to definition of the model. The output including several categories results is a tensor, which can easily investigated by procedure exemplified in the code.