CODEWITHSUNDEEP
pythonkerasneural-networklstm
Ciaran O Brien
Ciaran O Brien

Reputation: 384

Creating a LSTM Neural Network outputting two Y variables

I have a classification task based on Tweets. The Tweets are my only input (X) variable. I have two target (y) variables however, one target variable should output 1 or 0 for either positive or negative, while the other target variable should output 1 or 0 for either political or non-political.

I created a LSTM neural network which is running, but I can't seem to get it to output two target variables. Can somebody please advise?

My input shapes are:

X_train        y_train
(15203, 250) (15203, 2)
X_val         y_val
(3801, 250) (3801, 2)

My model is:

    model = Sequential()
    model.add(Embedding(MAX_NB_WORDS, EMBEDDING_DIM, input_length=X.shape[1]))
    model.add(SpatialDropout1D(0.2))
    model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
    
    # For two label classification
    model.add(Dense(2, activation='softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

I noticed I was not getting two target variables when I later ran this piece of code on unseen data

    new_text = ["Tony Gonzales (@TonyGonzales4TX) will be a GREAT Congressman for Texas! A Navy veteran, he is Strong on the Economy, Life and the Second Amendment. We need him to defeat the Radical Left in November. Tony has my Complete and Total Endorsement! #TX23"]
    seq = tokenizer.texts_to_sequences(new_text)
    padded = pad_sequences(seq, maxlen = 250)
    pred = model.predict(padded)
    labels = [0,1]
    print(pred, labels[np.argmax(pred)])

On this and all other tests, I noticed my predictions were only returning a binary classification. e.g. on the above I got 0.13 0.87 (these figures add up to 100 so are tied together)

However, in the above case, a positive political Tweet, I would have expected a dual result of circa 0.88 0.91

Any help would on how to get two output y variables would be much appreciated.

Upvotes: 3

Views: 802

Answers (1)

devspartan
devspartan

Reputation: 622

You can add 4 nodes in your output layer representing (negative, positive, political, non-political) and map you ytrain in that manner or you can try this:

x = your input numpy array
y1 = your sentiment one hot encoded output numpy array
y2 = your political one hot encoded output numpy array
# x, y1, y2 should have same length or shape[0]
data = tensorflow.data.Dataset.from_tensor_slices((x, [y1, y2]))
data = data.shuffle(len(x))

input =  Input(shape=(X.shape[1], ))
x = Embedding(MAX_NB_WORDS, EMBEDDING_DIM, input_length=X.shape[1])(input)
x = SpatialDropout1D(0.2)(x)
x = LSTM(100, dropout=0.2, recurrent_dropout=0.2)(x)
out1 = Dense(2, activation='softmax', name='sentiment')(x)
out2 = Dense(2, activation='softmax', name='political')(x)
model = Model(inputs = input, outputs=[out1, out2])
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

model.fit(data, other arguments....]  #supposing your output is one hot encoded

enter image description here

See if it works or not.

Upvotes: 4

Related Questions