CODEWITHSUNDEEP
pythontensorflowkerasneural-networktext-processing
leensal
leensal

Reputation: 1

Neural Network with several inputs (keras, text classification)

I am new to machine learning and experimented a bit with neural networks and did some research. I am currently trying to make a mini network for fake news detection. My data has several features (statement,speaker,date,topic..), so far I've tried using simply the text of the false and true statements as input for my network and used glove for word embeddings. I tried out the following network:

model = tf.keras.Sequential(
[
    # part 1: word and sequence processing
    tf.keras.layers.Embedding(embeddings_matrix.shape[0], embeddings_matrix.shape[1], weights=[embeddings_matrix], trainable=True),
    tf.keras.layers.Conv1D(128, 5, activation='relu'),
    tf.keras.layers.GlobalMaxPooling1D(),
    tf.keras.layers.Dropout(0.2),
    # part 2: classification
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])

It gives me over 90% training accuracy and around 65% testing accuracy. So now i want to try adding 2 more features, the speaker, which is given as [firstname,lastname] and the topic, which can be just one or several words (e.g vaccines or politics-elections-2016), which i decided to limit/pad to 5 words.

Now i don't know how to combine those features into one model. I don't think using word embeddings on the other features makes sense. Do i have to make 3 different networks and concatenate them into one? can I use both the speaker and topic as inputs for the same network? if i concatenate them, how would i do so (using the already classified output for each as input?) ?

Upvotes: 0

Views: 1296

Answers (1)

Sam H.
Sam H.

Reputation: 4359

I have in the past concatenated different input types like this. You can’t use Sequential anymore, the docs say

A Sequential model is not appropriate when: • Your model has multiple inputs or multiple outputs • Any of your layers has multiple inputs or multiple outputs

In the Keras.functional docs, there is a section called “Models with multiple inputs and outputs” that is almost the exact problem you are asking about.

However, I wrote an answer to say: don’t rule out turning your structured data into text, such as prepending “xxspeaker firstname lastname xxtopic topicname” to the text you currently have. It might not work for your current model, which seems pretty small... but if you were to be using a larger model, or fine-tuning a large LM for the task, like if you were using fast.ai or huggingface, you almost certainly have the capacity to just learn it from text.

Upvotes: 1

Related Questions