CODEWITHSUNDEEP
machine-learningtensorflownlpdeep-learning
voxter
voxter

Reputation: 925

Which deep learning model can classify categories which are not mutually exclusive

Examples : I have a sentence in job description : "Java senior engineer in UK ".

I want to use a deep learning model to predict it as 2 categories : English and IT jobs. If i use traditional classification model, it only can predict 1 label with softmax function at last layer . Thus, i can use 2 model neural networks to predict "Yes"/"No" with both categories, but if we have more categories, it is too expensive . So do we have any deeplearning or machine learning model to predict 2 or more categories at same time ?

"Edit" : With 3 labels by traditional approach , it will be encoded by [1,0,0] but in my case, it will be encoded by [1,1,0] or [1,1,1]

Example : if we have 3 labels, and a sentence may be fit with all of these labels. So if output from softmax function is [0.45 , 0.35 , 0.2 ] we should classify it into 3 labels or 2 labels , or may be one ?

The main problem when we do it is : what is good threshold to classify into 1, or 2 , or 3 labels ?

Upvotes: 2

Views: 980

Answers (1)

interjay
interjay

Reputation: 110164

If you have n different categories which can be true at the same time, have n outputs in your output layer with a sigmoid activation function. This will give each output a value between 0 and 1 independently.

Your loss function should be the mean of the negative log likelihood of the outputs. In tensorflow, this is:

linear_output = ...  # the output layer before applying activation function
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(
    logits=linear_output, labels=correct_outputs))
output = tf.sigmoid(linear_output)  # 0 to 1 for each category

Upvotes: 3

Related Questions