CODEWITHSUNDEEP
machine-learningscikit-learnstatisticsclassificationmodeling
Abhishek Arora
Abhishek Arora

Reputation: 994

Is it fair to compare the output of a binary model with a multi-class model?

Let's say we are trying to model a propensity of someone to take the product (e.g a car), and there are only 6 types of car the customer can take (say Car_a, Car_b, Car_c, Car_d, Car_e).

From marketing's perspective, I want to know which customer has a higher likelihood for taking the car (binary classification), and I may also want to know which car the customer is most likely to take (multi-class classification).

Now, my questions is, will the probability Pr_binary be comparable or equal to sum of [Pr(Car_a) + Pr(Car_b) + Pr(Car_c) + Pr(Car_d) + Pr(Car_e)] of the multi-class model?

Upvotes: 1

Views: 256

Answers (1)

carrdelling
carrdelling

Reputation: 1725

No, the expressions you are using are not equivalent.

Assuming (in your example) that there are only 5 types of cars, then the correct way of combining the probabilities would be:

P(buying) = 1 - P(not buying)

where:

P(not buying) = (1 - Pr(Car_a)) * (1 - Pr(Car_b)) * (1 - Pr(Car_c)) * (1 - Pr(Car_d)) * (1 - Pr(Car_d))

That should help you assess if the probabilities are similar in both cases for a specific customer.

Note, however, that this assumes that the output of your multi-class method is a probability and (if you are using a One VS All classifier) that all the individual probabilities are calibrated.

Upvotes: 1

Related Questions